Testing

About

testing group members 2017The Testing Research Group is one of the largest groups of its kind in the UK developing innovative approaches to software testing and quality assurance. Drawing equally on theory and practice, our aim it to produce practical software engineering solutions for industry, improving the productivity of developers and testers, and improving the reliability of software systems. The group investigates a number of areas, including model-based testing, search-based testing, security testing, reverse engineering, model-driven engineering, multi-agent modelling and XML data processing.

Crossing the boundaries between the recovery and verification of designs, to the generation of economical test-suites that exercise software completely, our research has created innovative tools for industry, including: EvoSuite, IGUANA, JWalk, Broker@Cloud, HOLTestGen (test generation), StateChum, SUMO (reverse engineering) and FLAME (agent-based modelling). Our work is funded by the EU, EPSRC, InnovateUK and industry.

Phil McMinn (Head of Testing Group) - Search-based and Interface Testing
Kirill Bogdanov - Reverse Engineering
Achim Brucker - Security Testing
John Derrick - Property-based Testing
Siobhán North - Evaluation of Algorithms
Anthony Simons - Model-Driven Engineering and Testing

Group members

Testing Research Group Members

Academic Staff

Dr Kirill Bogdanov
Dr Achim Brucker
Prof John Derrick (Acting VP and Head of Faculty of Science)
Dr Phil McMinn (Head of Group)
Dr Siobhán North
Dr Anthony Simons

Affiliated Academics

Prof Graham Birtwistle (University of Calgary)
Eur Ing Dr Anthony J Cowling (Honorary)
Dr Dimitris Dranidis (International Faculty)
Dr George Eleftherakis (International Faculty)
Dr Gordon Fraser ( University of Passau)
Prof W Michael L Holcombe (Honorary)
Prof Petros Kefalas (International Faculty)
Dr Mariam Kiran (University of Bradford)
Dr Pietro S Oliveto (EPSRC Early Career Fellow)
Dr Ioanna Stamatopoulou (International Faculty)
Dr Neil Walkinshaw (University of Leicester)

Research Staff

Dr Mathew Hall
Dr Jose Miguel Rojas Siles

Affiliated Researchers

Dr Michael Croucher (EPSRC Research Software Engineering Fellow)
Dr Raluca Lefticaru (University of Bradford)
Dr Paul Richmond (EPSRC Research Software Engineering Fellow)
Dr Ramsay Taylor

PhD Students

Mr Ibrahim Al thomali (Dr McMinn and Dr Simons)
Mr Nasser Albunian (Dr Sudholt, Dr Simons and Dr Fraser)
Ms Sadeen Alharbi (Dr Simons and Prof Green)
Mr Abdullah Faisal Alsharif (Dr McMinn and Dr Sudholt)
Ms Eidah Alzahrani (Dr Simons and Dr Richmond)
Mr Fesal Baxhaku (Dr Eleftherakis and Dr Vasilaki)
Mr Konstantinos Bratanis (Dr Dranidis and Dr Simons)
Ms Ermira Daka (Dr North)
Mr Thomas Gyeera (Dr Simons and Dr Stannett)
Mr Michael Herzberg (Dr Brucker and Dr Simons)
Mr Jose Carlos Medeiros de Campos (Dr McMinn)
Ms Marina Ntika (Dr Stannett and Dr Stamatopoulou)
Mr David Paterson (Dr McMinn)
Ms Krenare Pireva (Dr Cowling and Prof Kefalas)
Mr Michal Soucha (Dr Bogdanov and Prof Struth)
Mr Ben Clegg (Dr McMinn)
Mr Thomas Walsh (Dr McMinn and Dr Bogdanov)
Mr Thomas White (Dr McMinn)

Recent Alumni

Dr Shtwai Alsubai (PhD 2018)

Dr Hanaa Al Zadjali (PhD 2017)
Dr Sina Shamshiri (PhD 2017)
Dr Dimitrios Kourtesis (PhD 2017)

Mr Konstantions Rousis (MPhil 2016)
Mr Samer AlKhazraji (MPhil 2016)
Dr Abdullah Alsaeedi (PhD 2016)
Dr Rustem Dautov (PhD 2016)
Dr Fotios Gonidis (PhD 2016)

Dr Alaa Almelibari (PhD 2015)
Dr Ahmad Subahi (PhD 2015)
Dr Christopher Wright (PhD 2015)

Dr Othlapile Dinakenyane (PhD 2014)
Ms Isidora Petreska (MPhil 2014)
Dr Joseph Vella (PhD 2014)

Dr Sheeva Afshan (PhD 2013)
Dr Norah Farooqi (PhD 2013)
Dr Mathew Hall (PhD 2013)
Dr Ognen Paunovski (PhD 2013)
Dr Ervin Ramollari (PhD 2013)

Research

Research Themes

Model-based Testing

This formal testing method generates complete functional test suites from a model specification expressed as an extended finite state machine; and offers guarantees of correctness once testing is complete. The work has been used at Daimler for testing automotive systems designed using Harel Statecharts, applied to testing the gate-logic on low-power ARM chips, used for unit testing Java and also used to test software services for SAP and SingularLogic in the Cloud. Several software testing tools have grown out of this work, including StateTest, JWalk and Broker@Cloud.

Leaders: Dr Simons, Dr Bogdanov

Search-based Testing

Search-based testing uses a problem-specific fitness function to guide a search process (such as a genetic algorithm) to automatically generate test cases. Fitness is judged by instrumenting the tested code to measure what is covered by the tests. The approach has been applied to large C programs and also to object-oriented programs in Java. Several software tools have grown out of this work, including EvoSuite for Java, and IGUANA for C.

Leaders: Dr McMinn, Dr Fraser

Property-based Testing

Property-based testing provides a high-level approach to testing and is a complementary technique to the similar model-based testing. Rather than focusing on individual test cases to encapsulate the behaviour of a system, in property-based testing this behaviour is specified by properties, expressed in a logical form. The system is then tested by checking whether it has the required properties for randomly generated data, which may be inputs to functions, sequences of API calls, or other representations of test cases. This work has resulted in a code coverage tool (Smother) and a mutation testing tool (Mu2) for the Erlang language.

Leaders: Prof Derrick, Dr Bogdanov, Dr Taylor

Security Testing

Security encompasses information security, software engineering, security engineering, and formal methods. Our research in this area investigates all security aspects of distributed and service-oriented systems. This includes applied security aspects, such as access control or business-process modelling, as well as fundamental aspects, such as novel static and dynamic approaches for ensuring the security of applications. We participate in the development of interactive theorem proving environments for Z (HOL-Z) and UML/OCL (HOL-OCL, which is integrated into a formal MDE tool-chain) and a model-based test-case generator (HOL-TestGen).

Leaders: Professor Clark, Dr Brucker

Model-Driven Engineering

The aim of this approach is to create systems not by writing program code, but rather by generating it from high-level abstract models that are closer to end-user requirements. What initial models and languages should be chosen? How should models be checked? How should they be folded together to create more detailed system specifications? How should the transformation rules be verified? Our work so far has investigated dependently-typed languages as a means of verifying model transformations, has generated simple information systems from requirements and has also generated platform-specific test suites for SOAP or REST-based software services in the Cloud.

Leaders: Dr Simons, Dr Brucker, Prof Derrick

Reverse Engineering

This novel approach recovers specifications from legacy software systems. The reverse engineering method collects traces of the system's execution and performs grammar inference on the traces, to detect behavioural regularity. From this, finite state models are constructed, which allow further hypotheses about the specification to be generated and tested. The approach has been applied to recover both flat and nested state specifications, and used for the supervised re-modularisation of software systems. Software tools include StateChum (for reverse engineering) and SUMO (for supervised re-modularisation).

Leaders: Dr Bogdanov, Dr Hall, Dr Taylor

XML Data Processing

The rise of distributed data processing in the Cloud has led to a resurgence of non-relational key-value and tree-structured data formats like XML. This has opened up new research areas in XML data compression, distribution and storage, with associated issues of data indexing and query-processing. We have developed algorithms for fast searching in distributed XML databases using sparse binary matrix indexing; and for trust-based access control to XML data with dynamic learning. The technology has been applied in Botswana for the distributed mobile phone hosting of compressed XML databases.

Leaders: Dr North

Software Engineering

Why do software development projects frequently fail, and what can be done about it? This strand of our research has investigated both traditional and agile software development methods, to see how they work in practice. We have published internationally-famous critiques of the UML notation and associated development process, and also of the Agile method known as XP (eXtreme Programming). We have collaborated with work psychologists to monitor the behaviour and effectiveness of developer teams in our own software company Genesys Solutions, revealing how the constitution of teams affects how well they work.

Leaders: Dr Cowling, Dr Simons

Publications

Academic Articles

Here you can find research publications for the Testing Research Group, listed by academic.  The head link navigates to the official web page for the relevant academic (with highlighted favourite publications).  The remaining links navigate to their DBLP author page, their Google Scholar citations page and optionally a self-maintained publications page.

Academic Staff
Dr Kirill E Bogdanov DBLP GoogleScholar Web
Dr Achim D Brucker DBLP GoogleScholar Web
Prof John Derrick DBLP GoogleScholar
Dr Philip S McMinn DBLP GoogleScholar Web
Dr Siobhán D North DBLP GoogleScholar Web
Dr Anthony J H Simons DBLP GoogleScholar Web
Affiliated Academics
Prof Graham Birtwistle DBLP GoogleScholar
Eur Ing Dr Anthony J Cowling DBLP GoogleScholar
Dr Dimitris Dranidis DBLP GoogleScholar Web
Dr George Eleftherakis DBLP GoogleScholar Web
Dr Gordon Fraser DBLP GoogleScholar Web
Prof W Michael L Holcombe DBLP GoogleScholar
Prof Petros Kefalas DBLP GoogleScholar Web
Dr Mariam Kiran DBLP GoogleScholar Web
Dr Pietro S Oliveto DBLP GoogleScholar Web
Dr Ioanna Stamatopoulou DBLP GoogleScholar Web
Dr Neil Walkinshaw DBLP GoogleScholar Web
Research Staff
Dr Mathew Hall DBLP GoogleScholar
Dr Jose Miguel Rojas Siles DBLP GoogleScholar Web
Affiliated Researchers
Dr Mike Croucher GoogleScholar
Dr Raluca-Elena Lefticaru DBLP GoogleScholar Web
Dr Paul Richmond DBLP GoogleScholar Web
Dr Ramsay Taylor DBLP GoogleScholar

 Research Theses

Here you can find recently-published PhD (and MPhil) theses, which have been deposited in the White Rose eTheses Online repository.  Follow links to the abstract, and then to the full thesis (if public) or to a request form (if a time-embargo restriction has been placed on public release).

Recently Published Theses
Dr Shtwai Alsubai 2018 Child Prime Label Approaches to Evaluate XML Structured Queries
Dr Hanaa Al Zadjali 2017 Compressing Labels of Dynamic XML Data using Base-9 Scheme and Fibonacci Encoding
Dr Sina Shamshiri 2017 Automated Unit Testing of Evolving Software
Dr Dimitrios Kourtesis 2017 Policy-Driven Governance in Cloud Service Ecosystems
Mr Samer Al Khazraji 2016 A Labelling Technique Comparison for Indexing Large XML Database
Mr Konstantinos Rousis 2016 Disciplined Exploitation of Emergent Properties
Dr Abdullah Alsaeedi 2016 Improving Software Model Inference by Combining State Merging and Markov Models
Dr Rustem Dautov 2016 EXCLAIM Framework: a Monitoring and Analysis Framework to Support Self-Governance in Cloud Application Platforms
Dr Fotios Gonidis 2016 A Framework Enabling the Cross-Platform Development of Service-based Cloud Applications
Dr Christopher Wright 2016 Mutation Analysis of Relational Database Schemas
Dr Alaa Almelibari 2015 Labelling Dynamic XML Documents: a Group Based Approach
Dr Ahmad Subahi 2015 A Business User Model-Driven Engineering Method for Developing Information Systems
Dr Othlapile Dinakenyane 2014 SIQXC: Schema Independent Queryable XML Compression for Smartphones
Ms Isidora Petreska 2014 Biologically Inspired Agents: A Framework for Formal Modelling and Simulation of Agents with Elementary Spatial Attributes
Dr Joseph Vella 2014 An Object-Oriented Data and Query Model
Dr Sheeva Afshan 2013 Search-Based Generation of Human Readable Test Data and Its Impact on Human Oracle Costs
Dr Norah Farooqi 2013 Applying Dynamic Trust Based Access Control to Improve XML Databases Security
Dr Mathew Hall 2013 Improving Software Remodularisation
Dr Ognen Paunovski 2013 Exploring Emergent Phenomena: Towards Analysis and Synthesis of Emergent Formations in Complex Systems
Dr Ervin Ramollari 2013 Automated Verification and Testing of Third-Party Web Services

Last updated 30/01/2018

Seminars

 Seminars

Welcome to the Testing Research Group seminar series!  This is a public-facing forum which gives details of forthcoming seminars by members of the group and by our research visitors.  After the seminar has taken place, the same thread may also contain the slides that were presented and other ancillary data.

This forum may be viewed by the general public, but only authorised members of the Testing Research Group may post new messages.  Messages emailed to the testing-seminar-group@sheffield.ac.uk alias will appear in the newsfeed below, organised by thread. Authorised members can also interact with the newsfeed directly to post new messages or extend existing threads with new content.