Research

My research focusses on developing automated techniques for software testing, to help developers maintain "healthy" test suites that find bugs.

Recently my work has centred on helping developers detect and mitigate "flaky" software tests. I have also worked on mutation analysis approaches to help developers assess the quality of their test suites, including finding areas of code that while executed by tests may only be pseudo-tested. Other past areas of work have included detecting and repairing presentation failures and automatically generating tests using search-based test generation.

My work has been funded by the EPSRC and Meta. I am currently an associate editor for the Software Testing, Verification and Reliability journal, and I lead Sheffield's Testing Research Group.

Team

November 2024

• Owain Parry (Post-Doctoral Research Associate); working on the Test FLARE project (Test FLakiness Automated REproduction), EPSRC grant EP/X024539/1.
• Islam Elgendy (Teaching Assistant & Part-Time PhD Student); working on Diversifying Test Suites for Better Bug Finding.
• Zalán Lévai (PhD Student); working on Mutation Analysis for the Rust Programming Language.
• Megan Maton (PhD Student); working on Identifying Pseudo-Tested Code.
• Firhard Roslan (PhD Student — writing up); working on Test Suite Health.

We are part of the Testing Research Group at the University of Sheffield, one of the largest software testing groups in the UK.

I have previously supervised nine PhD students to completion as first supervisor.

PhD Opportunities

Publications

See my full list of my publications, also recorded on my Google Scholar and DBLP profiles.

Some of my papers that have recently appeared include:

@inproceedings{Gruber2024,
  author    = "Gruber, Martin and Roslan, Muhammad Firhard and Parry, Owain and Scharnb{\"o}ck,
               Fabian and McMinn, Phil and Fraser, Gordon",
  title     = "Do Automatic Test Generation Tools Generate Flaky Tests?",
  booktitle = "International Conference on Software Engineering (ICSE)",
  year      = "2024"
}

@inproceedings{Maton2024,
  author    = "Maton, Megan and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "Exploring Pseudo-Testedness: Empirically Evaluating Extreme Mutation Testing at the
               Statement Level",
  booktitle = "International Conference on Software Maintenance and Evolution (ICSME)",
  year      = "2024"
}

@inproceedings{Roslan2024,
  author    = "Roslan, Muhammad Firhard and Rojas, Jos{\'e} Miguel and McMinn, Phil",
  title     = "Private --- Keep Out? Understanding How Developers Account for Code Visibility in
               Unit Testing",
  booktitle = "International Conference on Software Maintenance and Evolution (ICSME)",
  year      = "2024"
}

Teaching

I teach the first-year COM1001 Introduction to Software Engineering module. Key to the module is team-based software development, writing automated tests, and improving code design through refactoring.

I also teach the third-year COM3529 Software Testing and Analysis module. Topics include how to write effective, maintainable tests, and how to analyse their quality. The module also covers more research-oriented topics such as automatically generating test cases (e.g., through fuzzing and search-based techniques), mutation analysis, and methods for test case priortisation.

I also supervise a range of third year dissertation projects on topics in software engineering, software testing, and on those provided by local industry partners.