Research

My research focusses on developing automated techniques for software testing, to help developers maintain "healthy" test suites that find bugs.

Recently my work has centred on helping developers detect and mitigate "flaky" software tests. I have also worked on mutation analysis approaches to help developers assess the quality of their test suites, including finding areas of code that while executed by tests may only be pseudo-tested. Other past areas of work have included detecting and repairing presentation failures and automatically generating tests using search-based test generation.

My work has been funded by the EPSRC and Meta. I am currently an associate editor for the Software Testing, Verification and Reliability journal, and I lead Sheffield's Testing Research Group.

Team

November 2024

• Owain Parry (Post-Doctoral Research Associate); working on the Test FLARE project (Test FLakiness Automated REproduction), EPSRC grant EP/X024539/1.
• Islam Elgendy (Teaching Assistant & Part-Time PhD Student); working on Diversifying Test Suites for Better Bug Finding.
• Zalán Lévai (PhD Student); working on Mutation Analysis for the Rust Programming Language.
• Megan Maton (PhD Student); working on Identifying Pseudo-Tested Code.
• Olek Osikowicz (PhD Student, as second supervisor — first supervisor Donghwan Shin); working on Testing Automatic Driving Systems.

We are part of the Testing Research Group at the University of Sheffield, one of the largest software testing groups in the UK.

I have previously supervised ten PhD students to completion as first supervisor.

Publications

See my full list of my publications, also recorded on my Google Scholar and DBLP profiles.

Some of my papers that have recently appeared include:

@inproceedings{Parry2025b,
  author    = "Parry, Owain and McMinn, Phil",
  title     = "{QAOA-PCA}: Enhancing Efficiency in the Quantum Approximate Optimization Algorithm
               via Principal Component Analysis",
  booktitle = "International Workshop on Empirical Studies for Quantum Software Engineering (E-QSE)",
  year      = "2025"
}

@inproceedings{Parry2025,
  author    = "Parry, Owain and Hilton, Michael and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "Systemic Flakiness: An Empirical Analysis of Co-Occurring Flaky Test Failures",
  booktitle = "International Conference on Evaluation and Assessment in Software Engineering (EASE)",
  year      = "2025"
}

@inproceedings{Elgendy2025b,
  author    = "Elgendy, Islam T. and Hierons, Robert M. and McMinn, Phil",
  title     = "Empirically Evaluating the Use of Bytecode for Diversity-Based Test Case
               Prioritisation",
  booktitle = "International Conference on Evaluation and Assessment in Software Engineering (EASE)",
  year      = "2025"
}

@inproceedings{Osikowicz2025,
  author    = "Osikowicz, Aleksander M. and McMinn, Phil and Shin, Donghwan",
  title     = "Empirically Evaluating Flaky Tests for Autonomous Driving Systems in Simulated
               Environments",
  booktitle = "Flaky Tests Workshop (FTW)",
  year      = "2025"
}

@inproceedings{McMinn2025,
  author    = "McMinn, Phil and Roslan, Muhammad Firhard and Kapfhammer, Gregory M.",
  title     = "Beyond Test Flakiness: A Manifesto for a Holistic Approach to Test Suite Health",
  booktitle = "Flaky Tests Workshop (FTW)",
  year      = "2025"
}

@article{Elgendy2025,
  author  = "Elgendy, Islam T. and Hierons, Robert M. and McMinn, Phil",
  title   = "A Systematic Mapping Study of the Metrics, Uses, and Subjects of Diversity-Based
             Testing Techniques",
  journal = "Software Testing, Verification and Reliability",
  year    = "2025"
}

Teaching

I teach the first-year COM1001 Introduction to Software Engineering module. Key to the module is team-based software development, writing automated tests, and improving code design through refactoring.

I also teach the third-year COM3529 Software Testing and Analysis module. Topics include how to write effective, maintainable tests, and how to analyse their quality. The module also covers more research-oriented topics such as automatically generating test cases (e.g., through fuzzing and search-based techniques), mutation analysis, and methods for test case priortisation.

I also supervise a range of third year dissertation projects on topics in software engineering, software testing, and on those provided by local industry partners.