Publications on Mutation Analysis

Marv: An Interactive Tool for Visualising and Reviewing the Results of Mutation Analysis
D Wells, Z Lévai, and P McMinn
International Conference on Automated Software Engineering (ASE Tools and Datasets Track), 2026
[PDF]
[DOI]
[]
@inproceedings{Wells2026,
  author    = "Wells, Daniel and L{\'e}vai, Zal{\'a}n and McMinn, Phil",
  title     = "{Marv}: An Interactive Tool for Visualising and Reviewing the Results of Mutation
               Analysis",
  booktitle = "International Conference on Automated Software Engineering (ASE Tools and Datasets
               Track)",
  year      = "2026",
  pages     = "3596--3600",
  publisher = "ACM",
  doi       = "10.1145/3832783.3834643"
}

multiplex: A Modular LLM-Based Mutation Framework
M Maton, GM Kapfhammer, and P McMinn
International Conference on Automated Software Engineering (ASE Tools and Datasets Track), 2026
[PDF]
[DOI]
[]
@inproceedings{Maton2026b,
  author    = "Maton, Megan and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "{multiplex}: A Modular {LLM}-Based Mutation Framework",
  booktitle = "International Conference on Automated Software Engineering (ASE Tools and Datasets
               Track)",
  year      = "2026",
  pages     = "3468--3472",
  publisher = "ACM",
  doi       = "10.1145/3832783.3834617"
}

Empirically Comparing Hazard-Guided LLM Mutation Techniques With Existing LLM- and Rule-Based Approaches
M Maton, GM Kapfhammer, and P McMinn
International Conference on Evaluation and Assessment in Software Engineering (EASE), 2026
[PDF]
[]
@inproceedings{Maton2026,
  author    = "Maton, Megan and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "Empirically Comparing Hazard-Guided {LLM} Mutation Techniques With Existing {LLM}-
               and Rule-Based Approaches",
  booktitle = "International Conference on Evaluation and Assessment in Software Engineering (EASE)",
  year      = "2026",
  publisher = "ACM"
}

How Effective are Coverage- and Diversity-Based Test Selection at Killing Stubborn Mutants?
IT Elgendy, RM Hierons, and P McMinn
International Conference on Software Testing, Verification and Validation (ICST), 2026
[PDF]
[DOI]
[]
@inproceedings{Elgendy2026,
  author    = "Elgendy, Islam T. and Hierons, Robert M. and McMinn, Phil",
  title     = "How Effective are Coverage- and Diversity-Based Test Selection at Killing Stubborn
               Mutants?",
  booktitle = "International Conference on Software Testing, Verification and Validation (ICST)",
  year      = "2026",
  pages     = "158--168",
  publisher = "IEEE",
  doi       = "10.1109/ICST69053.2026.00030"
}

mutest-rs: Flexible, Efficient Mutation Analysis Tool for Rust Programs, Using Extensive Static Analysis
Z Lévai, D Shin, and P McMinn
International Conference on Software Testing, Verification and Validation (ICST), 2026
[PDF]
[DOI]
[]
@inproceedings{Levai2026b,
  author    = "L{\'e}vai, Zal{\'a}n and Shin, Donghwan and McMinn, Phil",
  title     = "{mutest-rs}: Flexible, Efficient Mutation Analysis Tool for {Rust} Programs, Using
               Extensive Static Analysis",
  booktitle = "International Conference on Software Testing, Verification and Validation (ICST)",
  year      = "2026",
  pages     = "216--220",
  publisher = "IEEE",
  doi       = "10.1109/ICST69053.2026.00038"
}

Dynamic Mutation Scheduling: Highly Parallel, Efficient Evaluation of Mutations for Rust Programs Through Program Splitting
Z Lévai, D Shin, and P McMinn
International Conference on Software Testing, Verification and Validation (ICST), 2026
[PDF]
[DOI]
[]
@inproceedings{Levai2026a,
  author    = "L{\'e}vai, Zal{\'a}n and Shin, Donghwan and McMinn, Phil",
  title     = "Dynamic Mutation Scheduling: Highly Parallel, Efficient Evaluation of Mutations for
               {Rust} Programs Through Program Splitting",
  booktitle = "International Conference on Software Testing, Verification and Validation (ICST)",
  year      = "2026",
  pages     = "146--157",
  publisher = "IEEE",
  doi       = "10.1109/ICST69053.2026.00029"
}

A Comprehensive Empirical and Theoretical Analysis of Batching Algorithms for Efficient, Safe, Parallel Mutation Analysis in Rust
Z Lévai, D Shin, and P McMinn
ACM Transactions on Software Engineering and Methodology, 2026
[PDF]
[DOI]
[]
@article{Levai2026,
  author  = "L{\'e}vai, Zal{\'a}n and Shin, Donghwan and McMinn, Phil",
  title   = "A Comprehensive Empirical and Theoretical Analysis of Batching Algorithms for
             Efficient, Safe, Parallel Mutation Analysis in {Rust}",
  journal = "ACM Transactions on Software Engineering and Methodology",
  year    = "2026",
  pages   = "1--54",
  number  = "10",
  volume  = "35",
  doi     = "10.1145/3787851"
}

Test Flimsiness: Characterizing Flakiness Induced by Mutation to the Code Under Test
O Parry, GM Kapfhammer, M Hilton, and P McMinn
International Conference on Software Engineering (ICSE), 2026
[PDF]
[DOI]
[]
@inproceedings{Parry2026,
  author    = "Parry, Owain and Kapfhammer, Gregory M. and Hilton, Michael and McMinn, Phil",
  title     = "Test Flimsiness: Characterizing Flakiness Induced by Mutation to the Code Under Test",
  booktitle = "International Conference on Software Engineering (ICSE)",
  year      = "2026",
  pages     = "3195--3206",
  publisher = "ACM",
  doi       = "10.1145/3744916.3773125"
}

Where Tests Fall Short: Empirically Analyzing Oracle Gaps in Covered Code
M Maton, GM Kapfhammer, and P McMinn
International Symposium on Empirical Software Engineering and Measurement (ESEM), 2025
[PDF]
[DOI]
[]
@inproceedings{Maton2025,
  author    = "Maton, Megan and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "Where Tests Fall Short: Empirically Analyzing Oracle Gaps in Covered Code",
  booktitle = "International Symposium on Empirical Software Engineering and Measurement (ESEM)",
  year      = "2025",
  pages     = "1--11",
  publisher = "IEEE",
  doi       = "10.1109/ESEM64174.2025.00063"
}

PseudoSweep: A Pseudo-Tested Code Identifier
M Maton, GM Kapfhammer, and P McMinn
International Conference on Software Maintenance and Evolution (ICSME), 2024
[PDF]
[DOI]
[]
@inproceedings{Maton2024b,
  author    = "Maton, Megan and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "{PseudoSweep}: A Pseudo-Tested Code Identifier",
  booktitle = "International Conference on Software Maintenance and Evolution (ICSME)",
  year      = "2024",
  pages     = "873--877",
  publisher = "IEEE",
  doi       = "10.1109/ICSME58944.2024.00094"
}

Exploring Pseudo-Testedness: Empirically Evaluating Extreme Mutation Testing at the Statement Level
M Maton, GM Kapfhammer, and P McMinn
International Conference on Software Maintenance and Evolution (ICSME), 2024
[PDF]
[DOI]
[]
@inproceedings{Maton2024,
  author    = "Maton, Megan and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "Exploring Pseudo-Testedness: Empirically Evaluating Extreme Mutation Testing at the
               Statement Level",
  booktitle = "International Conference on Software Maintenance and Evolution (ICSME)",
  year      = "2024",
  pages     = "587--598",
  publisher = "IEEE",
  doi       = "10.1109/ICSME58944.2024.00059"
}

Batching Non-Conflicting Mutations for Efficient, Safe, Parallel Mutation Analysis in Rust
Z Lévai and P McMinn
International Conference on Software Testing, Verification and Validation (ICST), 2023
[PDF]
[DOI]
[]
@inproceedings{Levai2023,
  author    = "L{\'e}vai, Zal{\'a}n and McMinn, Phil",
  title     = "Batching Non-Conflicting Mutations for Efficient, Safe, Parallel Mutation Analysis
               in {Rust}",
  booktitle = "International Conference on Software Testing, Verification and Validation (ICST)",
  year      = "2023",
  pages     = "49--59",
  publisher = "IEEE",
  doi       = "10.1109/ICST57152.2023.00014"
}

An Empirical Study to Determine if Mutants Can Effectively Simulate Students' Programming Mistakes to Increase Tutors' Confidence in Autograding
BS Clegg, P McMinn, and G Fraser
ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE), 2021
[PDF]
[DOI]
[]
@inproceedings{Clegg2021a,
  author    = "Clegg, Benjamin S. and McMinn, Phil and Fraser, Gordon",
  title     = "An Empirical Study to Determine if Mutants Can Effectively Simulate Students'
               Programming Mistakes to Increase Tutors' Confidence in Autograding",
  booktitle = "ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE)",
  year      = "2021",
  pages     = "1055--1061",
  publisher = "ACM",
  doi       = "10.1145/3408877.3432411"
}

Automatic Detection and Removal of Ineffective Mutants for the Mutation Analysis of Relational Database Schemas
P McMinn, CJ Wright, CJ McCurdy, and GM Kapfhammer
IEEE Transactions on Software Engineering, 2019
[PDF]
[DOI]
[]
@article{McMinn2019,
  author  = "McMinn, Phil and Wright, Chris J. and McCurdy, Colton J. and Kapfhammer, Gregory M.",
  title   = "Automatic Detection and Removal of Ineffective Mutants for the Mutation Analysis of
             Relational Database Schemas",
  journal = "IEEE Transactions on Software Engineering",
  year    = "2019",
  pages   = "427--463",
  number  = "5",
  volume  = "45",
  doi     = "10.1109/TSE.2017.2786286"
}

Using Controlled Numbers of Real Faults and Mutants to Empirically Evaluate Coverage-Based Test Case Prioritization
D Paterson, GM Kapfhammer, G Fraser, and P McMinn
International Conference on Automation of Software Test (AST), 2018
[PDF]
[DOI]
[]
@inproceedings{Paterson2018,
  author    = "Paterson, David and Kapfhammer, Gregory M. and Fraser, Gordon and McMinn, Phil",
  title     = "Using Controlled Numbers of Real Faults and Mutants to Empirically Evaluate
               Coverage-Based Test Case Prioritization",
  booktitle = "International Conference on Automation of Software Test (AST)",
  year      = "2018",
  pages     = "57--63",
  publisher = "ACM",
  doi       = "10.1145/3194733.3194735"
}

mrstudyr: Retrospectively Studying the Effectiveness of Mutant Reduction Techniques
CJ McCurdy, P McMinn, and GM Kapfhammer
International Conference on Software Maintenance and Evolution (ICSME), 2016
[PDF]
[DOI]
[]
@inproceedings{McCurdy2016,
  author    = "McCurdy, Colton J. and McMinn, Phil and Kapfhammer, Gregory M.",
  title     = "{mrstudyr}: Retrospectively Studying the Effectiveness of Mutant Reduction
               Techniques",
  booktitle = "International Conference on Software Maintenance and Evolution (ICSME)",
  year      = "2016",
  pages     = "591--595",
  publisher = "IEEE",
  doi       = "10.1109/ICSME.2016.92"
}

Virtual Mutation Analysis of Relational Database Schemas
P McMinn, GM Kapfhammer, and CJ Wright
International Workshop on Automation of Software Test (AST), 2016
[PDF]
[DOI]
[]
@inproceedings{McMinn2016b,
  author    = "McMinn, Phil and Kapfhammer, Gregory M. and Wright, Chris J.",
  title     = "Virtual Mutation Analysis of Relational Database Schemas",
  booktitle = "International Workshop on Automation of Software Test (AST)",
  year      = "2016",
  pages     = "36--42",
  publisher = "ACM",
  doi       = "10.1145/2896921.2896933"
}

The Impact of Equivalent, Redundant and Quasi Mutants on Database Schema Mutation Analysis
CJ Wright, GM Kapfhammer, and P McMinn
International Conference on Quality Software (QSIC), 2014
[PDF]
[DOI]
[]
@inproceedings{Wright2014,
  author    = "Wright, Chris J. and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "The Impact of Equivalent, Redundant and Quasi Mutants on Database Schema Mutation
               Analysis",
  booktitle = "International Conference on Quality Software (QSIC)",
  year      = "2014",
  pages     = "57--66",
  publisher = "IEEE",
  doi       = "10.1109/QSIC.2014.26"
}

Efficient Mutation Analysis of Relational Database Structure Using Mutant Schemata and Parallelisation
CJ Wright, GM Kapfhammer, and P McMinn
International Workshop on Mutation Analysis (Mutation), 2013
[PDF]
[DOI]
[]
@inproceedings{Wright2013a,
  author    = "Wright, Chris J. and Kapfhammer, Gregory M. and McMinn, Phil",
  title     = "Efficient Mutation Analysis of Relational Database Structure Using Mutant Schemata
               and Parallelisation",
  booktitle = "International Workshop on Mutation Analysis (Mutation)",
  year      = "2013",
  pages     = "63--72",
  publisher = "IEEE",
  doi       = "10.1109/ICSTW.2013.15"
}

Mutation Operators for Agent-Based Models
SF Adra and P McMinn
International Workshop on Mutation Analysis (Mutation), 2010
[PDF]
[DOI]
[]
@inproceedings{Adra2010,
  author    = "Adra, Salem F. and McMinn, Phil",
  title     = "Mutation Operators for Agent-Based Models",
  booktitle = "International Workshop on Mutation Analysis (Mutation)",
  year      = "2010",
  pages     = "151--156",
  publisher = "IEEE",
  doi       = "10.1109/ICSTW.2010.9"
}