Do Automatic Test Generation Tools Generate Flaky Tests?
International Conference on Software Engineering (ICSE), 2024
@inproceedings{Gruber2024, author = "Gruber, Martin and Roslan, Muhammad Firhard and Parry, Owain and Scharnb{\"o}ck, Fabian and McMinn, Phil and Fraser, Gordon", title = "Do Automatic Test Generation Tools Generate Flaky Tests?", booktitle = "International Conference on Software Engineering (ICSE)", year = "2024" }
[Copy to Clipboard]
An Empirical Comparison of EvoSuite and DSpot for Improving Developer-Written Test Suites With Respect to Mutation Score
International Symposium on Search-Based Software Engineering (SSBSE), 2022
@inproceedings{Roslan2022, author = "Roslan, Muhammad Firhard and Rojas, Jos{\'e} Miguel and McMinn, Phil", title = "An Empirical Comparison of {EvoSuite} and {DSpot} for Improving Developer-Written Test Suites With Respect to Mutation Score", booktitle = "International Symposium on Search-Based Software Engineering (SSBSE)", year = "2022" }
[Copy to Clipboard]
An Investigation Into the Effect of Control and Data Dependence Chain Length on Predicate Testability
International Working Conference on Source Code Analysis & Manipulation (SCAM), 2020
@inproceedings{Binkley2020, author = "Binkley, David W. and Glenn, James and Alsharif, Abdullah and McMinn, Phil", title = "An Investigation Into the Effect of Control and Data Dependence Chain Length on Predicate Testability", booktitle = "International Working Conference on Source Code Analysis \& Manipulation (SCAM)", year = "2020" }
[Copy to Clipboard]
Random or Evolutionary Search for Object-Oriented Test Suite Generation?
Software Testing, Verification and Reliability, 2018
@article{Shamshiri2018, author = "Shamshiri, Sina and Rojas, Jos{\'e} Miguel and Gazzola, Luca and Fraser, Gordon and McMinn, Phil and Mariani, Leonardo and Arcuri, Andrea", title = "Random or Evolutionary Search for Object-Oriented Test Suite Generation?", journal = "Software Testing, Verification and Reliability", year = "2018", number = "4", volume = "28" }
[Copy to Clipboard]
Evaluation of CAVM, Austin, and CodeScroll for Test Data Generation for C
Korea Conference on Software Engineering (KCSE), 2018
[PDF]
[]
@inproceedings{You2018, author = "You, Byeonghyeon and Kim, Junhwi and Kwon, Minhyuk and McMinn, Phil and Yoo, Shin", title = "Evaluation of {CAVM}, {Austin}, and {CodeScroll} for Test Data Generation for {C}", booktitle = "Korea Conference on Software Engineering (KCSE)", year = "2018" }
[Copy to Clipboard]
Evaluating CAVM: A New Search-Based Test Data Generation Tool for C
International Symposium on Search-Based Software Engineering (SSBSE), 2017
@inproceedings{Kim2017, author = "Kim, Junhwi and You, Byeonghyeon and Kwon, Minhyuk and McMinn, Phil and Yoo, Shin", title = "Evaluating {CAVM}: A New Search-Based Test Data Generation Tool for {C}", booktitle = "International Symposium on Search-Based Software Engineering (SSBSE)", year = "2017", pages = "143--149" }
[Copy to Clipboard]
SchemaAnalyst: Search-Based Test Data Generation for Relational Database Schemas
International Conference on Software Maintenance and Evolution (ICSME), 2016
@inproceedings{McMinn2016, author = "McMinn, Phil and Wright, Chris J. and Kinneer, Cody and McCurdy, Colton J. and Camara, Michael and Kapfhammer, Gregory M.", title = "{SchemaAnalyst}: Search-Based Test Data Generation for Relational Database Schemas", booktitle = "International Conference on Software Maintenance and Evolution (ICSME)", year = "2016", pages = "586--590" }
[Copy to Clipboard]
AVMf: An Open-Source Framework and Implementation of the Alternating Variable Method
International Symposium on Search-Based Software Engineering (SSBSE), 2016
@inproceedings{McMinn2016a, author = "McMinn, Phil and Kapfhammer, Gregory M.", title = "{AVMf}: An Open-Source Framework and Implementation of the Alternating Variable Method", booktitle = "International Symposium on Search-Based Software Engineering (SSBSE)", year = "2016", pages = "259--266", series = "Lecture Notes in Computer Science", volume = "9962" }
[Copy to Clipboard]
Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study
ACM Transactions on Software Engineering and Methodology, 2015
@article{Fraser2015, author = "Fraser, Gordon and Staats, Matt and McMinn, Phil and Arcuri, Andrea and Padberg, Frank", title = "Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study", journal = "ACM Transactions on Software Engineering and Methodology", year = "2015", editor = "Harman, Mark and Pezz{\`e}, Mauro", number = "4", volume = "24" }
[Copy to Clipboard]
A Memetic Algorithm for Whole Test Suite Generation
Journal of Systems and Software, 2015
@article{Fraser2015a, author = "Fraser, Gordon and Arcuri, Andrea and McMinn, Phil", title = "A Memetic Algorithm for Whole Test Suite Generation", journal = "Journal of Systems and Software", year = "2015", pages = "311--327", volume = "103" }
[Copy to Clipboard]
Design and Analysis of Different Alternating Variable Searches for Search-Based Software Testing
Theoretical Computer Science, 2015
@article{Kempka2015, author = "Kempka, Joseph and McMinn, Phil and Sudholt, Dirk", title = "Design and Analysis of Different Alternating Variable Searches for Search-Based Software Testing", journal = "Theoretical Computer Science", year = "2015", pages = "1--20", volume = "605" }
[Copy to Clipboard]
Automatically Evaluating the Efficiency of Search-Based Test Data Generation for Relational Database Schemas
International Conference on Software Engineering and Knowledge Engineering (SEKE), 2015
@inproceedings{Kinneer2015, author = "Kinneer, Cody and Kapfhammer, Gregory M. and Wright, Chris J. and McMinn, Phil", title = "Automatically Evaluating the Efficiency of Search-Based Test Data Generation for Relational Database Schemas", booktitle = "International Conference on Software Engineering and Knowledge Engineering (SEKE)", year = "2015" }
[Copy to Clipboard]
The Effectiveness of Test Coverage Criteria for Relational Database Schema Integrity Constraints
ACM Transactions on Software Engineering and Methodology, 2015
@article{McMinn2015, author = "McMinn, Phil and Wright, Chris J. and Kapfhammer, Gregory M.", title = "The Effectiveness of Test Coverage Criteria for Relational Database Schema Integrity Constraints", journal = "ACM Transactions on Software Engineering and Methodology", year = "2015", pages = "8:1--8:49", number = "1", volume = "25" }
[Copy to Clipboard]
An Analysis of the Effectiveness of Different Coverage Criteria for Testing Relational Database Schema Integrity Constraints
Technical Report, Department of Computer Science, University of Sheffield, 2015
@techreport{McMinn2015a, author = "McMinn, Phil and Wright, Chris J. and Kapfhammer, Gregory M.", title = "An Analysis of the Effectiveness of Different Coverage Criteria for Testing Relational Database Schema Integrity Constraints", institution = "Department of Computer Science, University of Sheffield", year = "2015", number = "CS-15-01" }
[Copy to Clipboard]
Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges
International Conference on Automated Software Engineering (ASE), 2015
@inproceedings{Shamshiri2015, author = "Shamshiri, Sina and Just, Rene and Rojas, Jos{\'e} Miguel and Fraser, Gordon and McMinn, Phil and Arcuri, Andrea", title = "Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges", booktitle = "International Conference on Automated Software Engineering (ASE)", year = "2015", pages = "201--211" }
[Copy to Clipboard]
Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?
Genetic and Evolutionary Computation Conference (GECCO), 2015
@inproceedings{Shamshiri2015a, author = "Shamshiri, Sina and Rojas, Jos{\'e} Miguel and Fraser, Gordon and McMinn, Phil", title = "Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2015", pages = "1367--1374" }
[Copy to Clipboard]
Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost
International Conference on Software Testing, Verification and Validation (ICST), 2013
@inproceedings{Afshan2013, author = "Afshan, Sheeva and McMinn, Phil and Stevenson, Mark", title = "Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost", booktitle = "International Conference on Software Testing, Verification and Validation (ICST)", year = "2013", pages = "352--361" }
[Copy to Clipboard]
An Orchestrated Survey on Automated Software Test Case Generation
Journal of Systems and Software, 2013
@article{Anand2013, author = "Anand, Saswat and Burke, Edmund and Chen, Tsong Yueh and Clark, John and Cohen, Myra B. and Grieskamp, Wolfgang and Harman, Mark and Harrold, Mary Jean and McMinn, Phil", title = "An Orchestrated Survey on Automated Software Test Case Generation", journal = "Journal of Systems and Software", year = "2013", pages = "1978--2001", editor = "Bertolino, Antonia and Foster, Howard and Li, J. Jenny and Zhu, Hong", number = "8", volume = "86" }
[Copy to Clipboard]
Does Automated White-Box Test Generation Really Help Software Testers?
International Symposium on Software Testing and Analysis (ISSTA), 2013
@inproceedings{Fraser2013, author = "Fraser, Gordon and Staats, Matt and McMinn, Phil and Arcuri, Andrea and Padberg, Frank", title = "Does Automated White-Box Test Generation Really Help Software Testers?", booktitle = "International Symposium on Software Testing and Analysis (ISSTA)", year = "2013", pages = "291--301" }
[Copy to Clipboard]
Test Suite Generation With Memetic Algorithms
Genetic and Evolutionary Computation Conference (GECCO), 2013
@inproceedings{Fraser2013a, author = "Fraser, Gordon and Arcuri, Andrea and McMinn, Phil", title = "Test Suite Generation With Memetic Algorithms", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2013", pages = "1437--1444" }
[Copy to Clipboard]
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
International Conference on Software Testing, Verification and Validation (ICST), 2013
@inproceedings{Kapfhammer2013, author = "Kapfhammer, Gregory M. and McMinn, Phil and Wright, Chris J.", title = "Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems", booktitle = "International Conference on Software Testing, Verification and Validation (ICST)", year = "2013", pages = "31--40" }
[Copy to Clipboard]
A Theoretical Runtime and Empirical Analysis of Different Alternating Variable Searches for Search-Based Testing
Genetic and Evolutionary Computation Conference (GECCO), 2013
@inproceedings{Kempka2013, author = "Kempka, Joseph and McMinn, Phil and Sudholt, Dirk", title = "A Theoretical Runtime and Empirical Analysis of Different Alternating Variable Searches for Search-Based Testing", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2013", pages = "1445--1452" }
[Copy to Clipboard]
An Identification of Program Factors That Impact Crossover Performance in Evolutionary Test Input Generation for the Branch Coverage of C Programs
Information and Software Technology, 2013
@article{McMinn2013, author = "McMinn, Phil", title = "An Identification of Program Factors That Impact Crossover Performance in Evolutionary Test Input Generation for the Branch Coverage of {C} Programs", journal = "Information and Software Technology", year = "2013", pages = "153--172", number = "1", volume = "55" }
[Copy to Clipboard]
Search-Based Propagation of Regression Faults in Automated Regression Testing
International Workshop on Regression Testing (Regression), 2013
@inproceedings{Shamshiri2013, author = "Shamshiri, Sina and Fraser, Gordon and McMinn, Phil and Orso, Alex", title = "Search-Based Propagation of Regression Faults in Automated Regression Testing", booktitle = "International Workshop on Regression Testing (Regression)", year = "2013", pages = "396--399" }
[Copy to Clipboard]
Input Domain Reduction Through Irrelevant Variable Removal and Its Effect on Local, Global and Hybrid Search-Based Structural Test Data Generation
IEEE Transactions on Software Engineering, 2012
@article{McMinn2012, author = "McMinn, Phil and Harman, Mark and Hassoun, Youssef and Lakhotia, Kiran and Wegener, Joachim", title = "Input Domain Reduction Through Irrelevant Variable Removal and Its Effect on Local, Global and Hybrid Search-Based Structural Test Data Generation", journal = "IEEE Transactions on Software Engineering", year = "2012", pages = "453--477", number = "2", volume = "38" }
[Copy to Clipboard]
Search-Based Test Input Generation for String Data Types Using the Results of Web Queries
International Conference on Software Testing, Verification and Validation (ICST), 2012
@inproceedings{McMinn2012a, author = "McMinn, Phil and Shahbaz, Muzammil and Stevenson, Mark", title = "Search-Based Test Input Generation for String Data Types Using the Results of Web Queries", booktitle = "International Conference on Software Testing, Verification and Validation (ICST)", year = "2012", pages = "141--150" }
[Copy to Clipboard]
Symbolic Search-Based Testing
International Conference on Automated Software Engineering (ASE), 2011
@inproceedings{Baars2011, author = "Baars, Arthur and Harman, Mark and Hassoun, Youssef and Lakhotia, Kiran and McMinn, Phil and Tonella, Paolo and Vos, Tanja", title = "Symbolic Search-Based Testing", booktitle = "International Conference on Automated Software Engineering (ASE)", year = "2011", pages = "53--62" }
[Copy to Clipboard]
Search-Based Software Engineering: Techniques, Taxonomy, Tutorial
In Empirical Software Engineering and Verification, 2011
@incollection{Harman2011, author = "Harman, Mark and McMinn, Phil and de Souza, Jerffeson Teixeira and Yoo, Shin", title = "Search-Based Software Engineering: Techniques, Taxonomy, Tutorial", booktitle = "Empirical Software Engineering and Verification", year = "2011", pages = "1--59", editor = "Meyer, Bertrand and Nordio, Martin", publisher = "Springer", series = "Lecture Notes in Computer Science", volume = "7007" }
[Copy to Clipboard]
Search-Based Software Testing: Past, Present and Future
International Workshop on Search-Based Software Testing (SBST), 2011
@inproceedings{McMinn2011, author = "McMinn, Phil", title = "Search-Based Software Testing: Past, Present and Future", booktitle = "International Workshop on Search-Based Software Testing (SBST)", year = "2011", pages = "153--163" }
[Copy to Clipboard]
A Theoretical and Empirical Study of Search Based Testing: Local, Global and Hybrid Search
IEEE Transactions on Software Engineering, 2010
@article{Harman2010, author = "Harman, Mark and McMinn, Phil", title = "A Theoretical and Empirical Study of Search Based Testing: Local, Global and Hybrid Search", journal = "IEEE Transactions on Software Engineering", year = "2010", pages = "226--247", number = "2", volume = "36" }
[Copy to Clipboard]
Optimizing for the Number of Tests Generated in Search Based Test Data Generation With an Application to the Oracle Cost Problem
International Workshop on Search-Based Software Testing (SBST), 2010
@inproceedings{Harman2010a, author = "Harman, Mark and Kim, Sung Gon and Lakhotia, Kiran and McMinn, Phil and Yoo, Shin", title = "Optimizing for the Number of Tests Generated in Search Based Test Data Generation With an Application to the Oracle Cost Problem", booktitle = "International Workshop on Search-Based Software Testing (SBST)", year = "2010", pages = "182--191" }
[Copy to Clipboard]
An Empirical Investigation Into Branch Coverage for C Programs Using CUTE and AUSTIN
Journal of Systems and Software, 2010
@article{Lakhotia2010, author = "Lakhotia, Kiran and McMinn, Phil and Harman, Mark", title = "An Empirical Investigation Into Branch Coverage for {C} Programs Using {CUTE} and {AUSTIN}", journal = "Journal of Systems and Software", year = "2010", pages = "2379--2391", number = "12", volume = "83" }
[Copy to Clipboard]
How Does Program Structure Impact the Effectiveness of the Crossover Operator in Evolutionary Testing?
International Symposium on Search-Based Software Engineering (SSBSE), 2010
@inproceedings{McMinn2010, author = "McMinn, Phil", title = "How Does Program Structure Impact the Effectiveness of the Crossover Operator in Evolutionary Testing?", booktitle = "International Symposium on Search-Based Software Engineering (SSBSE)", year = "2010", pages = "9--18" }
[Copy to Clipboard]
Reducing Qualitative Human Oracle Costs Associated With Automatically Generated Test Data
International Workshop on Software Test Output Validation (STOV), 2010
@inproceedings{McMinn2010a, author = "McMinn, Phil and Stevenson, Mark and Harman, Mark", title = "Reducing Qualitative Human Oracle Costs Associated With Automatically Generated Test Data", booktitle = "International Workshop on Software Test Output Validation (STOV)", year = "2010", pages = "1--4" }
[Copy to Clipboard]
Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?
Testing: Academic and Industrial Conference — Practice And Research Techniques (TAIC PART), 2009
@inproceedings{Lakhotia2009, author = "Lakhotia, Kiran and McMinn, Phil and Harman, Mark", title = "Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?", booktitle = "Testing: Academic and Industrial Conference --- Practice And Research Techniques (TAIC PART)", year = "2009", pages = "95--104" }
[Copy to Clipboard]
Empirical Evaluation of a Nesting Testability Transformation for Evolutionary Testing
ACM Transactions on Software Engineering and Methodology, 2009
@article{McMinn2009, author = "McMinn, Phil and Binkley, David W. and Harman, Mark", title = "Empirical Evaluation of a Nesting Testability Transformation for Evolutionary Testing", journal = "ACM Transactions on Software Engineering and Methodology", year = "2009", pages = "11:1--11:27", number = "3", volume = "18" }
[Copy to Clipboard]
Search-Based Failure Discovery Using Testability Transformations to Generate Pseudo-Oracles
Genetic and Evolutionary Computation Conference (GECCO), 2009
@inproceedings{McMinn2009a, author = "McMinn, Phil", title = "Search-Based Failure Discovery Using Testability Transformations to Generate Pseudo-Oracles", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2009", pages = "1689--1696" }
[Copy to Clipboard]
Handling Dynamic Data Structures in Search-Based Testing
Genetic and Evolutionary Computation Conference (GECCO), 2008
@inproceedings{Lakhotia2008, author = "Lakhotia, Kiran and Harman, Mark and McMinn, Phil", title = "Handling Dynamic Data Structures in Search-Based Testing", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2008", pages = "1759--1766" }
[Copy to Clipboard]
The Impact of Input Domain Reduction on Search-Based Test Data Generation
Joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), 2007
@inproceedings{Harman2007, author = "Harman, Mark and Hassoun, Youssef and Lakhotia, Kiran and McMinn, Phil and Wegener, Joachim", title = "The Impact of Input Domain Reduction on Search-Based Test Data Generation", booktitle = "Joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE)", year = "2007", pages = "155--164" }
[Copy to Clipboard]
A Theoretical and Empirical Analysis of Evolutionary Testing and Hill Climbing for Structural Test Data Generation
International Symposium on Software Testing and Analysis (ISSTA), 2007
@inproceedings{Harman2007a, author = "Harman, Mark and McMinn, Phil", title = "A Theoretical and Empirical Analysis of Evolutionary Testing and Hill Climbing for Structural Test Data Generation", booktitle = "International Symposium on Software Testing and Analysis (ISSTA)", year = "2007", pages = "73--83" }
[Copy to Clipboard]
A Multi-Objective Approach to Search-Based Test Data Generation
Genetic and Evolutionary Computation Conference (GECCO), 2007
@inproceedings{Harman2007b, author = "Harman, Mark and Lakhotia, Kiran and McMinn, Phil", title = "A Multi-Objective Approach to Search-Based Test Data Generation", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2007", pages = "1098--1105" }
[Copy to Clipboard]
IGUANA: Input Generation Using Automated Novel Algorithms. A Plug and Play Research Tool
Technical Report, Department of Computer Science, University of Sheffield, 2007
@techreport{McMinn2007, author = "McMinn, Phil", title = "{IGUANA: Input Generation Using Automated Novel Algorithms}. A Plug and Play Research Tool", institution = "Department of Computer Science, University of Sheffield", year = "2007", number = "CS-07-14" }
[Copy to Clipboard]
Evolutionary Testing Using an Extended Chaining Approach
Evolutionary Computation, 2006
@article{McMinn2006, author = "McMinn, Phil and Holcombe, Mike", title = "Evolutionary Testing Using an Extended Chaining Approach", journal = "Evolutionary Computation", year = "2006", pages = "41--64", number = "1", volume = "14" }
[Copy to Clipboard]
The Species Per Path Approach to Search-Based Software Test Data Generation
International Symposium on Software Testing and Analysis (ISSTA), 2006
@inproceedings{McMinn2006a, author = "McMinn, Phil and Harman, Mark and Binkley, David W. and Tonella, Paolo", title = "The Species Per Path Approach to Search-Based Software Test Data Generation", booktitle = "International Symposium on Software Testing and Analysis (ISSTA)", year = "2006", pages = "13--24" }
[Copy to Clipboard]
Testability Transformation for Efficient Automated Test Data Search in the Presence of Nesting
UK Software Testing Workshop (UKTest), 2005
@inproceedings{McMinn2005, author = "McMinn, Phil and Binkley, David W. and Harman, Mark", title = "Testability Transformation for Efficient Automated Test Data Search in the Presence of Nesting", booktitle = "UK Software Testing Workshop (UKTest)", year = "2005", pages = "165--182" }
[Copy to Clipboard]
Evolutionary Testing of State-Based Programs
Genetic and Evolutionary Computation Conference (GECCO), 2005
@inproceedings{McMinn2005a, author = "McMinn, Phil and Holcombe, Mike", title = "Evolutionary Testing of State-Based Programs", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2005", pages = "1013--1020" }
[Copy to Clipboard]
Evolutionary Search for Test Data in the Presence of State Behaviour
PhD Thesis, University of Sheffield, 2005
@phdthesis{McMinn2005b, author = "McMinn, Phil", title = "Evolutionary Search for Test Data in the Presence of State Behaviour", school = "University of Sheffield", year = "2005" }
[Copy to Clipboard]
Search-Based Software Test Data Generation: A Survey
Software Testing, Verification and Reliability, 2004
@article{McMinn2004, author = "McMinn, Phil", title = "Search-Based Software Test Data Generation: A Survey", journal = "Software Testing, Verification and Reliability", year = "2004", pages = "105--156", number = "2", volume = "14" }
[Copy to Clipboard]
Hybridizing Evolutionary Testing With the Chaining Approach
Genetic and Evolutionary Computation Conference (GECCO), 2004
@inproceedings{McMinn2004a, author = "McMinn, Phil and Holcombe, Mike", title = "Hybridizing Evolutionary Testing With the Chaining Approach", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2004", pages = "1363--1374", series = "Lecture Notes in Computer Science", volume = "3103" }
[Copy to Clipboard]
The State Problem for Evolutionary Testing
Genetic and Evolutionary Computation Conference (GECCO), 2003
@inproceedings{McMinn2003, author = "McMinn, Phil and Holcombe, Mike", title = "The State Problem for Evolutionary Testing", booktitle = "Genetic and Evolutionary Computation Conference (GECCO)", year = "2003", pages = "2488--2498", series = "Lecture Notes in Computer Science", volume = "2724" }
[Copy to Clipboard]