What is a Flaky Test?
The cost of software failures is a huge burden to the worldwide economy that was estimated to be at least £1.3 trillion in 2017. Consequently, software testing, a vital defence against failures, contributes to a large proportion of software development effort and cost. Flaky tests are a particular strain on resources allocated to software development because they intermittently pass and fail without changes to tests or project code, with often maddening, non-obvious causes. Flaky tests are tests that fundamentally do not always tell the truth: they can fail when code is working, and pass when it isn't. Because developers can no longer trust the results of their tests, they are unable to gain confidence that software is working correctly, potentially exposing end-users to the consequences of software failures. Flaky tests are a common occurrence in industry, significantly disrupting software development — even for companies with the greatest amount of resources to tackle them, including Microsoft, Facebook, and Google.
What is this PhD position about?
This PhD will aim to develop automated approaches to help software engineers mitigate and alleviate the problem of flaky tests in their every-day development processes. These could involve developing automatic approaches to: (a) detecting potential flaky tests ahead of time; (b) discerning when a test is flaky and when there is a genuine test failure; (c) helping developers reproduce flaky tests so they can debug them; and (d) automatically repairing flaky tests so that they are no longer flaky.
The PhD work will complement on-going work on our EPSRC-funded Test FLARE project (Test FLakiness Automated REproduction).
When Will it Start, and What About Funding?
The PhD will start in October 2025. The position comes with fees paid.
If you are a "home" student (i.e., you are from the UK or have settled status) you will receive a stipend for living expenses that be amounts to approximately £20k per year, tax-free.
If you are an international student, you'll need another source of income to support your living expenses. Alternatively, you can receive the stipend and pay the difference between home and international study fees instead. International fees for Computer Science can be found here.
Where Will I Work and With Whom?
Testing Group Peak District Walk, September 2023
You will work in my team, and alongside Owain Parry, who is the Post-Doctoral Researcher on the Test FLARE project mentioned above.
You will have a desk in the Testing Research Group workspace, along with other members of the group. Our group is friendly, and regularly meets for social activities; while also being one of the largest Testing Research Groups in the UK.
I'm Interested — How Do I Find Out More?
In the first instance, let's chat! Email, me, Phil McMinn for an appointment. We can have a video call or we can meet in person if you already a Sheffield student and/or live locally.
In this meeting we can talk more about what the position and a PhD involves, what you might bring to the role, and how you can apply.
Finally, How Do I Apply?
You will need to make an application to the University using the Postgraduate Online Application Form.
Information on what documents are required and a link to the application form can be found here.
Make sure you name me — Phil McMinn — as your supervisor on the form, to ensure your application comes to me.
Your application needs to include a "research proposal" that outlines your reasons for applying, explains details of your skills and experience that are relevant to the position, and details any initial ideas you have about the project. (We can discuss this more if you contact me to chat first as mentioned above.)