Abstract: As software evolves, developers typically utilize the continuous development cycle, which builds the developers’ code and performs regression testing to ensure that their code changes do not break existing functionality. During regression testing, developers often waste time debugging their code changes because of spurious failures from flaky tests, which are tests that nondeterministically pass or fail on the same code. These spurious failures mislead developers because the failures are due to bugs that existed before the code changes. Another problem with the continuous development cycle is that the cycle can be time-consuming from doing extraneous work. My work on characterizing flaky tests has helped open the research topic of flaky tests and my work on optimizing continuous development builds have helped reduce the time developers spend on software development. Many companies (e.g., Facebook, Google, Microsoft) have since highlighted flaky tests and build times as major challenges in their software development.

In this talk, I will first describe my recent work on proactively detecting flaky tests. A prominent kind of flaky tests is order-dependent (OD) flaky tests, which pass when run in one test order but fail when run in a different test order. To exhaustively detect OD tests, one may need to run a prohibitively expensive number of test orders, e.g., N! test orders, where N is the number of tests. To help with this problem, I propose and discuss the tradeoffs of random-based and systematic-based repeated executions of the test suite. Overall, my work has helped detect more than 6000 flaky tests in over 300 open-source projects. For the second part of my talk, I will discuss our work on optimizing continuous development builds by dynamically analyzing the builds to identify work that is done to generate or modify files that are not used. By identifying and disabling such work from the builds, we find that developers can save substantial time during software development.

Short Bio: Dr. Wing Lam is an Assistant Professor in the Computer Science department at George Mason University. Dr. Lam works on several topics in software engineering, with a focus on software testing. His research improves software reliability and efficiency by characterizing bugs, developing novel techniques to detect and tame bugs, and optimizing the continuous development cycle. Dr. Lam has published in top-tier conferences such as ASE, ESEC/FSE, ICSE, ISSTA, OOPSLA, and TACAS. His techniques have helped detect and fix bugs in open-source projects and have impacted how Dragon Testing, Microsoft, and Tencent developers test their code. Dr. Lam has been the recipient of several awards, including an NSF CAREER award and an ACM SIGSOFT Outstanding Doctoral Dissertation Award. More information is available on his web page: https://cs.gmu.edu/~winglam