CSC News

Williams and Collaborators Win the Most Influential Paper Award at 2020 IEEE ICST Conference

Distinguished Professor Dr. Laurie Williams, alongside fellow researchers Thomas Zimmermann and Nachiappan Nagappan, recently won the Most Influential Paper Award at the 13th IEEE International Conference on Software Testing (ICST), Validation and Verification.

Williams is an influential leader in the development of vulnerability prediction models (VPM) that are used to predict the location of vulnerable code which can be used in the prioritization of which areas of the code to spend additional time for vulnerability detection. Her research has used the following as predictors: social networking metrics, complexity, unfiltered static analysis alerts, source code metrics, and stack traces from crash dumps. Her papers have influenced many other researchers to study this topic and remain the most cited in VPM. The influence was recognized in 2020 by an ICST Test of Time award for her 2010 paper on VPM with Microsoft.  

Zimmermann, a senior principal researcher at Microsoft Research, is a long-time collaborator of Williams.  Nagappan is a partner researcher at Microsoft Research who received the 2020 Harlan D. Mills Award for his “outstanding contributions to empirical software engineering and data-driven software development.” Additionally, Nagappan was William’s first PhD student in 2005 and holds a special place in her heart.  

For over 70 years, the IEEE Computer Society has existed to empower leaders in technology and innovation. The IEEE International Conference on Software Testing, Verification, and Validation 2020 provided a common forum for researchers around the world to present their ideas and findings in the area of Software Testing, Verification, and Validation. All papers were peer-reviewed, and their acceptance was based on their originality, quality, and relevance. 

The winning paper is "Searching for a Needle in a Haystack: Predicting Security Vulnerabilities for Windows Vista."  The abstract follows:

Many factors are believed to increase the vulnerability of software system; for example, the more widely deployed or popular is a software system the more likely it is to be attacked. Early identification of defects has been a widely investigated topic in software engineering research. Early identification of software vulnerabilities can help mitigate these attacks to a large degree by focusing better security verification efforts in these components. Predicting vulnerabilities is complicated by the fact that vulnerabilities are, most often, few in number and introduce significant bias by creating a sparse dataset in the population. As a result, vulnerability prediction can be thought of us preverbally “searching for a needle in a haystack.” In this paper, we present a large-scale empirical study on Windows Vista, where we empirically evaluate the efficacy of classical metrics like complexity, churn, coverage, dependency measures, and organizational structure of the company to predict vulnerabilities and assess how well these software measures correlate with vulnerabilities. We observed in our experiments that classical software measures predict vulnerabilities with a high precision but low recall values. The actual dependencies, however, predict vulnerabilities with a lower precision but substantially higher recall.

~snyder~