10/1/2018 By David Mercer, Illinois Computer Science
Written by By David Mercer, Illinois Computer Science
Continuous integration (CI) has become an increasingly important part of software creation, but developers are often limited by its speed and reliability.
As part of a $1.2 million grant from the National Science Foundation, Illinois Computer Science Professor Darko Marinov and two collaborators will devote $437,000 to a project they hope will eventually provide a set of publicly available tools to improve CI.
Continuous integration is the practice of regularly merging changes in software code into one shared version, not quite literally as they happen but typically multiple times a day. As the changes are integrated, regression tests are run to find out whether changes introduce bugs into the code.
“Developers do these actions typically several times per day. It can take anywhere from five, 10 minutes to literally hours (for each),” Marinov said. “If this thing is going to take two hours to give them the result back, they’re just going to idle -- they’re going to start reading emails, go on the web. It’s hard for them to start on a new technical task because by the time this comes back it could tell them, ‘Oh, you broke something, go and revise that.’”
Developers, he added, often face a tradeoff: speed versus quality.
“In principle I can give you the answer almost immediately, but its (quality is) very bad: ‘I think you broke something but I don’t know what’,” Marinov said.
To improve the performance of CI, Marinov and his collaborators, Assistant Professor Jonathan Bell of George Mason University and Assistant Professor Lingming Zhang of the University of Texas at Dallas, will focus on three things.
They hope to improve test selection – learning what needs to be tested based on the changes that have been made in the code and what doesn’t need to be tested – to speed up regression testing.
“It learns what we technically call dependencies – it learns some relationships between the test and the code. Then when you make the change, it simply asks, which of the tests have their dependencies changed?” Marinov said. “If a test had no dependency change, then it cannot change behavior.”
They also hope to improve reliability by focusing on so-called flaky tests – tests that themselves have bugs and incorrectly indicate problems in the code. And, finally, they hope to provide tools to help developers debug problems.
Marinov, Bell, and Zhang all have experience building testing tools used both in academia and in industry for companies such as eBay, Google, Groupon, Microsoft, and Yahoo!