Search-Based System Testing: High Coverage, no False Alarms
by
Florian Gross, Gordon Fraser, Andreas Zeller
July 2012.
See also
More information is available at http://www.exsyst.org/.
Abstract
Modern test case generation techniques can automatically achieve high code coverage. If they operate on the unit level, they run the risk of generating inputs infeasible in reality, which, when causing failures, are painful to identify and eliminate. Running a unit test generator on five open source Java programs, we found that all of the 181 reported failures were false failures?that is, indicating a problem in the generated test case rather than the program. By generating test cases at the GUI level, our EXSYST prototype can avoid such false alarms by construction. In our evaluation, it achieves higher coverage than search-based test generators at the unit level; yet, every failure can be shown to be caused by a real sequence of input events. Whenever a system interface is available, we recommend considering search-based system testing as an alternative to avoid false failures.
BibTeX Entry
@inproceedings{fgross-exsyst-2012,
title = "Search-Based System Testing: High Coverage, no False Alarms",
author = "Florian Gross and Gordon Fraser and Andreas Zeller",
year = "2012",
month = jul,
}