About SF100
Several promising techniques have been proposed to automate
different tasks in software testing, such as test data generation
for object-oriented software. However, reported studies in the
literature only show the feasibility of the proposed techniques,
because the choice of the employed artifacts in the case studies
(e.g., software applications) is usually done in a non-systematic
way. The chosen case study might be biased, and so it might not be
a valid representative of the addressed type of software (e.g.,
internet applications and embedded systems). The common trend seems
to be to accept this fact and get over it by simply discussing it in
a threats to validity section. To
achieve sound empirical results, we randomly selected 100 Java
projects from SourceForge, which is the most popular open source
repository (more than 300,000 projects with more than two million
registered users). The resulting benchmark not only is very large,
but more importantly it is statistically sound and representative
for open source projects.
Download
All projects selected for the case study are open source projects. To check the individual licenses please check the projects' websites (see table below). A download containing bytecode of all 100 projects together with their dependencies (check webpages for licenses) is available here:
SF100.tar.gz
Project details
Benchmark history
Not all projects on SourceForge are automatically compilable; to show the soundness of our selection, the history of compilation attempts can be downloaded here.