iBUGS
Bug repositories extracted from project history

Software Engineering Chair (Prof. Zeller)
Saarland University – Computer Science
Campus E9 1 (CISPA)
66123 Saarbrücken, Germany
E-mail: zeller @ cs.uni-saarland.de
Phone: +49 681 302-70970

Deutschsprachige Startseite Page d'acceuil en franšais English home page
  

The iBUGS project at Saarland University and University of Calgary provides a bug repository with data extracted from real projects.

What's new

  • April 11: There is a new subject JodaTime.
  • May. 09: Rhino is (finally) out of beta. We fixed several build issues and cleaned some unnecessary artifacts.
  • Nov. 08: Fixed a bug in the AspectJ infrastructure. The bug prevented some JUnit tests from being executed correctly.
  • Sep. 08: Martin Burger has provided a stylesheet that converts the repository.xml of AspectJ into a browseable html file. The stylesheet can be downloaded below.

Background

Researchers have proposed a number of approaches for automatic defect localization. The iBUGS project aims at providing a benchmark for researchers to evaluate the effectiveness of such tools. Our datasets complement the data of existing benchmark repositories such as the Software-artifact Infrastructure Repository (SIR). While the SIR mostly contains subjects with artificially generated faults, we provide subjects with real bugs.

How does it work?

We extract data from a project's source code management and bugtracking system. The techniques used are described in our ASE 2007 paper. For more comprehensive information, we also provide a technical report.
  • Valentin Dallmeier, Thomas Zimmermann. Extraction of Bug Localization Benchmarks from History. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007), Atlanta, Georgia, USA, November 2007. [PDF]
  • Valentin Dallmeier, Thomas Zimmermann. Automatic Extraction of Bug Localization Benchmarks from History. Technical Report, Saarland University. June 2007. [PDF]

Dataset – AspectJ

AspectJ is a compiler that enhances the Java language with support for aspect oriented programming. It's homepage is located at www.aspectj.org.

Characteristics

Number of bugs 350
Bugs commited together with a test 223
Size of Repository 260MB
Code size of the latest revision 75 kLOC
Number of tests in the latest revison 1178

Prerequisites

The AspectJ dataset requires a java development kit version 1.4.2, which is downloadable from java.sun.com. You cannot use a newer version of the jdk, since AspectJ won't compile. You will also need Ant 1.6.5 to work with the repository. The subject has been successfully tested with the following operating systems:
  • Debian Linux
  • Mac OS 10.4
  • Windows XP

Downloads

NameFilesRelease Date and Comments
AspectJ – Release 1.3Dataset (260MB)Mon, 06 Oct 2008
Bugfix release. This release fixes an issue that prevented some JUnit tests from being executed correctly.
New: Stylesheet to generate html out of the xml file.

Dataset – Rhino

Rhino is a javascript interpreter written in Java. It is maintained by the Mozilla Foundation.

Characteristics

Number of bugs 32
Bugs with associated test 29
Size of Repository 7MB
Code size of the latest revision 49 kLOC
Number of tests in the latest revison >1000

Prerequisites

The Rhino dataset requires a java development kit version 1.4.2, which is downloadable from java.sun.com. You cannot use a newer version of the jdk, since Rhino won't compile. You will also need Ant 1.6.5 to work with the repository. The subject has been successfully tested with the following operating systems:
  • Debian Linux
  • Mac OS 10.5 Note: Apple provides its own Java Framework which has a slightly different directory layout. In order for AspectJ to compile, you will need to create a symlink in the home directory of your jdk from "bin" to "Commands".
  • Windows XP

Downloads

NameFilesRelease Date
Rhino – Initial Release 0.1Dataset (7MB)Tue, May 19 2009

FAQ

QuestionAnswer
What do you mean by fixId?The fixId identifies a bug that is included in a subject. We might as well call it bugId, but since iBugs is based on the identification of fixes, we chose to call it fixId. You need to specify the fixId whenever you use one of the ant targets that came with the subject. You can find the fixIds for all bugs in the repository descriptor (repository.xml).
What are pre-fix and post-fix versions?Fixes for bugs are extracted from the version archive of a project. For every bug we provide a snapshot of the repository right before the fix was commited (pre-fix version) and a snapshot right after the fix was commited to the repository (post-fix version).
I cannot find the associated test case mentioned in the repository descriptor. Where is it?Most fixes also commit a test case for the problem that was fixed. Since the fix is only contained in the post-fix version, you sometimes won't find the associated test in the pre-fix version. If you want to execute the associated test in the pre-fix version, you need to copy the test files into the pre-fix version.
Why does the structure of the repository file for Rhino differ from that for AspectJ?This is due to the fact that Rhino provides JUnit tests only for the most recent bugs in the version history. We therefore cannot say for sure wether a test case failed and thus omit this information in the repository descriptor.
Why do I keep getting compiler errors in Rhino?The compile process uses the same virtual machine that you used to invoke ant (not the one specified in properties.xml). You have to invoke ant using a 1.4.2 virtual machine. Sorry for that.

Dataset – JodaTime (Beta)

Characteristics

Number of bugs 8
Bugs commited together with a test 6
Size of Repository 6MB
Code size of the latest revision 73 kLOC
Number of tests in the latest revison >1000

Beta

This dataset is still in beta, and there are a few things missing:
  • There is no list of tests. The subject supports running the test suite, but you will have to collect information about executed tests yourself.
  • Building and testing the contributions of JodaTime is currently not supported. I will welcome any contributions that achieve this using ant.
  • Due to an error in the build file of JodaTime the versions for bug 2903029 cannot be built. However, this requires only to change the source and target versions in file Jodatime/build.xml to 1.5 instead of 1.3 .

Prerequisites

The JodaTime dataset requires a java development kit version 1.5 or above. You will also need Ant 1.6.5 to work with the repository. The subject has been successfully tested with the following operating systems:
  • Gentoo Linux

Downloads

NameFilesRelease Date
JodaTime – Initial Release 0.1Dataset (5MB)Mon April 18 2011

Support

There is a google group at http://groups.google.com/group/ibugs. If you have any questions regarding iBUGS or a dataset, please join and post your question. The group is set up such that your email address will never be visible.

People

<webmaster@st.cs.uni-saarland.de> · http://www.st.cs.uni-saarland.de/ibugs/ · Updated: 2014-03-23 23:43