| |
The iBUGS project at Saarland University and University of Calgary provides a bug repository with data extracted from real projects.
What's new
- April 11: There is a new subject JodaTime.
- May. 09: Rhino is (finally) out of beta. We fixed several build issues and cleaned some unnecessary artifacts.
- Nov. 08: Fixed a bug in the AspectJ infrastructure. The bug prevented some JUnit tests from being executed correctly.
- Sep. 08: Martin Burger has provided a stylesheet that converts the
repository.xml of AspectJ into a browseable html file. The stylesheet can be downloaded below.
Background
Researchers have proposed a number of approaches for automatic defect localization. The iBUGS project aims at providing a benchmark for researchers to evaluate the effectiveness of such tools. Our datasets complement the data of existing benchmark repositories such as the Software-artifact Infrastructure Repository (SIR). While the SIR mostly contains subjects with artificially generated faults, we provide subjects with real bugs.
How does it work?
We extract data from a project's source code management and bugtracking system. The techniques used are described in our ASE 2007 paper. For more comprehensive information, we also provide a technical report.
-
Valentin Dallmeier, Thomas Zimmermann. Extraction of Bug Localization Benchmarks from History. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007), Atlanta, Georgia, USA, November 2007. [PDF]
-
Valentin Dallmeier, Thomas Zimmermann.
Automatic Extraction of Bug Localization Benchmarks from History.
Technical Report, Saarland University. June 2007.
[PDF]
Dataset – AspectJ
AspectJ is a compiler that enhances the Java language with support for aspect oriented programming. It's homepage is located at www.aspectj.org.
Characteristics
Number of bugs |
350 |
Bugs commited together with a test |
223 |
Size of Repository |
260MB |
Code size of the latest revision |
75 kLOC |
Number of tests in the latest revison |
1178 |
|
|
Prerequisites
The AspectJ dataset requires a java development kit version 1.4.2, which is downloadable from java.sun.com. You cannot use a newer version of the jdk, since AspectJ won't compile. You will also need Ant 1.6.5 to work with the repository. The subject has been successfully tested with the following operating systems:
- Debian Linux
- Mac OS 10.4
- Windows XP
Downloads
Name | Files | Release Date and Comments |
| AspectJ – Release 1.3 | Dataset (260MB) | Mon, 06 Oct 2008 Bugfix release. This release fixes an issue that prevented some JUnit tests from being executed correctly. |
| | | New: Stylesheet to generate html out of the xml file. |
Dataset – Rhino
Rhino is a javascript interpreter written in Java. It is maintained by the Mozilla Foundation.
Characteristics
Number of bugs |
32 |
Bugs with associated test |
29 |
Size of Repository |
7MB |
Code size of the latest revision |
49 kLOC |
Number of tests in the latest revison |
>1000 |
|
|
Prerequisites
The Rhino dataset requires a java development kit version 1.4.2, which is downloadable from java.sun.com. You cannot use a newer version of the jdk, since Rhino won't compile. You will also need Ant 1.6.5 to work with the repository. The subject has been successfully tested with the following operating systems:
- Debian Linux
- Mac OS 10.5 Note: Apple provides its own Java Framework which has a slightly different directory layout. In order for AspectJ to compile, you will need to create a symlink in the home directory of your jdk from "bin" to "Commands".
- Windows XP
Downloads
Name | Files | Release Date |
| Rhino – Initial Release 0.1 | Dataset (7MB) | Tue, May 19 2009 |
FAQ
Question | Answer |
What do you mean by fixId? | The fixId identifies a bug that is included in a subject. We might as well call it bugId, but since iBugs is based on the identification of fixes, we chose to call it fixId. You need to specify the fixId whenever you use one of the ant targets that came with the subject. You can find the fixIds for all bugs in the repository descriptor (repository.xml). |
What are pre-fix and post-fix versions? | Fixes for bugs are extracted from the version archive of a project. For every bug we provide a snapshot of the repository right before the fix was commited (pre-fix version) and a snapshot right after the fix was commited to the repository (post-fix version). |
I cannot find the associated test case mentioned in the repository descriptor. Where is it? | Most fixes also commit a test case for the problem that was fixed. Since the fix is only contained in the post-fix version, you sometimes won't find the associated test in the pre-fix version. If you want to execute the associated test in the pre-fix version, you need to copy the test files into the pre-fix version. |
Why does the structure of the repository file for Rhino differ from that for AspectJ? | This is due to the fact that Rhino provides JUnit tests only for the most recent bugs in the version history. We therefore cannot say for sure wether a test case failed and thus omit this information in the repository descriptor. |
Why do I keep getting compiler errors in Rhino? | The compile process uses the same virtual machine that you used to invoke ant (not the one specified in properties.xml). You have to invoke ant using a 1.4.2 virtual machine. Sorry for that. |
Dataset – JodaTime (Beta)
Characteristics
Number of bugs |
8 |
Bugs commited together with a test |
6 |
Size of Repository |
6MB |
Code size of the latest revision |
73 kLOC |
Number of tests in the latest revison |
>1000 |
|
|
Beta
This dataset is still in beta, and there are a few things missing:
- There is no list of tests. The subject supports running the test suite, but you will have to collect information about executed tests yourself.
- Building and testing the contributions of JodaTime is currently not supported. I will welcome any contributions that achieve this using ant.
- Due to an error in the build file of JodaTime the versions for bug 2903029 cannot be built. However, this requires only to change the source and target versions in file Jodatime/build.xml to 1.5 instead of 1.3 .
Prerequisites
The JodaTime dataset requires a java development kit version 1.5 or above. You will also need Ant 1.6.5 to work with the repository. The subject has been successfully tested with the following operating systems:
Downloads
Name | Files | Release Date |
| JodaTime – Initial Release 0.1 | Dataset (5MB) | Mon April 18 2011 |
Support
There is a google group at http://groups.google.com/group/ibugs. If you have any questions regarding iBUGS or a dataset, please join and post your question. The group is set up such that your email address will never be visible.
People
Impressum ● Datenschutzerklärung
<webmaster@st.cs.uni-saarland.de> · http://www.st.cs.uni-saarland.de/ibugs/?lang=fr · Stand: 2018-04-05 13:41
|
---|