Eclipse Bug Data!
Release 2.0a, 2007-12-28

[ News | FAQ | Usage | People ]

News

2010-03-25

An extension to the bug data sets is available. The change burst set can be found here. The data set is described in detail in our new publication Change Bursts as Defect Predictors.
 

2009-12-10

A revised version of the PROMISE 2007 paper "Predicting Defects for Eclipse" is now available. It supersedes the original, published version, being based on the revised 2.0a data (see below).
 

2009-12-09

A R file for all experiments is now available.
 

2007-12-28

We removed duplicates entries from the dataset that were inserted by a bug in the Perl API. Here's the new data (labelled as Release 2.0a), which extends the original data by complexity metrics and counts from abstract syntax trees.

• as ARRF relations:
  for file level (promise-2_0a-files-arff.zip, 4.48M) and
  for package level (promise-2_0a-packages-arff.zip, 770K)
   
• as comma-separated values:
  for file level (promise-2_0a-files-csv.zip, 4.47M) and
  for package level (promise-2_0a-packages-csv.zip, 768K)

If you publish material using the extended data (ARFF or CSV format), please cite our PROMISE 2007 paper and follow the acknowledgment guidelines posted on the PROMISE Repository web page.

In addition, we provide data as XML, which does not contain the metrics but references to bug reports and transactions that fixed the bugs (not included in the ARFF ad CSV files).

• as XML:
  a single file for both, file and package levels (promise-2_0a-xml.zip, 895K)

If you publish material based on the XML format, please cite our ISESE 2006 paper.
 

2007-12-12

We have fixed the clerical error in data acquisition and now provide a thoroughly revised and validated data set. The fixed version is labelled as Release 2.0.
 

2007-08-23

Regrettably, due to a clerical error, we have temporarily withdrawn all versions of the Eclipse bug data. Please come back to this page in the first week of December for the fixed files. We apologize for any inconvenience caused.
 

2006-11-06

Eclipse bug data now contains references to Bugzilla bug reports and CVS transactions.
See also: What's new in Release 1.1?
 

2006-06-16

Eclipse bug data is now publicly available for download.

Frequently Answered Questions (FAQ)

What is this all about?

What can I do with this data?

Here's a recipe for research based on this data:

1. Take your favorite hypothesis on where errors come from.

   Example: I think that side effects increase the chance of bugs.

2. Measure the amount of each factor in the Eclipse source code. To obtain the source code, see "Where do I get these Eclipse versions?", below.

   Example: In package org.eclipse.core.launcher, 55% of all methods have side effects. (This is a hypothetical result.)

3. Check whether there is a correlation between the factor you measured and the bug data as you find it here.

   Example: The more side-effects a component has, the more defects it has. (Again, this is a hypothetical result.)

One main result of our group so far is that specific imports correlate with the number of defects; in particular, importing "internal" components results in a higher defect rate. But as software developers, we still would like to know more about what makes software fail — in particular, whether there are any domain-independent ways to predict defects.

There are several analysis methods in the field that determine software properties. The question is whether any such properties correlate with software defects. To answer this question, one first needs defect data — and this is what we provide.

Two companion papers, If Your Bug Database Could Talk... and Predicting Defects for Eclipse.pdf, are available which contains all the details, including three simple and one large experiment. The next step is yours.

Where do I get these Eclipse versions?

Why do you share this data?

What is the copyright for this data?

The XML representation of the data is copyrighted, though, as are the texts enclosed in the archive. Since the companion paper is submitted for publication, we currently do not allow redistribution of the archive.

How can I reference this work?

Thomas Zimmermann, Rahul Premraj, and Andreas Zeller: Predicting Defects for Eclipse, In Proceedings of the Third International Workshop on Predictor Models in Software Engineering, May 2007.

@inproceedings{zimmermann-promise-2007,
    title = "Predicting Defects for Eclipse",
    author = "Thomas Zimmermann and Rahul Premraj and Andreas Zeller",
    year = "2007",
    month = "May",
    booktitle = "Proceedings of the Third International Workshop on Predictor Models in Software Engineering",
    location = "Minneapolis, MN, USA",
}

I need more data!

What's in the Data?

What is in the packages?

• If Your Bug Database Could Talk.pdf
• Predicting Defects for Eclipse.pdf (note that there is an updated version of this paper!)
• eclipse-defects-version-2.0.xml
• eclipse-defects-version-2.1.xml
• eclipse-defects-version-3.0.xml
• README.html (this file)

What is the data format?

The coarse structure of the XML files is described in the companion paper "If Your Bug Database Could Talk...":

• Packages are structured hierarchically, which means subpackages are nodes with their super packages as parent node in the tree. The following example describes the tree structure:

              <package name="org.eclipse">
                <counts> ... </counts>
                <compilationunit ...> ... </compilationunit>
                <package name="org.eclipse.core">
                  ...
                </package>
              </package>
  

• Here's an extract from the file eclipse-defects-version-2.0.xml:

            <?xml version="1.0" encoding="UTF-8"?>
            <!-- comments -->
            <defects project="eclipse" release="2.0" dataversion="1.0">
            <plug-in name="platform-launcher">
              <counts>
                <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/>
                <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/>
              </counts>
              <package name="org.eclipse">
                <counts>
                  <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/>
                  <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/>
                </counts>
                <package name="org.eclipse.core">
                  <counts>
                    <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/>
                    <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/>
                  </counts>
                  <package name="org.eclipse.core.launcher">
                    <counts>
                      <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/>
                      <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/>
                    </counts>
                    <compilationunit dir="/platform-launcher/library/" base="Main.java">
                      <counts>
                        <count id="pre" value="0"/>
                        <count id="post" value="0"/>
                      </counts>
                    </compilationunit>
                  </package>
                </package>
              </package>
            </plug-in>
   

• Defect counts are listed as count at the plug-in, package and compilationunit levels.
• The value field contains the actual number of pre- ("pre") and post-release defects ("post"). The average ("avg") and maximum ("max") values refer to the defects found in the compilation units ("compilationunits").
• Each compilation unit is listed separately ("compilationunit") within the enclosing package.
• The average ("avg") is the average number of defects per compilation unit.

What's new in Release 1.1?

          <fix kind="pre" bug_id="16191" revision_id="1.6" author="mvalenta">
            <message>16191: Sharing project already in repo, picking Base, results in tag</message>
          </fix>
          <fix kind="pre" bug_id="14737" revision_id="1.5" author="mvalenta">
            <message>14737: Add capability to move tags</message>
          </fix>

Why is the defect count for the parent different from the sum over all children?

This figure exemplifies how the defects were counted in our data. In the figure, Package X comprises two sub-packages X.Y and X.Z. Package X.Y consists of two compilation units; package X.Z has three. There are four defects, each indicated with a different colour. The blue defect affects just one compilation unit in package X.Y, whereas each of the other defects affects two compilation units. In package X.Z, each compilation unit has between one and two defects. The sum over all compilation units is five. However, the total defect count for package X.Z is only three, since three distinct defects occurred overall. For package X.Y, the sum is identical to the overall defect count, because each defect affected just one compilation unit.

How was the data collected?

The major challenge for this task was to map bug reports from the bug database to compilation units in the version archive. To this end, we used text analysis of the commit messages, we identified fixes in version archives (in contrast to other changes). Typically, fix messages contain links to the bug reports in the bug database by stating a the identification number of a bug report.

Furthermore, we needed to obtain the version of each defect. We did so by using the version field provided by the bug database. Note that the first reported version was used.

Interesting. Where can I learn more about this work?

http://www.st.cs.uni-saarland.de/softevo/

People

• Andreas Zeller (contact)
• Thomas Zimmermann
• Rahul Premraj
• Sascha Just
• Adrian Schröter

Impressum ● Datenschutzerklärung