scripts

Reprocuce experiments in the FSE 2024 paper

Requirements

• pip install joblib
• pip install tqdm
• pip install unidiff
• pip install javalang

Dependencies

There are dependencies to run the experiment, and you can install them using the following ways.

Manually

• Install python dependencies

  You can run ./install_requirements.sh or install the dependencies using commands in the Requirement Section

Dockerfile

• You can run ./build.sh to build the image directly, or build as the following steps.

  1. Build the main CatenaD4J image (skip this step if you have built the image)

     docker build -t catena4j:main -f ../Dockerfile .
     or
     curl https://raw.githubusercontent.com/universetraveller/CatenaD4J/main/Dockerfile -o Dockerfile.main && docker build -t catena4j:main -f ./Dockerfile.main .

  2. Build the image used for experiments

     docker build -t catena4j:experiments -f ./Dockerfile .

• When the image is built, run a new container using the new image

  docker run -it catena4j:experiments /bin/bash

Steps to reproduce the experiments

To reproduce all of the experiments, you should following the order in Section Content of this repository to do the experiments.

Each experiment has its own README file, and you can see it in experiment's directory.

In directory of each experiment, we provide a quick script ./run.sh that can invoke the experiment process in a single step (which means you can reproduce the experiments by running this script in each directory one by one).

Whole process of reproduction takes more than 1 days (if executing in parallel), so we also provide approaches to reproduce partial experiments as needs (please refer to README of each experiment for more information).

The first three experiments are to prepare data for the most important experiments presented in out paper (Tests Minimization and Bugs Isolation). For the two experiments, the required files (generated by the first three steps) are already prepared and provided under the directory, so you can reproduce the two experiments directly.

Content of this directory

1. construct_database contains scripts to prepare required data for following steps.
2. parse_patches is to identify hunks to fix automatically.
3. analyze_tests contains scripts to collect all possible identifiers of assertion statements for spliting tests.
4. split_tests is to generate new failing tests code to make sure the failing tests contain only single assertion statement.
5. generate_bugs contains an implementation of bug isolation algorithm to generate isolated bugs from existing bugs.

Notice

1. Bad code style and english

   When the project started I was a freshman and when the paper was accepted (2024) I am a junior student. I do agree that the experiment code written in the past year (2023) may be hard to read and bad in code style and apologize for that.

2. The current code is highly integreted with Defects4J

   The paper says the proposed algorithms could be used in datasets other than defects4j, and that is true. However, we select to focus on 6 projects in Defects4J v2.0 due to the heavy workload, so the current implementation of algorithms is highly integrated with Defects4J (e.g. invoke defects4j's commands in the processes).

   Futhermore, the defects4j dataset is easy to use that has good dependencies supports and do not need to create docker containers that make the using awkward and it is outstanding in data accessibility (e.g. metadata of bugs).

   Nevertheless, defects4j is not enough for APR research, in both bugs quality and quantity. We will try to implement the algorithms on other datasets if necessary.