Skip to content

Commit 5b87880

Browse files
authored
DOC improve README with bundle structure details
1 parent 6a4dd4f commit 5b87880

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,16 @@ It sets up a dummy classification task, evaluated with accuracy metric on a publ
66
## Structure of the bundle
77

88
- `ingestion_program/`: contains the ingestion program that will be run on participant's submissions. It is responsible for loading the code from the submission, passing the training data to train the model, and generating predictions on the test datasets.
9+
It contains:
10+
* `metadata.yaml`: A file describing how to run the ingestion program for `codabench`. For a single script ingestion program in `ingestion.py`, no need to edit it.
11+
* `ingestion.py`: A script to run the ingestion. The role of this script is to load the submission code and produce predictions that can be evaluated with the `scoring_program`.
12+
In our example, the submission define a `train_model` function that is called with the training data, and returns a model from which we can call the `predict` method.
13+
The predictions are then stored as a csv file, to be loaded with the `scoring_program`.
914
- `scoring_program/`: contains the scoring program that will be run to evaluate the predictions generated by the ingestion program. It loads the predictions and the ground truth labels, computes the evaluation metric (accuracy in this case), and outputs the score.
15+
It contains:
16+
* `metadata.yaml`: A file describing how to run the scoring program for `codabench`. For a single script ingestion program in `scoring.py`, no need to edit it.
17+
* `scoring.py`: A script to run the scoring. This script loads the prediction dumped from the ingestion program and produce a single json file containing the scores associated with the submission.
18+
In our example, we compute `accuracy` on two test sets (public and private) as well as runtime.
1019
- `solution/`: contains a sample solution submission that participants can use as a reference. Here, this is a simple Random Forest classifier.
1120
- `*_phase/`: contains the data for a given phase, including input data and reference labels. Running `setup_data.py` will generate dummy data for a development phase.
1221
- `competition.yaml`: configuration file for the codabench competition, specifying phases, tasks, and evaluation metrics.

0 commit comments

Comments
 (0)