Skip to content

Latest commit

 

History

History
174 lines (132 loc) · 4.79 KB

README.md

File metadata and controls

174 lines (132 loc) · 4.79 KB
Raw

Cirrus

An online judge, made by competitive programmers, for competitive programmers.

Built on top of the isolate sandbox for secure program execution, presented at the IOI conference. Server built with NodeJS and the Express framework.

Benefits

  • Lightweight. Problem data is cached to avoid any overhead.
  • Simple. Convention over configuration. Sane defaults are chosen - usable out of the box.
  • Customizable. Dynamic grading through an easy to use interface.

Problem Data

Problem data in seperated into input and output files, with their corresponding extensions config.inExt and config.outExt. Input files represent the input data for a given problem, while output files represent the expected output.

Each input file should have a corresponding output file. For example, name.in would correspond to name.out. Similarly, 1.in would correspond to 1.out. Conventionally, test case names start from 0 and increment upwards.

Graders

There are two provided graders.

default

The default grader is nice and will ignore minor formatting errors defined as trailing whitespace and extra newlines.

strict

The strict grader performs a byte by byte comparison of the contestant's output and the expected output, producing WA if there is any difference.

Adding Your Own

The graders are located in folders corresponding to their name under config.graderDir. You will need to specify an additional config.json that describes how to setup your grader.

Name Meaning Required
prepare This command is run to initialize the grader. For example, compilation. No
cmd The command to run the grader Yes
args Any arguments to pass to the grader No

All commands are run with the respective grader directory as the current working directory (e.g. graders/default).

Graders take in three environmental variables.

Name Meaning
INPUT_PATH A path to the input data file
ANSWER_PATH A path to the answer data file
RUN_CMD A command used to run the contestant's program. e.g. "/tmp/a.out"

All contestant programs will take in input from stdin, and output to stdout. The judge is responsible for writing a status code to it's stdout. Note: The entire stdout will be returned.

API

GET /langs

Returns

An JSON object with availble languages as keys. A language is a valid option if and only if it appears as a key.

{
  "c": {
    "compile": {
      "cmd": "gcc",
      "opts": [
        "-std=c11",
        ...
      ]
    },
    "out": "a.o"
  },
  "java": {
    ...
  },
  ...
}

GET /graders

Returns

An array of strings under the key graders, representing the valid graders.

{
  graders: ["default","strict"]
}

POST /run

Parameters

Name Meaning Required
lang The language of the program Yes
source The source code of the program Yes
filename The filename of the program. The source will be saved to a file with this name before compilation Yes
grader A grader to evaluate the program. Valid choices are found at /graders. If no grader is specified, stdout will be returned instead No
testsuite The name of the testsuite to use. Defaults to "global" No
tests The tests under testsuite to run. If none specified, all the tests under testsuite will be run No
compileOpts Isolate options for compiling No
executeOpts Isolate options for running No
Isolate Options
Name Meaning Required
time The maximum runtime of the program No
wallTime The maximum walltime. This clock does not stop when the program has lost the CPU or when it is waiting for an external event No
mem The maximum memory used by the program. Measured in KB. No

Compile and execute defaults are specified under config.isolate.compileDefaults and config.isolate.executeDefaults respectively.

{
  "lang": "c",
  "filename": "a.c",
  "source": "...",
  "compileOpts": {
    "time": 1
  },
  "executeOpts": {
    "time": 1,
    "wallTime": 10
  },
  "grader": "default",
  "testsuite": "global"
}

Returns

An array of objects representing the results of grading. Each object contains the name of the testcase. If a grader is specified, only a status is returned. Otherwise, stdout and stderr are returned.

Graded Example
[
  {
    "name": "0",
    "status": "AC",
    "err": null
  },
  {
    "name": "1",
    "status": "AC",
    "err": null
  },
  ...
]
Ungraded Example
[
  {
    "name": "0",
    "stderr": "",
    "stdout": "12",
    "err": null
  },
  {
    "name": "1",
    "stderr": "",
    "stdout": "910",
    "err": null
  },
  ...
]