Skip to content

[FEA]: Provide an endpoint for running evaluation #100

@AnuradhaKaruppiah

Description

@AnuradhaKaruppiah

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

High

Please provide a clear description of problem this feature solves

Request: Add a new API endpoint to trigger an evaluation run on a remote cluster and return the final status.

Proposed Endpoint:

POST /evaluate
{
  "config_file": "examples/simple/configs/eval_config.yml"
}

Expected Behavior:
The endpoint should initiate the evaluation using the provided config file and return a structured response indicating the run status: [success, failure, interrupted]

Describe your ideal solution

As /evaluate is a long running endpoint it should be async i.e. it returns a job id for polling -

{
  "job_id": "abc123",
  "status": "submitted"
}

Check on evaluation status -

GET /evaluate/status/{job_id}

Sample output

{
  "job_id": "abc123",
  "status": "running"  // or "success", "failure", "interrupted"
}

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I have searched the open feature requests and have found no duplicates for this feature request

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions