-
Notifications
You must be signed in to change notification settings - Fork 416
Closed
Labels
feature requestNew feature or requestNew feature or request
Description
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
High
Please provide a clear description of problem this feature solves
Request: Add a new API endpoint to trigger an evaluation run on a remote cluster and return the final status.
Proposed Endpoint:
POST /evaluate
{
"config_file": "examples/simple/configs/eval_config.yml"
}
Expected Behavior:
The endpoint should initiate the evaluation using the provided config file and return a structured response indicating the run status: [success, failure, interrupted]
Describe your ideal solution
As /evaluate is a long running endpoint it should be async i.e. it returns a job id for polling -
{
"job_id": "abc123",
"status": "submitted"
}
Check on evaluation status -
GET /evaluate/status/{job_id}
Sample output
{
"job_id": "abc123",
"status": "running" // or "success", "failure", "interrupted"
}
Additional context
No response
Code of Conduct
- I agree to follow this project's Code of Conduct
- I have searched the open feature requests and have found no duplicates for this feature request
Metadata
Metadata
Assignees
Labels
feature requestNew feature or requestNew feature or request