Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create endpoint for cross-validation #7408

Closed
3 tasks done
wochinge opened this issue Nov 30, 2020 · 0 comments · Fixed by #7457
Closed
3 tasks done

Create endpoint for cross-validation #7408

wochinge opened this issue Nov 30, 2020 · 0 comments · Fixed by #7457
Assignees
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR

Comments

@wochinge
Copy link
Contributor

wochinge commented Nov 30, 2020

Create an endpoint for running a cross-validation as described in the implementation proposal for intent insights here.

Todos:

  • Update API spec
  • add endpoint to run cross-validation
    • by default it's a synchronous call and the response / error is returned in the response payload
    • If a callback URL is passed as query param, then the server should return immediately. The result (and potential error!) of the cross-validation is passed to the caller by calling the callback
    • We need to make sure that the synchronous cross-validation doesn't block other calls. We can e.g. do this by using run_in_executor as we do it for the training
  • consider adding a start / end datetime to the cross-validation result. This would be useful for Rasa X (especially when reading this files from disk in local mode), to see on which data the cross-validation results are based upon
@wochinge wochinge added type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR area:rasa-oss 🎡 Anything related to the open source Rasa framework priority:high labels Nov 30, 2020
@wochinge wochinge mentioned this issue Dec 8, 2020
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant