This repository contains tools for automated evaluation of model performance and robustness.
The project is still under development, so the services will be announced later!
Polygon-Scoring is a web application designed to evaluate the effectiveness of the developed model based on calculated metrics. The service provides information about the robustness of the model using standard methods for augmenting source data. Also, the user can select a specific dataset to obtain information about the model's ability for OOD (Out-of-Distribution) detection.
To use Polygon-Scoring applicaton just follow several steps:
- Upload your model and data files
- Define input data size
- Select options for data augumentation
- Select a distribution for OOD Detection evaluation
- Submit the form and get your result
Polygon-Leaderboards is a web application for automated evaluation of the model performance and robustness on supervised datasets. Also, this service also provides the opportunity for users to compete in the accuracy and efficiency of their developed models by publishing the results of metrics for a specific task in leaderboards.
To use Polygon-Leaderboards applicaton just follow several steps:
- Create an account or sign in if you already have one
- Go to the "Upload" page
- Select the desired data modality
- Fill the form with your model and data file, input data size
- Submit the form
- Publish your results and view them in leaderboard
- Manage your published results in "User Page"
- Model file should be in .onnx format
- Data file should be in .zip format and consists of:
- markup.csv - file with filenames and their labels
- data - folder for input data
The general description is available here.
The tools were developed in ITMO University.
- A. Vatyan - team leader
- T. Polevaya
- D. Zmievskiy
- Alexandra Vatyan [email protected] for collaboration suggestions
- Tatyana Polevaya [email protected] for technical questions