Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved metrics #1952

Merged
merged 18 commits into from
Apr 21, 2023
Merged

Improved metrics #1952

merged 18 commits into from
Apr 21, 2023

Conversation

Adaickalavan
Copy link
Member

  • Made the metrics module configurable by supplying parameters through a Params class.
  • Params allows to specify vehicles to be ignored in dist_to_obstacles cost function. This would be applicable in platooning tasks.
  • Unified computation of dist_to_destination (previously known as completion) and steps (i.e., time taken) as functions inside the cost functions module, instead of computing them separately in a different module.
  • In the metrics module, the records which is the raw metrics data and the scoring which is the formula to compute the final results are now separated to provided greater flexibility for applying metrics to different environments.
  • Changed benchmark_runner_v0.py to only average records across scenarios within an environment. Records are not averaged across enviroments, because the scoring formula may differ in different environments.
  • Fixed CI tests for metrics.

Base automatically changed from driving-smarts-2023 to master April 11, 2023 17:24
smarts/env/gymnasium/wrappers/metric/params.py Outdated Show resolved Hide resolved
smarts/env/gymnasium/wrappers/metric/params.py Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@Adaickalavan Adaickalavan mentioned this pull request Apr 19, 2023
@Adaickalavan Adaickalavan merged commit 7fe1639 into master Apr 21, 2023
@Adaickalavan Adaickalavan deleted the new-metrics branch April 21, 2023 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants