Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix mean absolute error metric calculation in model overview panel #1950

Merged
merged 1 commit into from
Feb 8, 2023

Conversation

imatiach-msft
Copy link
Contributor

Description

During customer session, I noticed the mean absolute error metric looked incorrect for their data. Indeed, it seems we were not taking the "mean" of the data but just summing the error. To fix the issue, I divided by the count to get the correct metric.

Before:
image

After:
image

Note how mean absolute error and mean squared error look a lot more similar now.

Checklist

  • I have added screenshots above for all UI changes.
  • I have added e2e tests for all UI changes.
  • Documentation was updated if it was needed.

Copy link
Contributor

@romanlutz romanlutz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! Also, another good reason why we shouldn't reinvent the wheel on such things and just reuse existing definitions. This seems like a pretty bad bug. Do you have plans to ship it in Studio soon?

@romanlutz
Copy link
Contributor

I guess we should also consider adding at least a few basic tests if we intend to keep these definitions around.

@imatiach-msft imatiach-msft force-pushed the ilmat/fix-mean-abs-err branch from d7133ca to 8e1bcc6 Compare February 8, 2023 06:09
@imatiach-msft imatiach-msft force-pushed the ilmat/fix-mean-abs-err branch from 8e1bcc6 to 7984752 Compare February 8, 2023 18:31
@imatiach-msft imatiach-msft merged commit 8b03fa8 into main Feb 8, 2023
@imatiach-msft imatiach-msft deleted the ilmat/fix-mean-abs-err branch February 8, 2023 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants