Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorporate error into handicap adjustments #70

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dexonsmith
Copy link
Contributor

Change get_handicap_adjustment to return a tuple with both rating and error, where the error comes from:

  • ±0.5 for rank differences that aren't (almost) the same as the number of handicap stones.
  • ±1 for rank difference of 9 (scaled up/down) for large/extreme handicaps.

Also change Glicko2Entry.copy to add incorporate the error into the rating deviation using root-of-sum-of-squares, which is the usual way to combine standard errors.

Change `get_handicap_adjustment` to return a tuple with both rating and error,
where the error comes from:

- ±0.5 for rank differences that aren't (almost) the same as the number of
  handicap stones.
- ±1 for rank difference of 9 (scaled up/down) for large/extreme handicaps.

Also change `Glicko2Entry.copy` to add incorporate the error into the rating
deviation using root-of-sum-of-squares, which is the usual way to combine
standard errors.
@dexonsmith dexonsmith requested a review from anoek March 31, 2024 21:45
@dexonsmith
Copy link
Contributor Author

Did a sniff test, but not really any analysis of whether the initial defaults are the numbers we actually want.

Let me know if you'd rather default to "off" for these until some analysis is done.

@dexonsmith
Copy link
Contributor Author

Let me know if you'd rather default to "off" for these until some analysis is done.

I mean, either way there will be analysis to do later, just a matter of the current default on the branch.

@anoek
Copy link
Member

anoek commented Apr 28, 2024

What are we looking to improve with this? Comparing the output from with and without this change, the end results are pretty similar, at least on the surface. That leads me to think it's probably fine to merge, but I'm a little hesitant to just click merge without building a bit more intuition about how this is improving results and what, if any, tradeoffs we're making.

@anoek
Copy link
Member

anoek commented May 10, 2024

From the https://github.com/dexonsmith/goratings/blob/document-ratings-v6/RatingsV6.md the added clarification for the purpose of this is:

Increase effective rating deviation of opponent for larger handicaps and strange komi values.

  • Reflects higher uncertainty that opponents' rating is correct, the more the board diverges fom "normal".
  • Reduces impact of high handicap games on ratings, enabling a policy decision to allow rated 19x19 games with 10+ stones.
  • Reduces impact of reverse-komi games on ratings, enabling a policy decision to allow handicap via reverse komi.

@anoek
Copy link
Member

anoek commented Jun 2, 2024

Marking this as draft for the summer so it's not on my "to check on and merge" list until we start revisiting stuff in the fall

@anoek anoek marked this pull request as draft June 2, 2024 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants