Switch bestbleu to chrF #908
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ZJaume
approved these changes
Oct 30, 2024
eu9ene
requested review from
gregtatum
and removed request for
a team and
jcristau
gregtatum
approved these changes
Oct 31, 2024
There was a problem hiding this comment.
I'm marking as approved, but I'm a bit concerned on knowing if this code is correct without tests. I looked at the extract-best task, and there is no logging and at the end there is an outfile artifact which content. Clearly it's writing out something, but it's hard to know what's going on here. I'm fine if you want to go ahead and merge, since it's existing code, but we are using new code paths that haven't been proven yet.
https://github.com/mozilla/firefox-translations-training/runs/32308849855
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
chrF is now considered more reliable than BLEU, and should work better for CJK
Based on advice from #748
+unify sacrebleu and mtdata versions everywhere
+update config generator and skip some datasets there (it's not final though)
closes #748
closes #911