feat(translation): overhaul WMT24++ and FLORES evaluation

[thompsonb]

Expand the translation benchmarks and standardize scoring around chrF,
FLORES-200 spBLEU, COMET, and target-language consistency.

• expand WMT24++ from five defaults to all 55 target locales
• expand FLORES from the six-language pair matrix to 219 English-to-X devtest pairs across 220 supported FLORES+ configurations
• store generated language names and FLORES devtest availability in checked-in metadata instead of querying or hardcoding them at runtime
• preserve complete FLORES language and script codes in prepared rows
• reject unknown or devtest-incompatible FLORES configurations
• deduplicate WMT target locales while preserving their requested order
• exclude WMT24++ rows marked as bad source data
• publish WMT preparation output atomically after model prefetch succeeds
• add CLI controls for COMET, spBLEU, and GlotLID prefetching

Replace language-specific SacreBLEU tokenization with NFC-normalized chrF
and FLORES-200 SentencePiece spBLEU scoring. Use sentence chrF as the
dense RL reward and report sentence and corpus chrF/spBLEU separately.
Emit a startup warning because the new spBLEU values are not comparable
with the previous BLEU metrics.

Add configurable target-language consistency scoring:

• use CLD2 for WMT24++ locale-to-language validation
• use GlotLID probabilities for FLORES languages and dialect mappings
• load language-detection backends lazily and reuse them across requests
• report per-rollout scores and per-pair, source, target, and global aggregate metrics
• warn when a language pair falls below the configured consistency threshold
• document CLD2 limitations for regional language variants

Prefetch the shared SentencePiece, COMET, and GlotLID assets for offline
verification. Update translation dependencies, configs, documentation,
and example rollouts for the new scoring contract. Add metadata
generation and mapping-validation utilities plus unit coverage for
preparation, scoring, aggregation, backend loading, and language-code
handling.

BREAKING CHANGE: The wmt_translation per-rollout reward now uses sentence-
level chrF instead of sentence-level BLEU; spBLEU remains a separately
reported metric. The sentence_bleu field and */bleu aggregate keys are
replaced by sentence_chrf, sentence_spbleu, */chrF, and */spBLEU.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.