Skip to content

fix(longmt_eval): use valid 'Apache 2.0' license literal - #1574

Closed
wprazuch wants to merge 1 commit into
mainfrom
wprazuch/fix-longmt-eval-license
Closed

fix(longmt_eval): use valid 'Apache 2.0' license literal#1574
wprazuch wants to merge 1 commit into
mainfrom
wprazuch/fix-longmt-eval-license

Conversation

@wprazuch

Copy link
Copy Markdown
Contributor

What

resources_servers/longmt_eval/configs/longmt_eval.yaml and longmt_pg19.yaml set
license: Apache-2.0 (hyphen). That is not a valid DatasetConfig.license literal — the
allowed value is Apache 2.0 (with a space), which the sibling longmt_wmt24pp.yaml already
uses correctly. This aligns all three configs.

Loading either of the two affected configs through get_global_config_dict() currently raises
a pydantic ValidationError:

'Apache 2.0':  OK
'Apache-2.0':  REJECTED (ValidationError)

⚠️ This does not fully green the longmt_eval data-validation CI

While investigating the Test job failure that shows up on every open PR, I found two
separate issues in longmt_eval (added in #1458):

  1. This PR — the invalid Apache-2.0 license literal (a latent config-parse bug).
  2. Still outstanding — the Test job's data validation (_validate_data_single in cli.py)
    requires resources_servers/longmt_eval/data/example_metrics.json, but the server only ships
    example_rollouts_agent_metrics.json. The missing file is what actually reds the Test job
    for all PRs (it validates all 114 servers), e.g. on fix(config): actionable error for unknown server cross-references #1561.

@jeffwillette — could you regenerate and commit data/example_metrics.json for longmt_eval?
It comes from:

ng_prepare_data "+config_paths=[resources_servers/longmt_eval/configs/longmt_eval.yaml]" \
    +output_dirpath=data/longmt_eval \
    +mode=example_validation

That step needs the server's GPU/SEGALE/COMET environment, which is why it's best done on your
side. Happy to fold it into this PR or you can land it separately — whichever is easier.

Testing

Apache 2.0 validates against DatasetConfig; pre-commit clean (verified-flag + README hooks
pass with no changes).

longmt_eval.yaml and longmt_pg19.yaml set `license: Apache-2.0`, which is not
a valid DatasetConfig.license literal — the allowed value is 'Apache 2.0' (with
a space), as already used in longmt_wmt24pp.yaml. Loading either config through
get_global_config_dict raises a pydantic ValidationError.

Does not by itself fix the longmt_eval CI data-validation failure (missing
data/example_metrics.json) — see PR description.

Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@wprazuch

Copy link
Copy Markdown
Contributor Author

Superseded by #1559, which landed the identical license fix (Apache-2.0Apache 2.0 in both configs) and added the missing data/example_metrics.json that was the actual CI blocker. Closing — nothing left to do here. Thanks @jeffwillette!

@wprazuch wprazuch closed this Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant