fix(longmt_eval): use valid 'Apache 2.0' license literal - #1574
Closed
wprazuch wants to merge 1 commit into
Closed
Conversation
longmt_eval.yaml and longmt_pg19.yaml set `license: Apache-2.0`, which is not a valid DatasetConfig.license literal — the allowed value is 'Apache 2.0' (with a space), as already used in longmt_wmt24pp.yaml. Loading either config through get_global_config_dict raises a pydantic ValidationError. Does not by itself fix the longmt_eval CI data-validation failure (missing data/example_metrics.json) — see PR description. Signed-off-by: Wojciech Prazuch <wprazuch@nvidia.com>
Contributor
Author
|
Superseded by #1559, which landed the identical license fix ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
resources_servers/longmt_eval/configs/longmt_eval.yamlandlongmt_pg19.yamlsetlicense: Apache-2.0(hyphen). That is not a validDatasetConfig.licenseliteral — theallowed value is
Apache 2.0(with a space), which the siblinglongmt_wmt24pp.yamlalreadyuses correctly. This aligns all three configs.
Loading either of the two affected configs through
get_global_config_dict()currently raisesa pydantic
ValidationError:longmt_evaldata-validation CIWhile investigating the
Testjob failure that shows up on every open PR, I found twoseparate issues in
longmt_eval(added in #1458):Apache-2.0license literal (a latent config-parse bug).Testjob's data validation (_validate_data_singleincli.py)requires
resources_servers/longmt_eval/data/example_metrics.json, but the server only shipsexample_rollouts_agent_metrics.json. The missing file is what actually reds theTestjobfor all PRs (it validates all 114 servers), e.g. on fix(config): actionable error for unknown server cross-references #1561.
@jeffwillette — could you regenerate and commit
data/example_metrics.jsonforlongmt_eval?It comes from:
ng_prepare_data "+config_paths=[resources_servers/longmt_eval/configs/longmt_eval.yaml]" \ +output_dirpath=data/longmt_eval \ +mode=example_validationThat step needs the server's GPU/SEGALE/COMET environment, which is why it's best done on your
side. Happy to fold it into this PR or you can land it separately — whichever is easier.
Testing
Apache 2.0validates againstDatasetConfig; pre-commit clean (verified-flag + README hookspass with no changes).