Skip to content

docs: clarify example.jsonl vs materialized_inputs, apply style guide #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lbliii wants to merge 1 commit into
base: ananthsub/docs-resources-server-checks
Choose a base branch
from
Open

docs: clarify example.jsonl vs materialized_inputs, apply style guide #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/add-benchmark/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Convert your source dataset to Gym JSONL format. Each line must have `responses_

**Data conversion**: Write conversion scripts in the **source repo** (e.g. your dataset repository), not in NeMo-Gym. Prompt files also belong in the source repo. Exception: when there is no external source repo. See `references/patterns.md` § "Data Conversion Script Pattern".

**`example.jsonl`**: Generate 5 entries for smoke testing, committed directly to git in `data/example.jsonl`. Each row must be in materialized Responses format (contain `responses_create_params`) — example datasets are not templated via `prompt_config`, so raw rows won't validate.
**`example.jsonl`**: Generate five entries for smoke testing, committed directly to git in `data/example.jsonl`. Each row must already be in Responses API format (contain `responses_create_params`), because example datasets are not templated through `prompt_config`, so raw rows will not validate. This file is distinct from the `*_materialized_inputs.jsonl` files generated during rollout collection.

**`train`/`validation` datasets**: Upload to the GitLab dataset registry — these must NOT be committed to git.

Expand Down
4 changes: 2 additions & 2 deletions fern/versions/latest/pages/data/prepare-validate.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ ng_prepare_data "+config_paths=[resources_servers/example_multi_step/configs/exa
</Info>

<Warning>
`example.jsonl` must already be in materialized Responses format (each row containing `responses_create_params`). Example-type datasets are not templated via `prompt_config`, so raw rows won't validate.
`example.jsonl` must already be in Responses API format (each row containing `responses_create_params`), because example datasets are not templated through `prompt_config`. Raw, untemplated rows will not validate. This file is distinct from the `*_materialized_inputs.jsonl` files generated during rollout collection.
</Warning>

### Training Preparation
Expand Down Expand Up @@ -388,7 +388,7 @@ datasets:

### License values

`license` is required for `train` and `validation` datasets. It is validated against an allowed set defined in code — see the `license` enum in [`nemo_gym/config_types.py`](https://github.com/NVIDIA-NeMo/Gym/blob/main/nemo_gym/config_types.py).
`license` is required for `train` and `validation` datasets. It is validated against an allowed set defined in code — refer to the `license` enum in [`nemo_gym/config_types.py`](https://github.com/NVIDIA-NeMo/Gym/blob/main/nemo_gym/config_types.py).

---

Expand Down