Skip to content

docs: Small update to sft documentation #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
parthchadha merged 5 commits into from
Mar 21, 2025
Merged

docs: Small update to sft documentation #12

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
24dfe5e
small update to sft doc
ashors1 Mar 21, 2025
25b0e9e
fix example run command
ashors1 Mar 21, 2025
7821fc3
fix other command
ashors1 Mar 21, 2025
495bf2c
change example to single-gpu
ashors1 Mar 21, 2025
a6c089f
Merge branch 'main' into ashors/sft-doc
parthchadha Mar 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/guides/sft.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ python examples/run_sft.py \
## Datasets

SFT datasets in Reinforcer are encapsulated using classes. Each SFT data class is expected to have the following attributes:
- `formatted_ds`: The dictionary of formatted datasets. This dictionary should contain `train` and `validation` splits, and each split should conform to the format described below.
- `task_spec`: The `TaskDataSpec` for this dataset. This should specify the name you choose for this dataset as well as the `custom_template` for this dataset. More on custom templates below.
1. `formatted_ds`: The dictionary of formatted datasets. This dictionary should contain `train` and `validation` splits, and each split should conform to the format described below.
2. `task_spec`: The `TaskDataSpec` for this dataset. This should specify the name you choose for this dataset as well as the `custom_template` for this dataset. More on custom templates below.

SFT datasets are expected to follow the HuggingFace chat format. Refer to the [chat dataset document](../design_docs/chat_datasets.md) for details. If your data is not in the correct format, simply write a preprocessing script to convert the data into this format. [data/hf_datasets/squad.py](../../nemo_reinforcer/data/hf_datasets/squad.py) has an example:

Expand Down Expand Up @@ -62,4 +62,7 @@ task_spec = TaskDataSpec(
)
```

By default, NeMo-Reinforcer has support for `Squad` and `OpenAssistant` datasets. If you would like to use a custom dataset, create a new dataset class with the expected attributes.
By default, NeMo-Reinforcer has support for `Squad` and `OpenAssistant` datasets. Both of these datasets are downloaded from HuggingFace and preprocessed on-the-fly, so there's no need to provide a path to any datasets on disk.

Adding a new dataset is a straightforward process.
As long as your custom dataset has the `formatted_ds` and `task_spec` attributes described above, it can serve as a drop-in replacement for Squad and OpenAssistant.