feat: add OpenAI format dataset for SFT

[AtsunoriFujita]

What does this PR do ?

This PR enables using the OpenAI format dataset from a json/jsonl when running SFT.

Issues

List issues that this PR closes (syntax):

Usage

Modify examples/configs/sft.yaml

 data:
  max_input_seq_length: ${policy.max_total_sequence_length}
  dataset_name: "openai_format"
  add_bos: true
  add_eos: true
  add_generation_prompt: false
  train_data_path: "datasets/train.json"
  val_data_path: "datasets/valid.json"
  chat_key: "messages"  # set chat_key like "messages" and "conversations"
  system_key: null  # (optional) set system_key (system_prompt key name) if dataset has
  system_prompt: null  # (optional) set system_prompt, e.g., "You are a helpful assistant." If system_key is not null, it will be prioritized.

Run SFT job

uv run examples/run_sft.py --config examples/configs/sft.yaml

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[ashors1]

Thanks for the PR! Could we add some documentation on this class and how it differs from prompt_response_dataset.py? Do you think it would make sense to consider merging this class with PromptResponseDataset?

[AtsunoriFujita]

Hi @ashors1, added docstrings and unit tests.

Could we add some documentation on this class and how it differs from prompt_response_dataset.py?

The openai_format deals with a single key (like messages or conversations) and single or multiple turns (user, assistant), similar to HuggingFaceTB/smoltalk.

[ashors1]

Looks good! Thank you for the contribution!

[terrykong]

@AtsunoriFujita could you run pre-commit on your change? It fails our linter job

[AtsunoriFujita]

Hi @terrykong, applied pre-commit.

[terrykong]

@AtsunoriFujita do you mind putting an example run command in the description so that users finding this PR can learn how to use this?

[AtsunoriFujita]

@terrykong, thank you. I added it.