Require datasets>=4.7.0 for Json dtype to prevent insertion of None values

[albertvillanova]

Use datasets Json dtype to avoid the insertion of None values.

• Note that this applies not only to tool-calling datasets, but also to multi-modal datasets. See:
  • BUGfix: Fix image_grid_thw IndexError in GRPOTrainer with Multimodal Models (Qwen3-VL) due to None Values in Chat Content #5364

CC: @SolarWindRider, @qgallouedec

This PR updates dataset handling to improve compatibility with mixed data types and simplifies preprocessing in several trainer modules. The most significant changes include upgrading the datasets library to support the Json new feature, using the on_mixed_types="use_json" option when loading datasets, and removing the remove_none_values cleaning step from all trainer classes: DPO, Reward, SFT.

Related to:

• Suggest the Json() type for tool calling dataset format #5307
• Fix datasets version supporting Json dtype in docs about tool calling dataset format #5310
• Align docs about tool calling in trainers with dataset format #5311
• Update tool handling to support JSON string schemas in trainers #5118

Changes

Dataset compatibility and preprocessing improvements:

• Upgraded the datasets library version requirement from >=3.0.0 to >=4.7.0 in pyproject.toml to enable support for Json type fields and the on_mixed_types="use_json" option.
• Updated dataset loading calls in scripts/generate_toolcall_dataset.py to use on_mixed_types="use_json", ensuring robust handling of mixed-type columns.

Trainer codebase simplification:

• Removed all imports and usage of the remove_none_values utility function from DPOTrainer, RewardTrainer, and SFTTrainer, eliminating a redundant data cleaning step now handled by the updated dataset loading logic.

---

Note

Medium Risk
Medium risk because it bumps the minimum datasets version and changes trainer preprocessing behavior, which could affect users with older/saved datasets that still contain injected None values.

Overview
Switches TRL to rely on datasets v4.7.0+ Json dtype handling for nested/mixed columns (via on_mixed_types="use_json" in dataset generation scripts) to prevent Arrow/Parquet from inserting spurious None values.

Removes the trainers’ automatic remove_none_values transform in SFTTrainer, DPOTrainer, and RewardTrainer, and documents the new expectation + manual workaround in MIGRATION.md for users with legacy datasets that still contain None.

^{Written by Cursor Bugbot for commit 27ba622. This will update automatically on new commits. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[qgallouedec]

here it's because we mix turns with and without "thinking" 👍

[qgallouedec]

here I'm not sure if it's need, json.dumps converts to str

[albertvillanova]

Oops, I forgot to remove json.dumps. 😅 I'm pushing a new commit for this.

[qgallouedec]

Looks good overall, just a few minor suggestions and comments.

I think bumping datasets to 4.7 is safe; I can’t think of any case where it would affect users.

For clarity in the release notes, could you update the title to explicitly mention this version bump?