Move truncate_dataset to experimental

[albertvillanova]

Move truncate_dataset to experimental.

Follow-up to:

• Remove post-collation truncation from SFT #5359

This PR moves the truncate_dataset utility function from trl/data_utils.py to trl/experimental/utils.py and updates all imports and tests accordingly. This change helps modularize experimental utilities and keeps the codebase organized. The associated tests for truncate_dataset have also been relocated to the experimental test suite.

Changes

Refactoring and Code Organization:

• Moved the truncate_dataset function from trl/data_utils.py to trl/experimental/utils.py, including its full implementation and docstring. All references in the codebase now import it from the new location.

• Removed truncate_dataset from the main trl/__init__.py and its imports, as it is now considered an experimental utility.

Testing updates:

• Moved the tests for truncate_dataset from tests/test_data_utils.py to tests/experimental/test_utils.py, ensuring that all test coverage remains intact in the new experimental context.

Dependency and Import Adjustments:

• Updated imports in trl/experimental/utils.py to include necessary dependencies for the new location of truncate_dataset (such as pyarrow, DatasetType, and _get_dataset_format).

---

Note

Medium Risk
Primarily a refactor, but it changes the public import path by removing truncate_dataset from the top-level/stable API, which can break downstream imports and any code relying on it.

Overview
Moves truncate_dataset out of the stable trl.data_utils API into trl.experimental.utils, including relocating its implementation and required pyarrow/format-handling imports.

Updates internal usage (notably experimental/gold_trainer.py) and test coverage by migrating truncate_dataset tests from tests/test_data_utils.py into tests/experimental/test_utils.py, and removes truncate_dataset from the public trl/__init__.py exports.

^{Written by Cursor Bugbot for commit 46fd46a. This will update automatically on new commits. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.