From 755107a18a401cf215cfac3be9373420318786fd Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Tue, 12 May 2026 11:16:07 +0200 Subject: [PATCH 1/2] Add OpenReward example to list of examples --- docs/source/example_overview.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/source/example_overview.md b/docs/source/example_overview.md index 244643d6b8d..e8a5acde6bf 100644 --- a/docs/source/example_overview.md +++ b/docs/source/example_overview.md @@ -105,6 +105,14 @@ These scripts demonstrate how to train models with [OpenEnv](openenv) environmen | [`examples/scripts/openenv/carla_vlm.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/carla_vlm.py) | GRPO training with CARLA for VLMs with multimodal tool responses (camera images). | | [`examples/scripts/openenv/carla_vlm_gemma.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/carla_vlm_gemma.py) | GRPO training with CARLA for Gemma 4 with multimodal tool responses (camera images). | +### OpenReward Scripts + +These scripts demonstrate how to train models with [OpenReward](openreward) environments using [`GRPOTrainer`]'s `environment_factory`. See the [OpenReward Integration](openreward) guide for more details. + +| File | Description | +| --- | --- | +| [`examples/scripts/openreward/seta.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openreward/seta.py) | GRPO training against the SETA ORS environment on the openreward.ai catalog. | + ## Distributed Training (for scripts) You can run scripts on multiple GPUs with 🤗 Accelerate: From e80402a3f3fea22d69918311c7ea2291c994c5ee Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Tue, 12 May 2026 11:20:33 +0200 Subject: [PATCH 2/2] Update --- docs/source/example_overview.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/docs/source/example_overview.md b/docs/source/example_overview.md index e8a5acde6bf..be91ef6aeb2 100644 --- a/docs/source/example_overview.md +++ b/docs/source/example_overview.md @@ -73,6 +73,7 @@ Scripts are maintained in the [`trl/scripts`](https://github.com/huggingface/trl | [`examples/scripts/online_dpo.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/online_dpo.py) | This script shows how to use the [`experimental.online_dpo.OnlineDPOTrainer`] to fine-tune a model. | | [`examples/scripts/online_dpo_vlm.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/online_dpo_vlm.py) | This script shows how to use the [`experimental.online_dpo.OnlineDPOTrainer`] to fine-tune a a Vision Language Model. | | [`examples/scripts/orpo.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/orpo.py) | This script shows how to use the [`experimental.orpo.ORPOTrainer`] to fine-tune a model to increase helpfulness and harmlessness using the [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset. | +| [`examples/scripts/openreward/seta.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openreward/seta.py) | This script shows how to use the [`GRPOTrainer`] to train a model against the SETA ORS environment on the openreward.ai catalog. See the [OpenReward Integration](openreward) guide for setup and usage. | | [`examples/scripts/ppo/ppo.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/ppo/ppo.py) | This script shows how to use the [`experimental.ppo.PPOTrainer`] to fine-tune a model to improve its ability to continue text with positive sentiment or physically descriptive language. | | [`examples/scripts/ppo/ppo_tldr.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/ppo/ppo_tldr.py) | This script shows how to use the [`experimental.ppo.PPOTrainer`] to fine-tune a model to improve its ability to generate TL;DR summaries. | | [`examples/scripts/prm.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/prm.py) | This script shows how to use the [`experimental.prm.PRMTrainer`] to fine-tune a Process-supervised Reward Model (PRM). | @@ -105,14 +106,6 @@ These scripts demonstrate how to train models with [OpenEnv](openenv) environmen | [`examples/scripts/openenv/carla_vlm.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/carla_vlm.py) | GRPO training with CARLA for VLMs with multimodal tool responses (camera images). | | [`examples/scripts/openenv/carla_vlm_gemma.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/carla_vlm_gemma.py) | GRPO training with CARLA for Gemma 4 with multimodal tool responses (camera images). | -### OpenReward Scripts - -These scripts demonstrate how to train models with [OpenReward](openreward) environments using [`GRPOTrainer`]'s `environment_factory`. See the [OpenReward Integration](openreward) guide for more details. - -| File | Description | -| --- | --- | -| [`examples/scripts/openreward/seta.py`](https://github.com/huggingface/trl/blob/main/examples/scripts/openreward/seta.py) | GRPO training against the SETA ORS environment on the openreward.ai catalog. | - ## Distributed Training (for scripts) You can run scripts on multiple GPUs with 🤗 Accelerate: