From a13f2913c03ff64fbfc87a3d9e2b65227f502806 Mon Sep 17 00:00:00 2001
From: sergiopaniego <sergiopaniegoblanco@gmail.com>
Date: Thu, 8 Oct 2026 16:22:58 +0200
Subject: [PATCH 1/2] Docs: framework-neutral Training overview and harness
 pages (#1296)

---
 docs/source/_toctree.yml                    |  6 +-
 docs/source/guides/async-sync.md            |  2 +-
 docs/source/guides/rewards.md               |  2 +-
 docs/source/guides/rl-integration.md        | 69 +++++++++------------
 docs/source/guides/task-api.md              |  2 +-
 docs/source/tutorials/browsergym-harness.md |  2 +-
 docs/source/tutorials/harbor-harness.md     |  7 ++-
 docs/source/tutorials/harnesses.md          |  8 +--
 docs/source/tutorials/index.md              |  4 +-
 docs/source/tutorials/rubrics.md            |  2 +-
 10 files changed, 47 insertions(+), 57 deletions(-)

diff --git a/docs/source/_toctree.yml b/docs/source/_toctree.yml
index 888c088857..43b2192628 100644
--- a/docs/source/_toctree.yml
+++ b/docs/source/_toctree.yml
@@ -11,8 +11,6 @@
     title: Auto-Discovery
   - local: guides/catalog-discovery
     title: Catalog Discovery
-  - local: guides/rl-integration
-    title: RL Training
   - local: guides/harbor-provider-qualification
     title: Harbor Spec
   - local: guides/rewards
@@ -51,6 +49,8 @@
       title: Your First Environment
     title: Basics
   - sections:
+    - local: guides/rl-integration
+      title: Overview
     - local: tutorials/end-to-end-walkthrough
       title: Train a Reasoning Model
     - local: tutorials/wordle-grpo
@@ -64,7 +64,7 @@
     - local: tutorials/harnesses
       title: Overview
     - local: tutorials/browsergym-harness
-      title: "White-box: Train with TRL's Loop (BrowserGym)"
+      title: "White-box: Train a Web Agent (BrowserGym)"
     - local: tutorials/harbor-harness
       title: "Black-box: Train Real Agents (Harbor)"
     - local: tutorials/claude-code-harness
diff --git a/docs/source/guides/async-sync.md b/docs/source/guides/async-sync.md
index 993b06ea80..04bf1a2ecd 100644
--- a/docs/source/guides/async-sync.md
+++ b/docs/source/guides/async-sync.md
@@ -70,5 +70,5 @@ asyncio.run(main())
 
 ## Next Steps
 
-- [RL Framework Integration](rl-integration.md) - Using async with RL training
+- [Training with OpenEnv](rl-integration.md) - Using async with RL training
 - [Auto-Discovery](auto-discovery.md) - Discover and load environments
diff --git a/docs/source/guides/rewards.md b/docs/source/guides/rewards.md
index 4036fc0f9a..6b0df2fe60 100644
--- a/docs/source/guides/rewards.md
+++ b/docs/source/guides/rewards.md
@@ -89,5 +89,5 @@ Per-turn progress helps the agent explore, but the terminal outcome still domina
 
 - [Rubrics tutorial](../tutorials/rubrics.md) — full API walkthrough with composable examples.
 - [RFC 004](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md) — design rationale.
-- [RL Framework Integration](rl-integration.md) — consume the reward signal in a training loop.
+- [Training with OpenEnv](rl-integration.md) — consume the reward signal in a training loop.
 - [Concepts](concepts.md) — where the rubric plugs in.
diff --git a/docs/source/guides/rl-integration.md b/docs/source/guides/rl-integration.md
index f9052cc464..f02317ae46 100644
--- a/docs/source/guides/rl-integration.md
+++ b/docs/source/guides/rl-integration.md
@@ -1,41 +1,39 @@
-# RL Framework Integration
+# Training with OpenEnv
 
-> [!NOTE]
-> This page is still being filled in. TRL integration is covered below; torchforge and SkyRL integrations are planned.
+OpenEnv is a contract between environments and the code that uses them, not a trainer. An environment exposes `reset()`, `step()` and `state()` (and its tools over MCP), runs as a service anywhere, and computes its own reward, so any training framework that can call it can train on it. This page maps the ways to train on OpenEnv environments and the frameworks that support it.
 
-Use OpenEnv with popular RL frameworks like TRL, torchforge, and SkyRL.
+## Ways to Train
 
-## Overview
+| You want to | What the environment gives | Worked examples |
+|---|---|---|
+| Train a model with RL, with the trainer running the episode | Observations, tools and a reward. The trainer generates every turn, so it has the tokens and logprobs. This is the white-box path | [Wordle](../tutorials/wordle-grpo), [a reasoning model](../tutorials/end-to-end-walkthrough), [web tasks in BrowserGym](../tutorials/browsergym-harness) with TRL, [2048](../tutorials/rl-training-2048) with Unsloth |
+| Train the model behind a real agent (OpenCode, Claude Code, Codex, …) | Harbor runs the agent on a task, captures every model call, and returns a `TrainingTrace` with token ids, logprobs and the verifier's reward. This is the black-box path | [Harbor](../tutorials/harbor-harness), trained with TRL's `AsyncGRPOTrainer` |
+| Warm-start a model with supervised data | `openenv collect` runs a teacher in the environment and saves reward-labeled rollouts as a dataset | [Collecting rollouts for SFT](../tutorials/sft-warmup) |
+| Evaluate, not train | Scores from the environment's rubric | [Inspect AI](../tutorials/evaluation-inspect), [an agent inside the environment](../tutorials/claude-code-harness) |
 
-OpenEnv environments are designed to integrate seamlessly with RL training frameworks. The standard `step()`, `reset()`, `state()` API makes it easy to use environments in training loops.
+None of these paths ties the environment to a framework. `openenv.core` does not depend on any trainer, and a `TrainingTrace` or a collected dataset is plain data that any trainer can read. The worked examples use the frameworks named above because that is where the examples exist today. [Harnesses in OpenEnv](../tutorials/harnesses) compares the white-box and black-box paths in more detail.
 
-## TRL Integration
+## Integrations
 
-[TRL (Transformer Reinforcement Learning)](https://huggingface.co/docs/trl) is the recommended framework for training language models with RL.
+These frameworks and platforms train on OpenEnv environments. If your project supports OpenEnv, open a PR to add it here and to the [README](https://github.com/huggingface/OpenEnv#integrations).
 
-```python
-from trl import GRPOTrainer
-from openenv import AutoEnv, AutoAction
-
-env = AutoEnv.from_env("textarena")
-TextAction = AutoAction.from_env("textarena")
-
-# Use with TRL's GRPO trainer
-trainer = GRPOTrainer(
-    model=model,
-    reward_model=reward_model,
-    # ... TRL config
-)
-```
+| Framework | Example |
+|---|---|
+| [TRL](https://huggingface.co/docs/trl) | [OpenEnv integration guide](https://huggingface.co/docs/trl/openenv): `environment_factory` with `GRPOTrainer`, several environments at once, and harness training through Harbor |
+| [torchforge](https://meta-pytorch.org/torchforge/) | [GRPO on BlackJack](https://github.com/huggingface/OpenEnv/tree/main/examples/grpo_blackjack) |
+| [Unsloth](https://unsloth.ai) | [2048 with gpt-oss](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/OpenEnv_gpt_oss_(20B)_Reinforcement_Learning_2048_Game.ipynb) |
+| [SkyRL](https://github.com/NovaSky-AI/SkyRL) | [Training on OpenEnv environments](https://skyrl.readthedocs.io/en/latest/examples/openenv.html) |
+| [ART](https://art.openpipe.ai) | [OpenEnv integration](https://art.openpipe.ai/integrations/openenv-integration) |
+| [Oumi](https://github.com/oumi-ai/oumi) | [OpenEnv GRPO notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20OpenEnv%20GRPO%20with%20trl.ipynb) |
+| [Lightning AI](https://lightning.ai) | [OpenEnv templates](https://lightning.ai/templates?section=featured&query=openenv) |
+| [Miles](https://github.com/radixark/miles) | [GRPO on Terminal-Bench 2](https://github.com/radixark/miles/tree/main/examples/experimental/openenv) |
 
-See the [Wordle with GRPO](../tutorials/wordle-grpo.md) tutorial for a complete example.
+## Your Own Training Loop
 
-## Generic Training Loop
-
-For custom training setups:
+Every integration above comes down to the same calls. To plug OpenEnv into a framework that has no integration yet, drive the client from its rollout code:
 
 ```python
-from openenv import AutoEnv, AutoAction
+from openenv import AutoAction, AutoEnv
 
 env = AutoEnv.from_env("my-env")
 Action = AutoAction.from_env("my-env")
@@ -43,19 +41,10 @@ Action = AutoAction.from_env("my-env")
 with env.sync() as client:
     for episode in range(num_episodes):
         result = client.reset()
-
-        while not result.terminated:
-            # Get action from your policy
-            action = policy(result.observation)
-
-            # Take step
+        while not result.done:
+            action = policy(result.observation)  # your model picks the next action
             result = client.step(action)
-
-            # Update policy with reward
-            policy.update(result.reward)
+            # result.reward is the environment's reward for this step
 ```
 
-## Next Steps
-
-- [Reward Design](rewards.md) - Design effective reward functions
-- [Wordle with GRPO](../tutorials/wordle-grpo.md) - Complete TRL example
+The [Task API](task-api) lets a trainer list an environment's tasks and pick which one each episode runs, and [Rewards](rewards) covers how environments compute the reward.
diff --git a/docs/source/guides/task-api.md b/docs/source/guides/task-api.md
index 82b4580655..f379d468c2 100644
--- a/docs/source/guides/task-api.md
+++ b/docs/source/guides/task-api.md
@@ -272,6 +272,6 @@ Before shipping a dataset-backed environment:
 ## Related Reading
 
 - [Concepts](concepts.md)
-- [RL Training](rl-integration.md)
+- [Training with OpenEnv](rl-integration.md)
 - [Core API](../reference/core.md)
 - [CLI reference](../reference/cli.md)
diff --git a/docs/source/tutorials/browsergym-harness.md b/docs/source/tutorials/browsergym-harness.md
index a27bb80940..7e927ddcf6 100644
--- a/docs/source/tutorials/browsergym-harness.md
+++ b/docs/source/tutorials/browsergym-harness.md
@@ -1,6 +1,6 @@
 # White-Box: Train a Web Agent on BrowserGym
 
-This tutorial trains a model to complete web tasks in [BrowserGym](../environments/browsergym) with TRL's `GRPOTrainer`. It is the **white-box** harness path: TRL owns the agent loop. It samples every turn, parses the tool calls, runs them against the environment and feeds the results back, so it has the token ids and logprobs of everything the model generated without capturing anything. [Harnesses in OpenEnv](harnesses) compares it to the other paths.
+This tutorial trains a model to complete web tasks in [BrowserGym](../environments/browsergym) with TRL's `GRPOTrainer`. It is the **white-box** harness path: the trainer, here TRL, owns the agent loop. It samples every turn, parses the tool calls, runs them against the environment and feeds the results back, so it has the token ids and logprobs of everything the model generated without capturing anything. [Harnesses in OpenEnv](harnesses) compares it to the other paths, and [Training with OpenEnv](../guides/rl-integration) lists the other frameworks that train on OpenEnv environments.
 
 The full script is TRL's [`examples/grpo_browsergym/grpo_browsergym.py`](https://github.com/huggingface/trl/tree/main/examples/grpo_browsergym). This page walks through its parts.
 
diff --git a/docs/source/tutorials/harbor-harness.md b/docs/source/tutorials/harbor-harness.md
index 7455e3e3e5..e53b0fb057 100644
--- a/docs/source/tutorials/harbor-harness.md
+++ b/docs/source/tutorials/harbor-harness.md
@@ -1,6 +1,6 @@
 # Black-Box: Train Real Agents with Harbor
 
-Some agents can't be reimplemented as a TRL tool loop. A coding agent such as OpenCode, Claude Code or Codex has its own planner, tools, context management and stop condition, and the point is to train the model that drives *that* agent. This is the **black-box** harness path: the agent owns its loop, and OpenEnv records every model call it makes. Episodes are tasks: an instruction goes in, the agent works on its own, and a verifier checks the result. If the job is a conversation instead, or the agent has to use your environment's own tools, see [Evaluate Claude Code in an Environment](claude-code-harness). [Harnesses in OpenEnv](harnesses) compares the paths.
+Some agents can't be reimplemented as a trainer's tool loop. A coding agent such as OpenCode, Claude Code or Codex has its own planner, tools, context management and stop condition, and the point is to train the model that drives *that* agent. This is the **black-box** harness path: the agent owns its loop, and OpenEnv records every model call it makes. Episodes are tasks: an instruction goes in, the agent works on its own, and a verifier checks the result. If the job is a conversation instead, or the agent has to use your environment's own tools, see [Evaluate Claude Code in an Environment](claude-code-harness). [Harnesses in OpenEnv](harnesses) compares the paths.
 
 OpenEnv does this through [Harbor](../environments/harbor), which supplies the tasks, the sandboxes, the agents and the verifiers. One `harbor_env` server runs any of 16 validated agents on any Harbor dataset, in any of Harbor's sandbox backends, and its capture proxy returns the token ids and logprobs of each model call together with the task's reward.
 
@@ -20,9 +20,9 @@ openenv harbor rollout \
 
 Against a hosted provider you get an evaluation rollout: the reward and the full trace. Against vLLM or SGLang you also get the token ids and logprobs that training needs. `openenv harbor info` checks first which agents, sandboxes and capture level this machine can use.
 
-## Train With TRL
+## Train on the Captures
 
-TRL trains on these rollouts with `AsyncGRPOTrainer` and a `HarnessRolloutWorker`: the agent runs each task to completion, TRL reads back the captured `TrainingTrace`, and the new weights go back into the same vLLM server.
+Each rollout comes back as a `TrainingTrace`: the token ids, logprobs and loss masks of every model call, plus the task's reward. It doesn't depend on a trainer, so any framework can train on it. TRL has a worked example: `AsyncGRPOTrainer` with a `HarnessRolloutWorker` runs each task to completion, reads back the trace, and syncs the new weights into the same vLLM server.
 
 - [TRL's guide to training on harnesses](https://huggingface.co/docs/trl/main/en/openenv#training-on-harnesses-training-real-coding-agents-harbor) explains how it works and how to wire it.
 - [`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) is the complete script, with a Hugging Face Jobs launcher.
@@ -32,3 +32,4 @@ TRL trains on these rollouts with `AsyncGRPOTrainer` and a `HarnessRolloutWorker
 
 - [The Harbor environment](../environments/harbor): supported agents and sandboxes, the web UI, deploying to Spaces and troubleshooting.
 - [TRL's Harbor integration](https://huggingface.co/docs/trl/harbor) for the white-box path on Harbor tasks, where TRL's own loop works through a Harbor task suite.
+- [Training with OpenEnv](../guides/rl-integration) for the other ways to train and the frameworks that support them.
diff --git a/docs/source/tutorials/harnesses.md b/docs/source/tutorials/harnesses.md
index ecd72e2425..bf1476807f 100644
--- a/docs/source/tutorials/harnesses.md
+++ b/docs/source/tutorials/harnesses.md
@@ -4,18 +4,18 @@ A harness is the loop around a model: it sends the conversation to the model, ru
 
 | | White-box | Black-box | Agent inside the environment |
 |---|---|---|---|
-| Who owns the loop | TRL (`environment_factory`) | A real agent (OpenCode, Claude Code, Codex, …) | A real agent (Claude Code in the tutorial) |
+| Who owns the loop | The trainer (for example TRL's `environment_factory`) | A real agent (OpenCode, Claude Code, Codex, …) | A real agent (Claude Code in the tutorial) |
 | What the environment provides | Tools and a reward for the model being trained | Tasks: an instruction, a sandbox, and a verifier | Its own tools and state, injected into the agent over MCP |
 | Who is on the other side | Nobody, the model works through the task | Nobody, the agent runs to completion | A user, simulated or real, over several turns |
-| Trains a model | Yes | Yes, from captured token ids and logprobs (against vLLM or SGLang) | No, nothing is captured |
+| Trains a model | Yes | Yes, from the captured token ids and logprobs in a `TrainingTrace` (against vLLM or SGLang) | No, nothing is captured |
 | Serves real users | No | No | Yes, over `WS /harness` |
 | Example | Web tasks in BrowserGym | "Fix this repository", "analyze these files" | Customer service on τ²-bench |
 | Start here | [BrowserGym](browsergym-harness) | [Harbor](harbor-harness) | [Claude Code on τ²-bench](claude-code-harness) |
 
 ## Which One to Use
 
-- **You want to train a model on your environment and the loop can be simple.** Use the white-box path. TRL's `GRPOTrainer` runs the tool loop, so token ids and logprobs come for free, and the environment only supplies tools and a reward. [Wordle](wordle-grpo) is the same pattern on a text game.
-- **The job is a task that an agent does on its own, and a verifier checks the result.** Use Harbor, to train the model behind the agent or to evaluate it. The agent keeps its own planner, tools and context management in a sandbox, and a proxy records every model call it makes.
+- **You want to train a model on your environment and the loop can be simple.** Use the white-box path. The trainer runs the tool loop, so token ids and logprobs come for free, and the environment only supplies tools and a reward. The BrowserGym tutorial does it with TRL's `GRPOTrainer`, and [Training with OpenEnv](../guides/rl-integration) lists the other frameworks that train on OpenEnv environments.
+- **The job is a task that an agent does on its own, and a verifier checks the result.** Use Harbor, to train the model behind the agent or to evaluate it. The agent keeps its own planner, tools and context management in a sandbox, and a proxy records every model call it makes. The resulting `TrainingTrace` is plain data that doesn't depend on a trainer, and TRL's `AsyncGRPOTrainer` has a worked example.
 - **The job is a conversation, and the agent has to use your environment's tools.** Use `HarnessEnvironment` from [RFC 005](https://github.com/huggingface/OpenEnv/blob/main/rfcs/005-agentic-harnesses.md), to evaluate the agent against a simulated user or to serve it to real ones. The environment injects its tools, answers between turns and scores the conversation with a rubric.
 
 ### Harbor or `HarnessEnvironment`?
diff --git a/docs/source/tutorials/index.md b/docs/source/tutorials/index.md
index a8b8705825..d0c34e540e 100644
--- a/docs/source/tutorials/index.md
+++ b/docs/source/tutorials/index.md
@@ -3,8 +3,8 @@
 Choose a learning path from the sidebar:
 
 - **Basics:** start with [Hello World](openenv-tutorial) and build [Your First Environment](../guides/first-environment).
-- **Training:** train a [reasoning model](end-to-end-walkthrough), use [TRL](wordle-grpo) or [Unsloth](rl-training-2048), or collect rollouts for [SFT](sft-warmup).
-- **Harnesses:** [pick a path](harnesses): train with TRL's own loop on [BrowserGym](browsergym-harness) (white-box), train real agents through [Harbor](harbor-harness) (black-box), or [evaluate Claude Code inside an environment](claude-code-harness). The [OpenCode](opencode-agent-grpo) and [Pi](pi-agent-grpo) tutorials are deprecated.
+- **Training:** [the ways to train and the supported frameworks](../guides/rl-integration), then train a [reasoning model](end-to-end-walkthrough), use [TRL](wordle-grpo) or [Unsloth](rl-training-2048), or collect rollouts for [SFT](sft-warmup).
+- **Harnesses:** [pick a path](harnesses): train with the trainer's own loop on [BrowserGym](browsergym-harness) (white-box), train real agents through [Harbor](harbor-harness) (black-box), or [evaluate Claude Code inside an environment](claude-code-harness). The [OpenCode](opencode-agent-grpo) and [Pi](pi-agent-grpo) tutorials are deprecated.
 - **Evals:** follow [Evaluating with Environments](evaluation-inspect).
 
 For [MCP environments](mcp-environment) and [rubrics](rubrics), see Concepts in the sidebar.
diff --git a/docs/source/tutorials/rubrics.md b/docs/source/tutorials/rubrics.md
index 575050755c..3fcb61df72 100644
--- a/docs/source/tutorials/rubrics.md
+++ b/docs/source/tutorials/rubrics.md
@@ -379,4 +379,4 @@ The same rubric object used to compute training rewards doubles as the eval metr
 - **Real-world trajectory example** — walk through `envs/chess_env/server/rubrics.py` and `chess_environment.py` to see `ExponentialDiscountingTrajectoryRubric` wired into a game environment.
 - **Design details** — [RFC 004](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md) covers the rationale for the composable API and the "rewards inside the environment" invariant.
 - **Reward design basics** — the [Reward Design](../guides/rewards.md) guide covers sparse-vs-dense signals and common pitfalls that still apply on top of any rubric composition.
-- **Training loop integration** — see the [RL Framework Integration](../guides/rl-integration.md) guide and the [TRL OpenEnv integration guide](https://huggingface.co/docs/trl/main/en/openenv) for the recommended `environment_factory` pattern.
+- **Training loop integration** — see [Training with OpenEnv](../guides/rl-integration.md) and the [TRL OpenEnv integration guide](https://huggingface.co/docs/trl/main/en/openenv) for the recommended `environment_factory` pattern.

From 8a558dd92d4b3a46b3c9883523c5f579cfce27f4 Mon Sep 17 00:00:00 2001
From: sergiopaniego <sergiopaniegoblanco@gmail.com>
Date: Thu, 8 Oct 2026 16:35:32 +0200
Subject: [PATCH 2/2] Docs: rename guides/rl-integration to guides/training to
 match the page title

Adds a redirect so existing links to guides/rl-integration keep working.
---
 docs/source/_redirects.yml                            | 1 +
 docs/source/_toctree.yml                              | 2 +-
 docs/source/guides/async-sync.md                      | 2 +-
 docs/source/guides/rewards.md                         | 2 +-
 docs/source/guides/task-api.md                        | 2 +-
 docs/source/guides/{rl-integration.md => training.md} | 0
 docs/source/tutorials/browsergym-harness.md           | 2 +-
 docs/source/tutorials/harbor-harness.md               | 2 +-
 docs/source/tutorials/harnesses.md                    | 2 +-
 docs/source/tutorials/index.md                        | 2 +-
 docs/source/tutorials/rubrics.md                      | 2 +-
 examples/echo_world_model/backends/trl.md             | 2 +-
 examples/rubrics.ipynb                                | 2 +-
 13 files changed, 12 insertions(+), 11 deletions(-)
 create mode 100644 docs/source/_redirects.yml
 rename docs/source/guides/{rl-integration.md => training.md} (100%)

diff --git a/docs/source/_redirects.yml b/docs/source/_redirects.yml
new file mode 100644
index 0000000000..004c1bcc7b
--- /dev/null
+++ b/docs/source/_redirects.yml
@@ -0,0 +1 @@
+guides/rl-integration: guides/training
diff --git a/docs/source/_toctree.yml b/docs/source/_toctree.yml
index 43b2192628..404561f83c 100644
--- a/docs/source/_toctree.yml
+++ b/docs/source/_toctree.yml
@@ -49,7 +49,7 @@
       title: Your First Environment
     title: Basics
   - sections:
-    - local: guides/rl-integration
+    - local: guides/training
       title: Overview
     - local: tutorials/end-to-end-walkthrough
       title: Train a Reasoning Model
diff --git a/docs/source/guides/async-sync.md b/docs/source/guides/async-sync.md
index 04bf1a2ecd..c042da1737 100644
--- a/docs/source/guides/async-sync.md
+++ b/docs/source/guides/async-sync.md
@@ -70,5 +70,5 @@ asyncio.run(main())
 
 ## Next Steps
 
-- [Training with OpenEnv](rl-integration.md) - Using async with RL training
+- [Training with OpenEnv](training.md) - Using async with RL training
 - [Auto-Discovery](auto-discovery.md) - Discover and load environments
diff --git a/docs/source/guides/rewards.md b/docs/source/guides/rewards.md
index 6b0df2fe60..00e055d00b 100644
--- a/docs/source/guides/rewards.md
+++ b/docs/source/guides/rewards.md
@@ -89,5 +89,5 @@ Per-turn progress helps the agent explore, but the terminal outcome still domina
 
 - [Rubrics tutorial](../tutorials/rubrics.md) — full API walkthrough with composable examples.
 - [RFC 004](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md) — design rationale.
-- [Training with OpenEnv](rl-integration.md) — consume the reward signal in a training loop.
+- [Training with OpenEnv](training.md) — consume the reward signal in a training loop.
 - [Concepts](concepts.md) — where the rubric plugs in.
diff --git a/docs/source/guides/task-api.md b/docs/source/guides/task-api.md
index f379d468c2..f79c17d188 100644
--- a/docs/source/guides/task-api.md
+++ b/docs/source/guides/task-api.md
@@ -272,6 +272,6 @@ Before shipping a dataset-backed environment:
 ## Related Reading
 
 - [Concepts](concepts.md)
-- [Training with OpenEnv](rl-integration.md)
+- [Training with OpenEnv](training.md)
 - [Core API](../reference/core.md)
 - [CLI reference](../reference/cli.md)
diff --git a/docs/source/guides/rl-integration.md b/docs/source/guides/training.md
similarity index 100%
rename from docs/source/guides/rl-integration.md
rename to docs/source/guides/training.md
diff --git a/docs/source/tutorials/browsergym-harness.md b/docs/source/tutorials/browsergym-harness.md
index 7e927ddcf6..ca30b847c6 100644
--- a/docs/source/tutorials/browsergym-harness.md
+++ b/docs/source/tutorials/browsergym-harness.md
@@ -1,6 +1,6 @@
 # White-Box: Train a Web Agent on BrowserGym
 
-This tutorial trains a model to complete web tasks in [BrowserGym](../environments/browsergym) with TRL's `GRPOTrainer`. It is the **white-box** harness path: the trainer, here TRL, owns the agent loop. It samples every turn, parses the tool calls, runs them against the environment and feeds the results back, so it has the token ids and logprobs of everything the model generated without capturing anything. [Harnesses in OpenEnv](harnesses) compares it to the other paths, and [Training with OpenEnv](../guides/rl-integration) lists the other frameworks that train on OpenEnv environments.
+This tutorial trains a model to complete web tasks in [BrowserGym](../environments/browsergym) with TRL's `GRPOTrainer`. It is the **white-box** harness path: the trainer, here TRL, owns the agent loop. It samples every turn, parses the tool calls, runs them against the environment and feeds the results back, so it has the token ids and logprobs of everything the model generated without capturing anything. [Harnesses in OpenEnv](harnesses) compares it to the other paths, and [Training with OpenEnv](../guides/training) lists the other frameworks that train on OpenEnv environments.
 
 The full script is TRL's [`examples/grpo_browsergym/grpo_browsergym.py`](https://github.com/huggingface/trl/tree/main/examples/grpo_browsergym). This page walks through its parts.
 
diff --git a/docs/source/tutorials/harbor-harness.md b/docs/source/tutorials/harbor-harness.md
index e53b0fb057..9313874362 100644
--- a/docs/source/tutorials/harbor-harness.md
+++ b/docs/source/tutorials/harbor-harness.md
@@ -32,4 +32,4 @@ Each rollout comes back as a `TrainingTrace`: the token ids, logprobs and loss m
 
 - [The Harbor environment](../environments/harbor): supported agents and sandboxes, the web UI, deploying to Spaces and troubleshooting.
 - [TRL's Harbor integration](https://huggingface.co/docs/trl/harbor) for the white-box path on Harbor tasks, where TRL's own loop works through a Harbor task suite.
-- [Training with OpenEnv](../guides/rl-integration) for the other ways to train and the frameworks that support them.
+- [Training with OpenEnv](../guides/training) for the other ways to train and the frameworks that support them.
diff --git a/docs/source/tutorials/harnesses.md b/docs/source/tutorials/harnesses.md
index bf1476807f..68ad286c31 100644
--- a/docs/source/tutorials/harnesses.md
+++ b/docs/source/tutorials/harnesses.md
@@ -14,7 +14,7 @@ A harness is the loop around a model: it sends the conversation to the model, ru
 
 ## Which One to Use
 
-- **You want to train a model on your environment and the loop can be simple.** Use the white-box path. The trainer runs the tool loop, so token ids and logprobs come for free, and the environment only supplies tools and a reward. The BrowserGym tutorial does it with TRL's `GRPOTrainer`, and [Training with OpenEnv](../guides/rl-integration) lists the other frameworks that train on OpenEnv environments.
+- **You want to train a model on your environment and the loop can be simple.** Use the white-box path. The trainer runs the tool loop, so token ids and logprobs come for free, and the environment only supplies tools and a reward. The BrowserGym tutorial does it with TRL's `GRPOTrainer`, and [Training with OpenEnv](../guides/training) lists the other frameworks that train on OpenEnv environments.
 - **The job is a task that an agent does on its own, and a verifier checks the result.** Use Harbor, to train the model behind the agent or to evaluate it. The agent keeps its own planner, tools and context management in a sandbox, and a proxy records every model call it makes. The resulting `TrainingTrace` is plain data that doesn't depend on a trainer, and TRL's `AsyncGRPOTrainer` has a worked example.
 - **The job is a conversation, and the agent has to use your environment's tools.** Use `HarnessEnvironment` from [RFC 005](https://github.com/huggingface/OpenEnv/blob/main/rfcs/005-agentic-harnesses.md), to evaluate the agent against a simulated user or to serve it to real ones. The environment injects its tools, answers between turns and scores the conversation with a rubric.
 
diff --git a/docs/source/tutorials/index.md b/docs/source/tutorials/index.md
index d0c34e540e..f5b9ee862a 100644
--- a/docs/source/tutorials/index.md
+++ b/docs/source/tutorials/index.md
@@ -3,7 +3,7 @@
 Choose a learning path from the sidebar:
 
 - **Basics:** start with [Hello World](openenv-tutorial) and build [Your First Environment](../guides/first-environment).
-- **Training:** [the ways to train and the supported frameworks](../guides/rl-integration), then train a [reasoning model](end-to-end-walkthrough), use [TRL](wordle-grpo) or [Unsloth](rl-training-2048), or collect rollouts for [SFT](sft-warmup).
+- **Training:** [the ways to train and the supported frameworks](../guides/training), then train a [reasoning model](end-to-end-walkthrough), use [TRL](wordle-grpo) or [Unsloth](rl-training-2048), or collect rollouts for [SFT](sft-warmup).
 - **Harnesses:** [pick a path](harnesses): train with the trainer's own loop on [BrowserGym](browsergym-harness) (white-box), train real agents through [Harbor](harbor-harness) (black-box), or [evaluate Claude Code inside an environment](claude-code-harness). The [OpenCode](opencode-agent-grpo) and [Pi](pi-agent-grpo) tutorials are deprecated.
 - **Evals:** follow [Evaluating with Environments](evaluation-inspect).
 
diff --git a/docs/source/tutorials/rubrics.md b/docs/source/tutorials/rubrics.md
index 3fcb61df72..7f1bb3e694 100644
--- a/docs/source/tutorials/rubrics.md
+++ b/docs/source/tutorials/rubrics.md
@@ -379,4 +379,4 @@ The same rubric object used to compute training rewards doubles as the eval metr
 - **Real-world trajectory example** — walk through `envs/chess_env/server/rubrics.py` and `chess_environment.py` to see `ExponentialDiscountingTrajectoryRubric` wired into a game environment.
 - **Design details** — [RFC 004](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md) covers the rationale for the composable API and the "rewards inside the environment" invariant.
 - **Reward design basics** — the [Reward Design](../guides/rewards.md) guide covers sparse-vs-dense signals and common pitfalls that still apply on top of any rubric composition.
-- **Training loop integration** — see [Training with OpenEnv](../guides/rl-integration.md) and the [TRL OpenEnv integration guide](https://huggingface.co/docs/trl/main/en/openenv) for the recommended `environment_factory` pattern.
+- **Training loop integration** — see [Training with OpenEnv](../guides/training.md) and the [TRL OpenEnv integration guide](https://huggingface.co/docs/trl/main/en/openenv) for the recommended `environment_factory` pattern.
diff --git a/examples/echo_world_model/backends/trl.md b/examples/echo_world_model/backends/trl.md
index c90a512783..5f262e51c5 100644
--- a/examples/echo_world_model/backends/trl.md
+++ b/examples/echo_world_model/backends/trl.md
@@ -1,7 +1,7 @@
 # ECHO on TRL
 
 [TRL](https://huggingface.co/docs/trl) is the framework OpenEnv recommends for RL
-training (see `docs/source/guides/rl-integration.md` and the Wordle-GRPO
+training (see `docs/source/guides/training.md` and the Wordle-GRPO
 tutorial), so it is the natural place to run ECHO from an OpenEnv rollout.
 
 A runnable CPU proof-of-concept ships here:
diff --git a/examples/rubrics.ipynb b/examples/rubrics.ipynb
index dfbe74540a..8238200ba5 100644
--- a/examples/rubrics.ipynb
+++ b/examples/rubrics.ipynb
@@ -441,7 +441,7 @@
    "cell_type": "markdown",
    "id": "765896d7-09ea-4a31-a560-a362cd17faeb",
    "metadata": {},
-   "source": "## Next Steps\n\n- **Real-world trajectory example** \u2014 walk through `envs/chess_env/server/rubrics.py` and `chess_environment.py` to see `ExponentialDiscountingTrajectoryRubric` wired into a game environment.\n- **Design details** \u2014 [RFC 004](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md) covers the rationale for the composable API and the \"rewards inside the environment\" invariant.\n- **Reward design basics** \u2014 the [Reward Design](https://huggingface.co/docs/openenv/guides/rewards) guide covers sparse-vs-dense signals and common pitfalls that still apply on top of any rubric composition.\n- **Training loop integration** \u2014 see the [RL Framework Integration](https://huggingface.co/docs/openenv/guides/rl-integration) guide and the [TRL OpenEnv integration guide](https://huggingface.co/docs/trl/main/en/openenv) for the recommended `environment_factory` pattern."  }
+   "source": "## Next Steps\n\n- **Real-world trajectory example** \u2014 walk through `envs/chess_env/server/rubrics.py` and `chess_environment.py` to see `ExponentialDiscountingTrajectoryRubric` wired into a game environment.\n- **Design details** \u2014 [RFC 004](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md) covers the rationale for the composable API and the \"rewards inside the environment\" invariant.\n- **Reward design basics** \u2014 the [Reward Design](https://huggingface.co/docs/openenv/guides/rewards) guide covers sparse-vs-dense signals and common pitfalls that still apply on top of any rubric composition.\n- **Training loop integration** \u2014 see the [Training with OpenEnv](https://huggingface.co/docs/openenv/guides/training) guide and the [TRL OpenEnv integration guide](https://huggingface.co/docs/trl/main/en/openenv) for the recommended `environment_factory` pattern."  }
  ],
  "metadata": {
   "kernelspec": {