Skip to content
Merged
29 changes: 13 additions & 16 deletions docs/source/mcp-environment-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,48 +80,45 @@ The same rule applies to `reset()` and `reset_async()`.

Environment-specific MCP clients such as `EchoEnv` and `FinQAEnv` inherit from `MCPToolClient`.

Those clients expose convenience methods:
Those clients target a running environment server and expose convenience methods:

- `list_tools()`
- `call_tool()`
- `call_tool()` — **async**, must be awaited

These are helpers, not a separate environment lifecycle.

### Default behavior
### Client behavior

By default, the convenience methods still go through the OpenEnv session path.
`MCPToolClient` and its base `MCPClientBase` only support `mode="production"`; construction raises `ValueError` for other modes. For direct in-process training or eval code, instantiate the environment class and call `env.step(CallToolAction(...))` instead of using `MCPToolClient`.

- `list_tools()` wraps `step(ListToolsAction())`
- `call_tool()` wraps `step(CallToolAction(...))`
- `list_tools()` wraps `step(ListToolsAction())` and returns the `list[Tool]` directly.
- `call_tool(name, **kwargs)` returns the **unwrapped tool return value** directly — not the `CallToolObservation`, and not the runtime result object you would get from `obs.result`.
- `call_tool()` raises `RuntimeError` on any tool error. Use `step(CallToolAction(...))` when you need to inspect `ToolError.error_type` or continue after a failed tool call.

This preserves:

- episode context
- rewards
- step counting
- trajectory semantics
Remote `step(CallToolAction(...))` calls return a `StepResult`; the full observation is on `result.observation`, while `result.reward` carries the serialized reward field.

### Direct MCP behavior

When production MCP access is explicitly enabled on the client, the same convenience methods use the HTTP `/mcp` JSON-RPC endpoint directly.

That path is for tool-serving behavior, not the training loop.
That path is for tool-serving behavior, not the training loop. It bypasses reward computation, step counts, trajectory tracking, and `done` handling.

## Which Pattern Should You Use?

Use `step(CallToolAction(...))` when you need the full OpenEnv result object:
Use `step(CallToolAction(...))` when you need the full `CallToolObservation`:

- `reward`
- `done`
- observation metadata
- `obs.result`, a runtime result object typed as `Any`; FastMCP commonly returns `fastmcp.client.client.CallToolResult` with `.data`, `.content`, and `.structured_content`, but serialized clients or custom envs may surface a dict or plain value
- trajectory-compatible behavior

Use `call_tool()` when you only want the tool result and do not need to manually inspect the full `StepResult`.
Use `await env.call_tool(name, **kwargs)` when you only want the tool's raw return value and do not need to inspect the full observation. It is async and unwraps the result for you.

In other words:

- `step(...)` is the canonical simulation pattern
- `call_tool()` is a convenience wrapper
- `call_tool()` is an async convenience wrapper that returns the unwrapped tool output

## Concrete Examples

Expand Down
2 changes: 2 additions & 0 deletions docs/source/tutorials/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ If you're new to OpenEnv, we recommend starting with the [Getting Started](/auto
|----------|---------------|-----|----------|
| [OpenEnv Tutorial](openenv-tutorial.md) | Full introduction to OpenEnv: install, connect to a hosted environment, step through an episode, define a reward function, and run a basic training loop. Start here if you're new. | No | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/OpenEnv_Tutorial.ipynb) |
| [End-to-end walkthrough](end-to-end-walkthrough.md) | The full pipeline in one page: connect to `reasoning_gym`, wire it into TRL via `environment_factory`, fine-tune with GRPO on `chain_sum`, read the reward delta from training logs, and push the checkpoint to the Hub. | Yes | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/end_to_end_walkthrough.ipynb) |
| [Building and using MCP environments](mcp-environment.md) | Consume and build MCP-backed environments: list and call tools through `step()`, register Python functions as tools with FastMCP, and understand the dual API boundary. | No | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/mcp_environment.ipynb) |
| [Rubrics](rubrics.md) | Compose reward functions from reusable pieces using `openenv.core.rubrics`: `Gate`, `WeightedSum`, `LLMJudge`, and `TrajectoryRubric`. | No | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/rubrics.ipynb) |
| [Wordle GRPO](wordle-grpo.md) | Train an agent to play Wordle using GRPO via TRL's `environment_factory`. Shows the multi-turn tool-calling loop: the model guesses a word each turn and receives letter-position feedback until it wins or the episode ends. | Yes | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/openenv_wordle_grpo.ipynb) |
| [RL Training with 2048](rl-training-2048.md) | Train a language model to play the 2048 tile-sliding game with GRPO. Covers game-state representation and reward shaping for a puzzle environment. | Yes | — |
Expand All @@ -22,6 +23,7 @@ If you're new to OpenEnv, we recommend starting with the [Getting Started](/auto
:hidden:
openenv-tutorial
end-to-end-walkthrough
mcp-environment
rubrics
wordle-grpo
rl-training-2048
Expand Down
Loading