From d951a746d7dfb7117bc7b65bc31429c07cb03bb8 Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Tue, 21 Apr 2026 15:52:54 +0200 Subject: [PATCH 1/5] docs(mcp): add tutorial + tighten lifecycle guide Adds a hands-on MCP tutorial (`docs/source/tutorials/mcp-environment.md`) covering the two sides of the OpenEnv MCP contract: consuming an MCP environment from a simulation loop (ListToolsAction / CallToolAction via step(), error handling, step vs call_tool trade-off) and building one by subclassing MCPEnvironment + FastMCP tool registration. The walkthrough uses `envs/echo_env/server/echo_environment.py` as the worked example and points readers at `examples/echo_mcp_demo.py` for an end-to-end run. The tutorial is built to complement, not duplicate, the existing `docs/source/mcp-environment-lifecycle.md` FAQ-style guide: the tutorial teaches how to do it, the lifecycle guide explains when/why. Every code sample was executed against real source before shipping. That process surfaced three inaccuracies which are fixed here: - `env.step(CallToolAction(...)).result` is a `CallToolResult` wrapper, not the raw return value. Tutorial uses `obs.result.data` and explains the wrapper (`.data`, `.content`, `.structured_content`). - `obs.error.error_type` is a `ToolErrorType` enum, not a string. - `MCPEnvironment._step_impl` is abstract; a subclass cannot be instantiated without it. The tutorial's echo example includes it and the prose now says "`_step_impl` is required, `step` is not". Also tightens two gaps in the lifecycle guide surfaced while writing the tutorial and verified against `MCPToolClient.call_tool`: - `call_tool()` is async and must be awaited (not previously stated). - `call_tool()` returns the unwrapped tool value; `step(CallToolAction( ...)).result` returns a `CallToolResult` wrapper. The "Which Pattern Should You Use?" section now calls out both shapes explicitly. Adds the tutorial to `docs/source/tutorials/index.md` toctree. Verified: all four tutorial snippets execute end-to-end against EchoEnvironment; sphinx-build passes with zero warnings on the new or edited pages; internal links resolve (`#which-pattern-should-you-use` anchor in lifecycle guide confirmed in rendered HTML). Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/mcp-environment-lifecycle.md | 13 +- docs/source/tutorials/index.md | 2 + docs/source/tutorials/mcp-environment.md | 170 +++++++++++++++++++++++ 3 files changed, 179 insertions(+), 6 deletions(-) create mode 100644 docs/source/tutorials/mcp-environment.md diff --git a/docs/source/mcp-environment-lifecycle.md b/docs/source/mcp-environment-lifecycle.md index 1d8a0f8bd1..11074cfce5 100644 --- a/docs/source/mcp-environment-lifecycle.md +++ b/docs/source/mcp-environment-lifecycle.md @@ -83,7 +83,7 @@ Environment-specific MCP clients such as `EchoEnv` and `FinQAEnv` inherit from ` Those clients expose convenience methods: - `list_tools()` -- `call_tool()` +- `call_tool()` — **async**, must be awaited These are helpers, not a separate environment lifecycle. @@ -91,8 +91,8 @@ These are helpers, not a separate environment lifecycle. By default, the convenience methods still go through the OpenEnv session path. -- `list_tools()` wraps `step(ListToolsAction())` -- `call_tool()` wraps `step(CallToolAction(...))` +- `list_tools()` wraps `step(ListToolsAction())` and returns the `list[Tool]` directly. +- `call_tool(name, **kwargs)` wraps `step(CallToolAction(tool_name=name, arguments=kwargs))` and returns the **unwrapped tool return value** directly — not the `CallToolObservation`, and not the `CallToolResult` wrapper you would get from `obs.result`. This preserves: @@ -109,19 +109,20 @@ That path is for tool-serving behavior, not the training loop. ## Which Pattern Should You Use? -Use `step(CallToolAction(...))` when you need the full OpenEnv result object: +Use `step(CallToolAction(...))` when you need the full `CallToolObservation`: - `reward` - `done` - observation metadata +- `obs.result`, a `CallToolResult` wrapper whose `.data` attribute carries the raw return value and whose `.content` / `.structured_content` carry the MCP-protocol payloads - trajectory-compatible behavior -Use `call_tool()` when you only want the tool result and do not need to manually inspect the full `StepResult`. +Use `await env.call_tool(name, **kwargs)` when you only want the tool's raw return value and do not need to inspect the full observation. It is async and unwraps the result for you. In other words: - `step(...)` is the canonical simulation pattern -- `call_tool()` is a convenience wrapper +- `call_tool()` is an async convenience wrapper that returns the unwrapped tool output ## Concrete Examples diff --git a/docs/source/tutorials/index.md b/docs/source/tutorials/index.md index 0bc7be4d7a..1a44f5544c 100644 --- a/docs/source/tutorials/index.md +++ b/docs/source/tutorials/index.md @@ -11,6 +11,7 @@ If you're new to OpenEnv, we recommend starting with the [Getting Started](/auto - **[OpenEnv Tutorial](openenv-tutorial.md)** - A comprehensive introduction to OpenEnv, covering installation, basic usage, and core concepts. - **[Wordle GRPO Training](wordle-grpo.md)** - Learn how to train an agent to play Wordle using Group Relative Policy Optimization (GRPO). - **[RL Training with 2048](rl-training-2048.md)** - Train a language model to play 2048 using GRPO reinforcement learning. *(GPU Required)* +- **[Building and Using MCP Environments](mcp-environment.md)** - Consume and build MCP-backed environments: list and call tools through `step()`, register Python functions as tools with FastMCP, and understand the dual API boundary. ```{toctree} :maxdepth: 2 @@ -18,4 +19,5 @@ If you're new to OpenEnv, we recommend starting with the [Getting Started](/auto openenv-tutorial wordle-grpo rl-training-2048 +mcp-environment ``` diff --git a/docs/source/tutorials/mcp-environment.md b/docs/source/tutorials/mcp-environment.md new file mode 100644 index 0000000000..57490f86cc --- /dev/null +++ b/docs/source/tutorials/mcp-environment.md @@ -0,0 +1,170 @@ +# Building and Using MCP Environments + +OpenEnv speaks **MCP** ([Model Context Protocol](https://modelcontextprotocol.io)) as its agent-facing interface. If an environment exposes tools, those tools live behind MCP — not behind ad-hoc REST endpoints or custom action schemas. This tutorial walks through the two sides of that contract: **consuming** an MCP environment from a training / simulation loop, and **building** one from scratch. + +By the end you should be able to: + +- List and call tools on any MCP environment through `step()`. +- Write a minimal `MCPEnvironment` subclass that exposes Python functions as tools. +- Understand which API path (`/ws` vs `/mcp`) belongs to which audience. + +## Why MCP? + +OpenEnv's dual API boundary splits responsibilities between two audiences: + +- **Training / orchestration infrastructure** uses the Gym-style control plane — `reset()`, `step()`, `state()` — over WebSocket (`/ws`). This is what the trainer needs to roll out episodes, compute rewards, and enforce termination. +- **Agents** use MCP tools over the `/mcp` JSON-RPC endpoint. Tools are what the model calls to act on the world; they have discoverable schemas and deterministic names. + +Standardising on MCP for agent actions means a single environment can be trained via GRPO, served for inference through an MCP-compatible client, and inspected with off-the-shelf MCP tooling — without separate interfaces to maintain. + +:::{note} +In simulation mode, MCP tool calls flow **through** `step()`. The trainer stays in control of timing, rewards, and termination; the MCP action types are just a standardised action schema. The [MCP environment lifecycle guide](../mcp-environment-lifecycle.md) covers the split in depth. +::: + +## Consuming an MCP Environment + +The two MCP action types are `ListToolsAction` (discover what's available) and `CallToolAction` (invoke one). They behave like any other Gym action — pass them to `step()` and inspect the returned observation. + +### Discovering tools + +```python +from echo_env.server.echo_environment import EchoEnvironment +from openenv.core.env_server.mcp_types import ListToolsAction, ListToolsObservation + +env = EchoEnvironment() +env.reset() + +obs = env.step(ListToolsAction()) +assert isinstance(obs, ListToolsObservation) + +for tool in obs.tools: + print(f"{tool.name}: {tool.description}") +``` + +Each `Tool` carries a `name`, a `description`, and an `input_schema` (JSON Schema) describing the accepted arguments. The schema is what lets a language-model agent know which parameters to fill in when it emits a tool call. + +### Calling a tool + +```python +from openenv.core.env_server.mcp_types import CallToolAction, CallToolObservation + +obs = env.step( + CallToolAction( + tool_name="echo_message", + arguments={"message": "Hello from MCP!"}, + ) +) + +assert isinstance(obs, CallToolObservation) +print(obs.tool_name) # "echo_message" +print(obs.result.data) # "Hello from MCP!" — the raw tool return value +print(obs.error) # None +``` + +`obs.result` is a `CallToolResult` wrapper exposing the return value in a few shapes: `.data` is the raw Python value the tool returned, `.structured_content` is its JSON-encoded form, and `.content` is the MCP protocol's list of typed content blocks (useful when a tool returns rich multi-part output). `obs.error` is set only when the **framework** could not deliver the call (transport failure, unknown tool name, malformed arguments). Tool-specific failures — a business-logic error that the tool itself raised — come back inside `result`, so callers can handle them like any domain-specific response. + +### Error handling + +```python +obs = env.step( + CallToolAction(tool_name="does_not_exist", arguments={}), +) + +assert isinstance(obs, CallToolObservation) +print(obs.error.error_type) # ToolErrorType.TOOL_NOT_FOUND +print(obs.error.message) # "Unknown tool: 'does_not_exist'" +``` + +The `ToolError.error_type` enum (`TOOL_NOT_FOUND`, `INVALID_ARGS`, `EXECUTION_ERROR`, `TRANSPORT_ERROR`, `TIMEOUT`) lets training code distinguish between bugs in the agent, bugs in the environment, and transient infrastructure issues — which often warrant different reward signals. + +### `step(CallToolAction(...))` vs `call_tool()` + +Environment clients that inherit from `MCPToolClient` (such as `EchoEnv` and `FinQAEnv`) expose a shorter **async** `await env.call_tool("name", arg=value)` helper. Functionally equivalent in simulation mode — it still goes through the step loop and still updates rewards, step counts, and trajectory state — but returns the tool's raw return value directly instead of a `CallToolObservation`. Use `step(CallToolAction(...))` when you need the whole observation (reward, done, metadata); reach for `call_tool()` in async scripts where the result is all you care about. The [lifecycle guide](../mcp-environment-lifecycle.md#which-pattern-should-you-use) covers the exact trade-offs. + +## Building an MCP Environment + +The provider side is just as small. Subclass `MCPEnvironment`, create a `FastMCP` server, register tools with the `@mcp.tool` decorator, and pass the server to `super().__init__`. Here is the echo environment, trimmed from [`envs/echo_env/server/echo_environment.py`](https://github.com/meta-pytorch/OpenEnv/blob/main/envs/echo_env/server/echo_environment.py) down to the parts this tutorial covers: + +```python +from uuid import uuid4 + +from fastmcp import FastMCP + +from openenv.core.env_server.mcp_environment import MCPEnvironment +from openenv.core.env_server.types import Action, Observation, State + + +class EchoEnvironment(MCPEnvironment): + SUPPORTS_CONCURRENT_SESSIONS = True + + def __init__(self): + mcp = FastMCP("echo_env") + + @mcp.tool + def echo_message(message: str) -> str: + """Echo back the provided message. + + Args: + message: The message to echo back + + Returns: + The same message that was provided + """ + return message + + @mcp.tool + def echo_with_length(message: str) -> dict: + """Echo back the message with its length. + + Args: + message: The message to echo back + + Returns: + Dictionary with the message and its length + """ + return {"message": message, "length": len(message)} + + super().__init__(mcp) + self._state = State(episode_id=str(uuid4()), step_count=0) + + def reset(self, seed=None, episode_id=None, **kwargs) -> Observation: + self._state = State(episode_id=episode_id or str(uuid4()), step_count=0) + return Observation(done=False, reward=0.0, metadata={"status": "ready"}) + + def _step_impl(self, action: Action, timeout_s=None, **kwargs) -> Observation: + # Called for non-MCP actions. Echo exposes MCP tools only, + # so anything that isn't ListToolsAction / CallToolAction is an error. + return Observation( + done=False, + reward=0.0, + metadata={"error": f"Unsupported action type: {type(action).__name__}"}, + ) + + @property + def state(self) -> State: + return self._state +``` + +A few things worth calling out: + +- **Docstring → schema.** `FastMCP` inspects each tool's signature and Google-style docstring to build the `input_schema` automatically. The `Args:` block becomes parameter descriptions, and type hints become JSON types. No hand-written schemas. +- **Reserved names.** `reset`, `step`, `state`, and `close` are reserved and cannot be tool names — they belong to the infrastructure boundary. Trying to register a tool with one of those names raises at construction time. +- **`_step_impl` is required, `step` is not.** `MCPEnvironment.step` already routes `ListToolsAction` and `CallToolAction` through the FastMCP server for you. Your subclass only has to implement `_step_impl`, which the base class calls for any **non-MCP** action. In pure-MCP environments like Echo it just returns an error observation; in environments that mix tool calls with other action types (e.g. a terminal "submit" action) it's where that extra dispatch lives. +- **Rewards and `done` still work.** Because MCP actions flow through `step()`, you can compute rewards, flip `done`, and emit metadata just like in any other OpenEnv environment. + +## Running the Demo End-to-End + +The repo ships a self-contained walkthrough at [`examples/echo_mcp_demo.py`](https://github.com/meta-pytorch/OpenEnv/blob/main/examples/echo_mcp_demo.py). Run it directly from the repo root: + +```bash +PYTHONPATH=src:envs uv run python examples/echo_mcp_demo.py +``` + +You will see the discovery call, two tool invocations, and an error case printed in sequence — the same four steps covered above, end-to-end against the real `EchoEnvironment`. + +## Next Steps + +- **MCP lifecycle details** — the [MCP Environment Lifecycle guide](../mcp-environment-lifecycle.md) covers `step()` vs `step_async()`, the `call_tool()` convenience path, and common debugging questions. +- **A richer MCP environment** — [`envs/finqa_env/`](https://github.com/meta-pytorch/OpenEnv/tree/main/envs/finqa_env) shows tool calls participating in episode progression, rewards, and terminal submission — not just a stateless echo. +- **Design rationale** — [RFC 003](https://github.com/meta-pytorch/OpenEnv/blob/main/rfcs/003-mcp-support.md) explains why OpenEnv picked MCP as the agent boundary and how tool-calling and CodeAct styles share the same plumbing. +- **Serving tools to an external agent** — the `/mcp` JSON-RPC endpoint is available alongside `/ws` on any MCP environment server. Point an MCP-compatible client at it for production inference without going through the step loop. From 5854b73d77bda523cf54d05d3235e7861447da02 Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Tue, 21 Apr 2026 16:59:56 +0200 Subject: [PATCH 2/5] docs(mcp): reframe training section + add eval + adoption note MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second pass on the tutorial, driven by the "is this actually useful for an ML engineer starting with OpenEnv?" lens: - Opening "Why MCP?" now leads with the question every reader will ask — "if my tools are just local Python functions, do I even need MCP?" — and answers it honestly (process boundary, env reuse, external agents, schema discovery). The dual-API-boundary explanation becomes a sub-section of "Why", not the top of it. - "Using MCP Tools in a Training Loop" is restructured so the framework-agnostic rollout loop is the prominent path. TRL's environment_factory is a sub-section clearly labelled as a TRL API (not an MCP API) — with an explicit note that environment_factory and MCP are orthogonal and compose, but neither implies the other (Wordle uses environment_factory without MCP; MCP envs work fine without environment_factory). - New "Using MCP Tools for Evaluation" section, parallel to the "Using Rubrics for Evaluation" section in the Rubrics tutorial: plain-Python offline loop over a static dataset, linking to the Reward Design guide for scoring, honest about the in-flight state of src/openenv/core/evals/. - New note on MCP adoption in OpenEnv: RFC 003 is aspirational and still In Review; today only echo_env + finqa_env inherit from the canonical openenv.core.env_server.mcp_environment.MCPEnvironment (calendar_env uses a local wrapper with the same shape). The other ~27 envs (textarena / openspiel / chess / browsergym / ...) still use custom action types without MCP plumbing. Readers should check an env's inheritance before assuming the patterns here apply. Also fixes four issues caught in a pre-push review pass: - Broken internal link to rubrics.md (file lives in an unmerged PR, not on main) - replaced with the Reward Design guide, which does exist on main. - Framework-agnostic rollout snippet referenced env.last_observation, which is not an attribute on the Environment base class (see src/openenv/core/env_server/interfaces.py). Rewrote the snippet to use the obs variable returned from env.reset() / previous env.step(), which is the actual idiom. - Tightened the adoption note to not overstate calendar_env's MCP inheritance (it uses a local wrapper, not the canonical base). - Softened "Swap TextArenaAction for CallToolAction" in Next Steps to call out that the swap is structural (single-field vs tool_name + arguments), not cosmetic. Every code sample was re-executed against the real EchoEnvironment before commit. Build is clean: sphinx-build produces zero warnings attributable to this file. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/tutorials/mcp-environment.md | 122 ++++++++++++++++++++--- 1 file changed, 106 insertions(+), 16 deletions(-) diff --git a/docs/source/tutorials/mcp-environment.md b/docs/source/tutorials/mcp-environment.md index 57490f86cc..3044fa6197 100644 --- a/docs/source/tutorials/mcp-environment.md +++ b/docs/source/tutorials/mcp-environment.md @@ -1,27 +1,88 @@ -# Building and Using MCP Environments +# MCP Tools in OpenEnv Environments -OpenEnv speaks **MCP** ([Model Context Protocol](https://modelcontextprotocol.io)) as its agent-facing interface. If an environment exposes tools, those tools live behind MCP — not behind ad-hoc REST endpoints or custom action schemas. This tutorial walks through the two sides of that contract: **consuming** an MCP environment from a training / simulation loop, and **building** one from scratch. +Most agentic work ends up needing the same thing: a way for the model to **call tools** and receive structured feedback, whether that is during RL training or offline evaluation. OpenEnv standardises that surface with **[MCP](https://modelcontextprotocol.io)** (Model Context Protocol), so the same tool interface works during training, eval, inference, and external serving. This tutorial covers the four paths you will walk in practice — wiring an MCP-backed environment into a training loop, using the same env for offline eval, inspecting the API underneath both of those, and building your own MCP environment when no existing one fits. -By the end you should be able to: +## Why MCP? -- List and call tools on any MCP environment through `step()`. -- Write a minimal `MCPEnvironment` subclass that exposes Python functions as tools. -- Understand which API path (`/ws` vs `/mcp`) belongs to which audience. +If your tools are just local Python functions, you do not need MCP — pass them to your trainer directly (TRL, torchforge, whatever) and you are done. MCP earns its complexity when the tool surface has to exist as a **process boundary**, not a function call: -## Why MCP? +- **The env runs elsewhere** — in a Docker container, a Hugging Face Space, a remote server. MCP is the transport that crosses that boundary. +- **You want to reuse someone else's env** — the OpenEnv catalog, third-party envs, and community hubs all expose their tools over MCP, so the same env works in your training run without rewriting its interface. +- **You want the env to be callable by other agents** — Claude Desktop, Cursor, inference servers, and any MCP-compatible client can plug into an MCP server. A private Python function doesn't get that for free. +- **You need tool discovery and schemas** — `list_tools()` + auto-generated JSON schemas are part of the protocol; models see them the same way they see any MCP server's tools. -OpenEnv's dual API boundary splits responsibilities between two audiences: +In short: MCP is the answer when your env is more than a helper function in your training script — when the same tools have to be usable from training, inference, and external clients without maintaining three interfaces. -- **Training / orchestration infrastructure** uses the Gym-style control plane — `reset()`, `step()`, `state()` — over WebSocket (`/ws`). This is what the trainer needs to roll out episodes, compute rewards, and enforce termination. -- **Agents** use MCP tools over the `/mcp` JSON-RPC endpoint. Tools are what the model calls to act on the world; they have discoverable schemas and deterministic names. +### The dual API boundary -Standardising on MCP for agent actions means a single environment can be trained via GRPO, served for inference through an MCP-compatible client, and inspected with off-the-shelf MCP tooling — without separate interfaces to maintain. +Inside OpenEnv, MCP plays a specific role in a two-surface split: -:::{note} +- **Training / orchestration infrastructure** uses the Gym-style control plane — `reset()`, `step()`, `state()` — over WebSocket (`/ws`). This is what the trainer needs to roll out episodes, compute rewards, and enforce termination. +- **Agents** use MCP tools over the `/mcp` JSON-RPC endpoint. Tools are what the model calls to act on the world. + +```{note} In simulation mode, MCP tool calls flow **through** `step()`. The trainer stays in control of timing, rewards, and termination; the MCP action types are just a standardised action schema. The [MCP environment lifecycle guide](../mcp-environment-lifecycle.md) covers the split in depth. -::: +``` + +```{note} +**MCP adoption in OpenEnv is still in flight.** [RFC 003](https://github.com/meta-pytorch/OpenEnv/blob/main/rfcs/003-mcp-support.md) proposes MCP as the standard interface for *all* agent-facing actions, but it is still **In Review**. Today only a handful of envs are MCP-backed: `echo_env` and `finqa_env` inherit from the canonical `openenv.core.env_server.mcp_environment.MCPEnvironment`; `calendar_env` uses a local wrapper with the same shape. The majority (`textarena_env` / Wordle, `openspiel_env`, `chess_env`, `browsergym_env`, and most others) still use custom action types that you pass through `env.step(CustomAction(...))` without MCP plumbing. Before using the patterns in this tutorial against a specific env, check whether it inherits from an `MCPEnvironment` base; if not, the env's own action schema applies instead. +``` + +## Using MCP Tools in a Training Loop + +An MCP-backed env is consumed like any other OpenEnv env from the trainer's side. At the atomic level, each agent action is: + +```python +obs = env.step(CallToolAction(tool_name=..., arguments=...)) +# obs.result.data — tool reply the model sees +# obs.reward — env's reward for this turn (may be None) +# obs.done — episode terminated +``` + +That is the only MCP-specific piece. Everything around it — how the trainer generates actions, how tool schemas are surfaced to the model, how rewards are collected — belongs to your training framework, not to MCP. + +### Framework-agnostic rollout loop + +If you drive the rollout yourself (a custom loop, [torchforge](https://github.com/pytorch-labs/torchforge), an external agent server), you own the full generation path and call `env.step()` directly: + +```python +obs = env.reset() +total_reward = 0.0 +for turn in range(max_turns): + tool_call = model.decide(obs) # your agent picks a tool + args from the latest observation + obs = env.step( + CallToolAction(tool_name=tool_call.name, arguments=tool_call.arguments) + ) + total_reward += obs.reward or 0.0 + if obs.done: + break +``` + +Whatever policy / generation code you use, `env.step(CallToolAction(...))` is the only line that talks to the MCP env. -## Consuming an MCP Environment +### TRL `environment_factory` + +[TRL](https://huggingface.co/docs/trl)'s `GRPOTrainer` takes an `environment_factory` class whose public methods auto-register as discoverable tools — the trainer then handles the multi-turn generation loop for you. The [Wordle GRPO tutorial](wordle-grpo.md) shows the full recipe (wrapper class, reward function, `GRPOTrainer` construction) with a non-MCP env. For an MCP-backed env, only the tool method bodies change; they call through to `env.step(CallToolAction(...))`: + +```python +def echo(self, message: str) -> str: + """Echo back a message. + + Args: + message: The message to echo. + """ + obs = self.env.step( + CallToolAction(tool_name="echo_message", arguments={"message": message}) + ) + self.reward = obs.reward or 0.0 # Echo has no domain reward — swap in FinQA or similar + return obs.result.data # what the model sees as the tool's reply +``` + +`environment_factory` is a TRL API, not an MCP API. It works equally well with non-MCP envs (Wordle uses it with `TextArenaAction`), and MCP envs work equally well without it (the rollout-loop path above). They compose, but they are orthogonal. + +The rest of this tutorial is for the other paths: the API **underneath** `env.step(CallToolAction(...))` (useful when you need the full observation or want to debug), **using the same env for offline eval**, and **building your own MCP environment** from scratch. + +## Under the Hood: `CallToolAction` and `ListToolsAction` The two MCP action types are `ListToolsAction` (discover what's available) and `CallToolAction` (invoke one). They behave like any other Gym action — pass them to `step()` and inspect the returned observation. @@ -81,9 +142,37 @@ The `ToolError.error_type` enum (`TOOL_NOT_FOUND`, `INVALID_ARGS`, `EXECUTION_ER Environment clients that inherit from `MCPToolClient` (such as `EchoEnv` and `FinQAEnv`) expose a shorter **async** `await env.call_tool("name", arg=value)` helper. Functionally equivalent in simulation mode — it still goes through the step loop and still updates rewards, step counts, and trajectory state — but returns the tool's raw return value directly instead of a `CallToolObservation`. Use `step(CallToolAction(...))` when you need the whole observation (reward, done, metadata); reach for `call_tool()` in async scripts where the result is all you care about. The [lifecycle guide](../mcp-environment-lifecycle.md#which-pattern-should-you-use) covers the exact trade-offs. +## Using MCP Tools for Evaluation + +The same mechanics work outside a training loop. For an offline eval — benchmarking a model's tool use on a static dataset, regression-testing a deployed agent, or scoring a policy — drop the trainer and drive the step loop yourself: + +```python +from echo_env.server.echo_environment import EchoEnvironment +from openenv.core.env_server.mcp_types import CallToolAction + +env = EchoEnvironment() +env.reset() + +results = [] +for sample in eval_dataset: + tool_call = model.decide(sample) # your agent picks a tool + arguments + obs = env.step( + CallToolAction(tool_name=tool_call.name, arguments=tool_call.arguments), + ) + results.append({ + "prompt": sample.prompt, + "reply": obs.result.data, + "reward": obs.reward or 0.0, + "error": obs.error, + }) + env.reset() +``` + +Pair the loop with a scoring function of your choice — the [Reward Design](../guides/rewards.md) guide covers common patterns (test-pass rate, LLM-as-judge quality, compliance gates) — and aggregate across the dataset. The eval harness integration in `src/openenv/core/evals/` is still evolving; until that bridge lands, this plain-Python loop is the canonical pattern. + ## Building an MCP Environment -The provider side is just as small. Subclass `MCPEnvironment`, create a `FastMCP` server, register tools with the `@mcp.tool` decorator, and pass the server to `super().__init__`. Here is the echo environment, trimmed from [`envs/echo_env/server/echo_environment.py`](https://github.com/meta-pytorch/OpenEnv/blob/main/envs/echo_env/server/echo_environment.py) down to the parts this tutorial covers: +Reach for this path when no existing environment covers the tools your agent needs — e.g. a new coding sandbox, a game, a proprietary API wrapper. The provider side is small: subclass `MCPEnvironment`, create a `FastMCP` server, register tools with the `@mcp.tool` decorator, and pass the server to `super().__init__`. Here is the echo environment, trimmed from [`envs/echo_env/server/echo_environment.py`](https://github.com/meta-pytorch/OpenEnv/blob/main/envs/echo_env/server/echo_environment.py) down to the parts this tutorial covers: ```python from uuid import uuid4 @@ -160,10 +249,11 @@ The repo ships a self-contained walkthrough at [`examples/echo_mcp_demo.py`](htt PYTHONPATH=src:envs uv run python examples/echo_mcp_demo.py ``` -You will see the discovery call, two tool invocations, and an error case printed in sequence — the same four steps covered above, end-to-end against the real `EchoEnvironment`. +You will see the discovery call, two tool invocations, and an error case printed in sequence — the same four steps the "Under the hood" section covers, end-to-end against the real `EchoEnvironment`. ## Next Steps +- **End-to-end training recipe** — the [Wordle GRPO tutorial](wordle-grpo.md) walks through a full GRPO training run with `environment_factory`. The wrapper-class shape is the same for an MCP-backed env; inside each tool method, build a `CallToolAction(tool_name=..., arguments={...})` instead of Wordle's single-field `TextArenaAction(message=guess)`. - **MCP lifecycle details** — the [MCP Environment Lifecycle guide](../mcp-environment-lifecycle.md) covers `step()` vs `step_async()`, the `call_tool()` convenience path, and common debugging questions. - **A richer MCP environment** — [`envs/finqa_env/`](https://github.com/meta-pytorch/OpenEnv/tree/main/envs/finqa_env) shows tool calls participating in episode progression, rewards, and terminal submission — not just a stateless echo. - **Design rationale** — [RFC 003](https://github.com/meta-pytorch/OpenEnv/blob/main/rfcs/003-mcp-support.md) explains why OpenEnv picked MCP as the agent boundary and how tool-calling and CodeAct styles share the same plumbing. From 99a9fbf3145593f780eb85333aae9ed802e10c82 Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Fri, 24 Apr 2026 11:28:13 +0200 Subject: [PATCH 3/5] docs(mcp): correct error paths and call_tool divergence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes three review issues on #602: - Tutorial claimed tool exceptions come back in `obs.result`, but per mcp_environment.py:543-551 they land in `obs.error` as EXECUTION_ERROR with `result=None`. A reader following the old text would silently miss tool failures. Rewrite the paragraph to state that `obs.error` carries every failure mode and that callers must branch on `obs.error is None` before touching `obs.result.data`. - Apply the same guard in the eval-loop snippet. - Document that `call_tool()` raises `RuntimeError` on any `obs.error` (mcp_client.py:430-435) — not a drop-in swap for `step()` when you need to branch on `error_type`. - Soften the FastMCP-dependent `TOOL_NOT_FOUND` message to `e.g. ...` so it does not break if FastMCP changes its wording. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/tutorials/mcp-environment.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/tutorials/mcp-environment.md b/docs/source/tutorials/mcp-environment.md index 3044fa6197..0f166adb90 100644 --- a/docs/source/tutorials/mcp-environment.md +++ b/docs/source/tutorials/mcp-environment.md @@ -122,7 +122,7 @@ print(obs.result.data) # "Hello from MCP!" — the raw tool return value print(obs.error) # None ``` -`obs.result` is a `CallToolResult` wrapper exposing the return value in a few shapes: `.data` is the raw Python value the tool returned, `.structured_content` is its JSON-encoded form, and `.content` is the MCP protocol's list of typed content blocks (useful when a tool returns rich multi-part output). `obs.error` is set only when the **framework** could not deliver the call (transport failure, unknown tool name, malformed arguments). Tool-specific failures — a business-logic error that the tool itself raised — come back inside `result`, so callers can handle them like any domain-specific response. +`obs.result` is a `CallToolResult` wrapper exposing the return value in a few shapes: `.data` is the raw Python value the tool returned, `.structured_content` is its JSON-encoded form, and `.content` is the MCP protocol's list of typed content blocks (useful when a tool returns rich multi-part output). `obs.error` carries **every** failure mode — transport errors, unknown tool names, malformed arguments, **and** exceptions raised from inside the tool function itself (as `ToolErrorType.EXECUTION_ERROR`). On an error, `obs.result` is `None`. Always branch on `obs.error is None` before reading `obs.result.data`, or a tool that raised will look like a successful call that returned `None`. ### Error handling @@ -133,14 +133,14 @@ obs = env.step( assert isinstance(obs, CallToolObservation) print(obs.error.error_type) # ToolErrorType.TOOL_NOT_FOUND -print(obs.error.message) # "Unknown tool: 'does_not_exist'" +print(obs.error.message) # human-readable message from FastMCP, e.g. "Unknown tool: 'does_not_exist'" ``` The `ToolError.error_type` enum (`TOOL_NOT_FOUND`, `INVALID_ARGS`, `EXECUTION_ERROR`, `TRANSPORT_ERROR`, `TIMEOUT`) lets training code distinguish between bugs in the agent, bugs in the environment, and transient infrastructure issues — which often warrant different reward signals. ### `step(CallToolAction(...))` vs `call_tool()` -Environment clients that inherit from `MCPToolClient` (such as `EchoEnv` and `FinQAEnv`) expose a shorter **async** `await env.call_tool("name", arg=value)` helper. Functionally equivalent in simulation mode — it still goes through the step loop and still updates rewards, step counts, and trajectory state — but returns the tool's raw return value directly instead of a `CallToolObservation`. Use `step(CallToolAction(...))` when you need the whole observation (reward, done, metadata); reach for `call_tool()` in async scripts where the result is all you care about. The [lifecycle guide](../mcp-environment-lifecycle.md#which-pattern-should-you-use) covers the exact trade-offs. +Environment clients that inherit from `MCPToolClient` (such as `EchoEnv` and `FinQAEnv`) expose a shorter **async** `await env.call_tool("name", arg=value)` helper. It still goes through the step loop and still updates rewards, step counts, and trajectory state, but returns the tool's raw return value directly instead of a `CallToolObservation` — and it **raises `RuntimeError`** on any `obs.error` (transport failure, unknown tool, or a tool exception), so you cannot branch on `error_type` without a `try/except`. Use `step(CallToolAction(...))` when you need the whole observation (reward, done, metadata, or graceful error classification); reach for `call_tool()` in async scripts where the raw result is all you care about and a failure is allowed to propagate. The [lifecycle guide](../mcp-environment-lifecycle.md#which-pattern-should-you-use) covers the exact trade-offs. ## Using MCP Tools for Evaluation @@ -161,7 +161,7 @@ for sample in eval_dataset: ) results.append({ "prompt": sample.prompt, - "reply": obs.result.data, + "reply": obs.result.data if obs.error is None else None, "reward": obs.reward or 0.0, "error": obs.error, }) From 4dbf75750c463747fe86f7d8496c0ce527ca2121 Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Mon, 27 Apr 2026 15:17:47 +0200 Subject: [PATCH 4/5] docs(mcp): match echo error string + flag call_tool production-only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Change `Unsupported action type` to `Unknown action type` so the trimmed snippet matches `envs/echo_env/server/echo_environment.py:160`. - Add a note that `call_tool()` is production-only — `MCPToolClient` raises `ValueError` outside production mode — to steer simulation callers to `step(CallToolAction(...))`. --- docs/source/tutorials/mcp-environment.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/source/tutorials/mcp-environment.md b/docs/source/tutorials/mcp-environment.md index 0f166adb90..b2e5732325 100644 --- a/docs/source/tutorials/mcp-environment.md +++ b/docs/source/tutorials/mcp-environment.md @@ -142,6 +142,10 @@ The `ToolError.error_type` enum (`TOOL_NOT_FOUND`, `INVALID_ARGS`, `EXECUTION_ER Environment clients that inherit from `MCPToolClient` (such as `EchoEnv` and `FinQAEnv`) expose a shorter **async** `await env.call_tool("name", arg=value)` helper. It still goes through the step loop and still updates rewards, step counts, and trajectory state, but returns the tool's raw return value directly instead of a `CallToolObservation` — and it **raises `RuntimeError`** on any `obs.error` (transport failure, unknown tool, or a tool exception), so you cannot branch on `error_type` without a `try/except`. Use `step(CallToolAction(...))` when you need the whole observation (reward, done, metadata, or graceful error classification); reach for `call_tool()` in async scripts where the raw result is all you care about and a failure is allowed to propagate. The [lifecycle guide](../mcp-environment-lifecycle.md#which-pattern-should-you-use) covers the exact trade-offs. +```{note} +`call_tool()` is production-only: `MCPToolClient.__init__` raises `ValueError` if `mode != "production"`. In simulation contexts, route tool calls through `step(CallToolAction(...))` instead. +``` + ## Using MCP Tools for Evaluation The same mechanics work outside a training loop. For an offline eval — benchmarking a model's tool use on a static dataset, regression-testing a deployed agent, or scoring a policy — drop the trainer and drive the step loop yourself: @@ -226,7 +230,7 @@ class EchoEnvironment(MCPEnvironment): return Observation( done=False, reward=0.0, - metadata={"error": f"Unsupported action type: {type(action).__name__}"}, + metadata={"error": f"Unknown action type: {type(action).__name__}"}, ) @property From 5d5396e07bca36be82e85a35b06be6108eb88669 Mon Sep 17 00:00:00 2001 From: sergiopaniego Date: Tue, 5 May 2026 11:44:46 +0200 Subject: [PATCH 5/5] docs(mcp): add companion Colab notebook and badge Co-Authored-By: Claude Sonnet 4.6 --- docs/source/tutorials/mcp-environment.md | 2 + examples/mcp_environment.ipynb | 516 +++++++++++++++++++++++ 2 files changed, 518 insertions(+) create mode 100644 examples/mcp_environment.ipynb diff --git a/docs/source/tutorials/mcp-environment.md b/docs/source/tutorials/mcp-environment.md index b2e5732325..7d12a6e119 100644 --- a/docs/source/tutorials/mcp-environment.md +++ b/docs/source/tutorials/mcp-environment.md @@ -1,5 +1,7 @@ # MCP Tools in OpenEnv Environments +[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/mcp_environment.ipynb) + Most agentic work ends up needing the same thing: a way for the model to **call tools** and receive structured feedback, whether that is during RL training or offline evaluation. OpenEnv standardises that surface with **[MCP](https://modelcontextprotocol.io)** (Model Context Protocol), so the same tool interface works during training, eval, inference, and external serving. This tutorial covers the four paths you will walk in practice — wiring an MCP-backed environment into a training loop, using the same env for offline eval, inspecting the API underneath both of those, and building your own MCP environment when no existing one fits. ## Why MCP? diff --git a/examples/mcp_environment.ipynb b/examples/mcp_environment.ipynb new file mode 100644 index 0000000000..0c9e5cbc2e --- /dev/null +++ b/examples/mcp_environment.ipynb @@ -0,0 +1,516 @@ +{ + "nbformat": 4, + "nbformat_minor": 5, + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.11.0" + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# MCP Tools in OpenEnv Environments\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Most agentic work ends up needing the same thing: a way for the model to **call tools** and receive structured feedback, whether that is during RL training or offline evaluation. OpenEnv standardises that surface with **MCP** (Model Context Protocol), so the same tool interface works during training, eval, inference, and external serving. This tutorial covers the four paths you will walk in practice \u2014 wiring an MCP-backed environment into a training loop, using the same env for offline eval, inspecting the API underneath both of those, and building your own MCP environment when no existing one fits.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install -q \"openenv-core @ git+https://github.com/meta-pytorch/OpenEnv.git\"\n", + "!pip install -q \"openenv-echo-env @ git+https://github.com/meta-pytorch/OpenEnv.git#subdirectory=envs/echo_env\"\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Why MCP?\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If your tools are just local Python functions, you do not need MCP \u2014 pass them to your trainer directly (TRL, torchforge, whatever) and you are done. MCP earns its complexity when the tool surface has to exist as a **process boundary**, not a function call:\n", + "\n", + "- **The env runs elsewhere** \u2014 in a Docker container, a Hugging Face Space, a remote server. MCP is the transport that crosses that boundary.\n", + "- **You want to reuse someone else's env** \u2014 the OpenEnv catalog, third-party envs, and community hubs all expose their tools over MCP, so the same env works in your training run without rewriting its interface.\n", + "- **You want the env to be callable by other agents** \u2014 Claude Desktop, Cursor, inference servers, and any MCP-compatible client can plug into an MCP server. A private Python function doesn't get that for free.\n", + "- **You need tool discovery and schemas** \u2014 `list_tools()` + auto-generated JSON schemas are part of the protocol; models see them the same way they see any MCP server's tools.\n", + "\n", + "In short: MCP is the answer when your env is more than a helper function in your training script \u2014 when the same tools have to be usable from training, inference, and external clients without maintaining three interfaces.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### The dual API boundary\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Inside OpenEnv, MCP plays a specific role in a two-surface split:\n", + "\n", + "- **Training / orchestration infrastructure** uses the Gym-style control plane \u2014 `reset()`, `step()`, `state()` \u2014 over WebSocket (`/ws`). This is what the trainer needs to roll out episodes, compute rewards, and enforce termination.\n", + "- **Agents** use MCP tools over the `/mcp` JSON-RPC endpoint. Tools are what the model calls to act on the world.\n", + "\n", + "> **Note:** In simulation mode, MCP tool calls flow **through** `step()`. The trainer stays in control of timing, rewards, and termination; the MCP action types are just a standardised action schema.\n", + "\n", + "> **Note:** **MCP adoption in OpenEnv is still in flight.** RFC 003 proposes MCP as the standard interface for *all* agent-facing actions, but it is still **In Review**. Today only a handful of envs are MCP-backed: `echo_env` and `finqa_env` inherit from `openenv.core.env_server.mcp_environment.MCPEnvironment`; `calendar_env` uses a local wrapper with the same shape. The majority (`textarena_env` / Wordle, `openspiel_env`, `chess_env`, `browsergym_env`, and most others) still use custom action types. Before using the patterns in this tutorial against a specific env, check whether it inherits from an `MCPEnvironment` base; if not, the env's own action schema applies instead.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using MCP Tools in a Training Loop\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "An MCP-backed env is consumed like any other OpenEnv env from the trainer's side. At the atomic level, each agent action is:\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "from openenv.core.env_server.mcp_types import CallToolAction\n", + "\n", + "obs = env.step(CallToolAction(tool_name=..., arguments=...))\n", + "# obs.result.data \u2014 tool reply the model sees\n", + "# obs.reward \u2014 env's reward for this turn (may be None)\n", + "# obs.done \u2014 episode terminated\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Framework-agnostic rollout loop\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you drive the rollout yourself (a custom loop, torchforge, an external agent server), you own the full generation path and call `env.step()` directly:\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "obs = env.reset()\n", + "total_reward = 0.0\n", + "for turn in range(max_turns):\n", + " tool_call = model.decide(obs) # your agent picks a tool + args from the latest observation\n", + " obs = env.step(\n", + " CallToolAction(tool_name=tool_call.name, arguments=tool_call.arguments)\n", + " )\n", + " total_reward += obs.reward or 0.0\n", + " if obs.done:\n", + " break\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### TRL `environment_factory`\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[TRL](https://huggingface.co/docs/trl)'s `GRPOTrainer` takes an `environment_factory` class whose public methods auto-register as discoverable tools \u2014 the trainer then handles the multi-turn generation loop for you. The [Wordle GRPO tutorial](https://github.com/meta-pytorch/OpenEnv/blob/main/docs/source/tutorials/wordle-grpo.md) shows the full recipe with a non-MCP env. For an MCP-backed env, only the tool method bodies change; they call through to `env.step(CallToolAction(...))`:\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "def echo(self, message: str) -> str:\n", + " \"\"\"Echo back a message.\n", + "\n", + " Args:\n", + " message: The message to echo.\n", + " \"\"\"\n", + " obs = self.env.step(\n", + " CallToolAction(tool_name=\"echo_message\", arguments={\"message\": message})\n", + " )\n", + " self.reward = obs.reward or 0.0 # Echo has no domain reward \u2014 swap in FinQA or similar\n", + " return obs.result.data # what the model sees as the tool's reply\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`environment_factory` is a TRL API, not an MCP API. It works equally well with non-MCP envs (Wordle uses it with `TextArenaAction`), and MCP envs work equally well without it (the rollout-loop path above). They compose, but they are orthogonal.\n", + "\n", + "The rest of this tutorial is for the other paths: the API **underneath** `env.step(CallToolAction(...))` (useful when you need the full observation or want to debug), **using the same env for offline eval**, and **building your own MCP environment** from scratch.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Under the Hood: `CallToolAction` and `ListToolsAction`\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The two MCP action types are `ListToolsAction` (discover what's available) and `CallToolAction` (invoke one). They behave like any other Gym action \u2014 pass them to `step()` and inspect the returned observation.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Discovering tools\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from echo_env.server.echo_environment import EchoEnvironment\n", + "from openenv.core.env_server.mcp_types import ListToolsAction, ListToolsObservation\n", + "\n", + "# Running EchoEnvironment in-process (simulation mode).\n", + "# In production, swap this for: EchoEnv(base_url=\"https://openenv-echo-env.hf.space\")\n", + "env = EchoEnvironment()\n", + "env.reset()\n", + "\n", + "obs = env.step(ListToolsAction())\n", + "assert isinstance(obs, ListToolsObservation)\n", + "\n", + "for tool in obs.tools:\n", + " print(f\"{tool.name}: {tool.description}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each `Tool` carries a `name`, a `description`, and an `input_schema` (JSON Schema) describing the accepted arguments. The schema is what lets a language-model agent know which parameters to fill in when it emits a tool call.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Calling a tool\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from openenv.core.env_server.mcp_types import CallToolAction, CallToolObservation\n", + "\n", + "obs = env.step(\n", + " CallToolAction(\n", + " tool_name=\"echo_message\",\n", + " arguments={\"message\": \"Hello from MCP!\"},\n", + " )\n", + ")\n", + "\n", + "assert isinstance(obs, CallToolObservation)\n", + "print(obs.tool_name) # \"echo_message\"\n", + "print(obs.result.data) # \"Hello from MCP!\" \u2014 the raw tool return value\n", + "print(obs.error) # None\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`obs.result` is a `CallToolResult` wrapper exposing the return value in a few shapes: `.data` is the raw Python value the tool returned, `.structured_content` is its JSON-encoded form, and `.content` is the MCP protocol's list of typed content blocks (useful when a tool returns rich multi-part output). `obs.error` carries **every** failure mode \u2014 transport errors, unknown tool names, malformed arguments, **and** exceptions raised from inside the tool function itself (as `ToolErrorType.EXECUTION_ERROR`). On an error, `obs.result` is `None`. Always branch on `obs.error is None` before reading `obs.result.data`, or a tool that raised will look like a successful call that returned `None`.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Error handling\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "obs = env.step(\n", + " CallToolAction(tool_name=\"does_not_exist\", arguments={}),\n", + ")\n", + "\n", + "assert isinstance(obs, CallToolObservation)\n", + "print(obs.error.error_type) # ToolErrorType.TOOL_NOT_FOUND\n", + "print(obs.error.message)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `ToolError.error_type` enum (`TOOL_NOT_FOUND`, `INVALID_ARGS`, `EXECUTION_ERROR`, `TRANSPORT_ERROR`, `TIMEOUT`) lets training code distinguish between bugs in the agent, bugs in the environment, and transient infrastructure issues \u2014 which often warrant different reward signals.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `step(CallToolAction(...))` vs `call_tool()`\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Environment clients that inherit from `MCPToolClient` (such as `EchoEnv` and `FinQAEnv`) expose a shorter **async** `await env.call_tool(\"name\", arg=value)` helper. It still goes through the step loop and still updates rewards, step counts, and trajectory state, but returns the tool's raw return value directly instead of a `CallToolObservation` \u2014 and it **raises `RuntimeError`** on any `obs.error` (transport failure, unknown tool, or a tool exception), so you cannot branch on `error_type` without a `try/except`. Use `step(CallToolAction(...))` when you need the whole observation (reward, done, metadata, or graceful error classification); reach for `call_tool()` in async scripts where the raw result is all you care about and a failure is allowed to propagate.\n", + "\n", + "> **Note:** `call_tool()` is production-only: `MCPToolClient.__init__` raises `ValueError` if `mode != \"production\"`. In simulation contexts, route tool calls through `step(CallToolAction(...))` instead.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using MCP Tools for Evaluation\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The same mechanics work outside a training loop. For an offline eval \u2014 benchmarking a model's tool use on a static dataset, regression-testing a deployed agent, or scoring a policy \u2014 drop the trainer and drive the step loop yourself:\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "from echo_env.client import EchoEnv\n", + "from openenv.core.env_server.mcp_types import CallToolAction\n", + "\n", + "eval_env = EchoEnv(base_url=\"https://openenv-echo-env.hf.space\")\n", + "\n", + "results = []\n", + "for sample in eval_dataset:\n", + " await eval_env.reset()\n", + " tool_call = model.decide(sample) # your agent picks a tool + arguments\n", + " obs = await eval_env.step(\n", + " CallToolAction(tool_name=tool_call.name, arguments=tool_call.arguments),\n", + " )\n", + " results.append({\n", + " \"prompt\": sample.prompt,\n", + " \"reply\": obs.result.data if obs.error is None else None,\n", + " \"reward\": obs.reward or 0.0,\n", + " \"error\": obs.error,\n", + " })\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Pair the loop with a scoring function of your choice \u2014 the Reward Design guide covers common patterns (test-pass rate, LLM-as-judge quality, compliance gates) \u2014 and aggregate across the dataset. The eval harness integration in `src/openenv/core/evals/` is still evolving; until that bridge lands, this plain-Python loop is the canonical pattern.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Building an MCP Environment\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Reach for this path when no existing environment covers the tools your agent needs \u2014 e.g. a new coding sandbox, a game, a proprietary API wrapper. The provider side is small: subclass `MCPEnvironment`, create a `FastMCP` server, register tools with the `@mcp.tool` decorator, and pass the server to `super().__init__`. Here is the echo environment, trimmed from [`envs/echo_env/server/echo_environment.py`](https://github.com/meta-pytorch/OpenEnv/blob/main/envs/echo_env/server/echo_environment.py) down to the parts this tutorial covers:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from uuid import uuid4\n", + "\n", + "from fastmcp import FastMCP\n", + "\n", + "from openenv.core.env_server.mcp_environment import MCPEnvironment\n", + "from openenv.core.env_server.types import Action, Observation, State\n", + "\n", + "\n", + "class EchoEnvironment(MCPEnvironment):\n", + " SUPPORTS_CONCURRENT_SESSIONS = True\n", + "\n", + " def __init__(self):\n", + " mcp = FastMCP(\"echo_env\")\n", + "\n", + " @mcp.tool\n", + " def echo_message(message: str) -> str:\n", + " \"\"\"Echo back the provided message.\n", + "\n", + " Args:\n", + " message: The message to echo back\n", + "\n", + " Returns:\n", + " The same message that was provided\n", + " \"\"\"\n", + " return message\n", + "\n", + " @mcp.tool\n", + " def echo_with_length(message: str) -> dict:\n", + " \"\"\"Echo back the message with its length.\n", + "\n", + " Args:\n", + " message: The message to echo back\n", + "\n", + " Returns:\n", + " Dictionary with the message and its length\n", + " \"\"\"\n", + " return {\"message\": message, \"length\": len(message)}\n", + "\n", + " super().__init__(mcp)\n", + " self._state = State(episode_id=str(uuid4()), step_count=0)\n", + "\n", + " def reset(self, seed=None, episode_id=None, **kwargs) -> Observation:\n", + " self._state = State(episode_id=episode_id or str(uuid4()), step_count=0)\n", + " return Observation(done=False, reward=0.0, metadata={\"status\": \"ready\"})\n", + "\n", + " def _step_impl(self, action: Action, timeout_s=None, **kwargs) -> Observation:\n", + " # Called for non-MCP actions. Echo exposes MCP tools only,\n", + " # so anything that isn't ListToolsAction / CallToolAction is an error.\n", + " return Observation(\n", + " done=False,\n", + " reward=0.0,\n", + " metadata={\"error\": f\"Unknown action type: {type(action).__name__}\"},\n", + " )\n", + "\n", + " @property\n", + " def state(self) -> State:\n", + " return self._state\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A few things worth calling out:\n", + "\n", + "- **Docstring \u2192 schema.** `FastMCP` inspects each tool's signature and Google-style docstring to build the `input_schema` automatically. The `Args:` block becomes parameter descriptions, and type hints become JSON types. No hand-written schemas.\n", + "- **Reserved names.** `reset`, `step`, `state`, and `close` are reserved and cannot be tool names \u2014 they belong to the infrastructure boundary. Trying to register a tool with one of those names raises at construction time.\n", + "- **`_step_impl` is required, `step` is not.** `MCPEnvironment.step` already routes `ListToolsAction` and `CallToolAction` through the FastMCP server for you. Your subclass only has to implement `_step_impl`, which the base class calls for any **non-MCP** action. In pure-MCP environments like Echo it just returns an error observation; in environments that mix tool calls with other action types (e.g. a terminal \"submit\" action) it's where that extra dispatch lives.\n", + "- **Rewards and `done` still work.** Because MCP actions flow through `step()`, you can compute rewards, flip `done`, and emit metadata just like in any other OpenEnv environment.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Running the Demo End-to-End\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The repo ships a self-contained walkthrough at [`examples/echo_mcp_demo.py`](https://github.com/meta-pytorch/OpenEnv/blob/main/examples/echo_mcp_demo.py). Run it directly from the repo root:\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once the hosted Space is updated to the current `openenv-core`, you can run the full end-to-end demo against it:\n\n", + "```python\n", + "# Colab demo: connect to the hosted echo_env Space and run the four steps\n", + "from echo_env.client import EchoEnv\n", + "from openenv.core.env_server.mcp_types import ListToolsAction, CallToolAction\n", + "\n", + "demo_env = EchoEnv(base_url=\"https://openenv-echo-env.hf.space\")\n", + "await demo_env.reset()\n", + "\n", + "# 1. Discover tools\n", + "obs = await demo_env.step(ListToolsAction())\n", + "for tool in obs.tools:\n", + " print(f\"{tool.name}: {tool.description}\")\n", + "\n", + "# 2. Call echo_message\n", + "obs = await demo_env.step(CallToolAction(tool_name=\"echo_message\", arguments={\"message\": \"Hello from MCP!\"}))\n", + "print(obs.result.data)\n", + "\n", + "# 3. Call echo_with_length\n", + "obs = await demo_env.step(CallToolAction(tool_name=\"echo_with_length\", arguments={\"message\": \"Hello\"}))\n", + "print(obs.result.data)\n", + "\n", + "# 4. Error case\n", + "obs = await demo_env.step(CallToolAction(tool_name=\"does_not_exist\", arguments={}))\n", + "print(obs.error.error_type)\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Next Steps\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- **End-to-end training recipe** \u2014 the [Wordle GRPO tutorial](https://github.com/meta-pytorch/OpenEnv/blob/main/docs/source/tutorials/wordle-grpo.md) walks through a full GRPO training run with `environment_factory`. The wrapper-class shape is the same for an MCP-backed env; inside each tool method, build a `CallToolAction(tool_name=..., arguments={...})` instead of Wordle's single-field `TextArenaAction(message=guess)`.\n", + "- **MCP lifecycle details** \u2014 the MCP Environment Lifecycle guide covers `step()` vs `step_async()`, the `call_tool()` convenience path, and common debugging questions.\n", + "- **A richer MCP environment** \u2014 [`envs/finqa_env/`](https://github.com/meta-pytorch/OpenEnv/tree/main/envs/finqa_env) shows tool calls participating in episode progression, rewards, and terminal submission \u2014 not just a stateless echo.\n", + "- **Design rationale** \u2014 [RFC 003](https://github.com/meta-pytorch/OpenEnv/blob/main/rfcs/003-mcp-support.md) explains why OpenEnv picked MCP as the agent boundary and how tool-calling and CodeAct styles share the same plumbing.\n", + "- **Serving tools to an external agent** \u2014 the `/mcp` JSON-RPC endpoint is available alongside `/ws` on any MCP environment server. Point an MCP-compatible client at it for production inference without going through the step loop.\n" + ] + } + ] +} \ No newline at end of file