Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ From the repo root:
doc-builder preview openenv docs/source
```

Then open the URL printed in the terminal (usually `http://localhost:5173/openenv/main/en/index`).
Then open the URL printed in the terminal and navigate to `/index` (usually `http://localhost:5173/index`).

## Adding an Environment to the Docs

Expand Down
70 changes: 39 additions & 31 deletions docs/source/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@
title: Getting Started
title: Get Started
- sections:
- local: guides/first-environment
title: Your First Environment
- local: getting_started/environment-builder
title: Deploying an Env
- local: guides/auto-discovery
Expand All @@ -16,11 +14,15 @@
- local: guides/rl-integration
title: RL Training
- local: guides/harbor-provider-qualification
title: Harbor Provider Qualification
title: Harbor Spec
- local: guides/rewards
title: Rewards
- local: guides/concepts
title: Concepts
title: Core Concepts
- local: tutorials/mcp-environment
title: MCP Environments
- local: tutorials/rubrics
title: Rubrics
- isExpanded: false
sections:
- local: guides/customizing-web-ui
Expand All @@ -37,34 +39,40 @@
title: Connecting to Servers
- local: guides/runtime-providers
title: Runtime Providers
title: Advanced Guides
title: Guides
title: Advanced Concepts
title: Concepts
- sections:
- local: tutorials/index
title: Overview
- local: tutorials/openenv-tutorial
title: Hello World
- local: tutorials/end-to-end-walkthrough
title: Train a reasoning model with TRL
- local: tutorials/mcp-environment
title: MCP Environments
- local: tutorials/rubrics
title: Rubrics
- local: tutorials/wordle-grpo
title: RL Training with TRL
- local: tutorials/rl-training-2048
title: RL Training with Unsloth
- local: tutorials/evaluation-inspect
title: Evaluating with Inspect AI
- local: tutorials/browsergym-harness
title: RL Training with an Agentic Harness
- local: tutorials/opencode-agent-grpo
title: Coding Agent Training with TRL
- local: tutorials/sft-warmup
title: SFT Training with Environments
- local: tutorials/pi-agent-grpo
title: Training a Real Coding Agent (Pi)
title: Tutorials
- sections:
- local: tutorials/index
title: Overview
- local: tutorials/openenv-tutorial
title: Hello World
- local: guides/first-environment
title: Your First Environment
title: Basics
- sections:
- local: tutorials/end-to-end-walkthrough
title: Train a Reasoning Model
- local: tutorials/wordle-grpo
title: RL Training with TRL
- local: tutorials/rl-training-2048
title: RL Training with Unsloth
- local: tutorials/sft-warmup
title: SFT Training with Environments
title: Training
- sections:
- local: tutorials/browsergym-harness
title: Agentic Harness
- local: tutorials/opencode-agent-grpo
title: OpenCode
- local: tutorials/pi-agent-grpo
title: Pi
title: Harnesses
- sections:
- local: tutorials/evaluation-inspect
title: Evaluating with Environments
title: Evals
title: Learn
- isExpanded: false
sections:
- local: environments
Expand Down
2 changes: 1 addition & 1 deletion docs/source/environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ The OpenEnv community has built a catalog of ready-to-run environments that cove
</div>
<div class="border dark:border-gray-700 p-5 rounded-lg shadow">
<div class="font-bold mb-2">Web Search</div>
<p class="text-sm">Web search environment for RL research with configurable grids, partial observability, and customizable rewards.</p>
<p class="text-sm">Web search environment that lets agents query Google Search through the Serper.dev API and receive formatted results.</p>
<div class="flex gap-2 mt-3">
<a href="environments/websearch" class="!no-underline border dark:border-gray-700 px-3 py-1 rounded text-sm hover:shadow">📄 Docs</a>
<a href="https://huggingface.co/spaces/lawhy/web_search" class="!no-underline border dark:border-gray-700 px-3 py-1 rounded text-sm hover:shadow">🤗 HF</a>
Expand Down
2 changes: 1 addition & 1 deletion docs/source/environments/agent_world_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ A background cleanup daemon automatically kills sessions idle longer than `MAX_I

## Full Agent Interaction Example

See [`examples/agent_world_model/example_usage.py`](../../examples/agent_world_model/example_usage.py) for a complete example of an LLM-powered agent that:
See [`examples/agent_world_model/example_usage.py`](https://github.com/huggingface/OpenEnv/blob/main/examples/agent_world_model/example_usage.py) for a complete example of an LLM-powered agent that:

1. Discovers available tools via `list_tools`
2. Iteratively calls tools to accomplish the task
Expand Down
2 changes: 1 addition & 1 deletion docs/source/environments/calendar.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,4 @@ python client.py --scenario scenario_config.json
Output will be saved to `response_output/` folder with execution details, tool calls, and verification results.

**Notebook Evaluation:**
For interactive evaluation and testing, use the: [`Jupyter notebook`](client_notebooks/OpenEnv_and_mcp_Single_Gym_Client_Meta_Turing.ipynb)
For interactive evaluation and testing, use the: [`Jupyter notebook`](https://github.com/huggingface/OpenEnv/blob/main/envs/calendar_env/client_notebooks/OpenEnv_and_mcp_Single_Gym_Client_Meta_Turing.ipynb)
18 changes: 9 additions & 9 deletions docs/source/environments/carla.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,11 +152,11 @@ result = await env.reset(scenario_config={

## Examples

The [`examples/carla_env/`](../../examples/carla_env/) directory contains inference scripts. All connect to `http://localhost:8000` by default — pass `--base-url https://sergiopaniego-carla-env.hf.space` for the live Space.
The [`examples/carla_env/`](https://github.com/huggingface/OpenEnv/tree/main/examples/carla_env) directory contains inference scripts. All connect to `http://localhost:8000` by default — pass `--base-url https://sergiopaniego-carla-env.hf.space` for the live Space.

### Trolley Problems

**[trolley_problems.py](../../examples/carla_env/trolley_problems.py)** — LLM evaluation across all trolley scenarios.
**[trolley_problems.py](https://github.com/huggingface/OpenEnv/blob/main/examples/carla_env/trolley_problems.py)** — LLM evaluation across all trolley scenarios.

```bash
uv run python trolley_problems.py --model qwen3-235b --scenario classic-3v1
Expand All @@ -168,16 +168,16 @@ Available keys: `equal-1v1`, `saves-3v0`, `deadzone-3v1`, `classic-3v1`, `classi

### Maze Navigation

**[maze_navigation.py](../../examples/carla_env/maze_navigation.py)** — LLM navigation with rolling action history.
**[maze_navigation.py](https://github.com/huggingface/OpenEnv/blob/main/examples/carla_env/maze_navigation.py)** — LLM navigation with rolling action history.

```bash
uv run python maze_navigation.py --model qwen3-235b --scenario maze-1
uv run python maze_navigation.py --model gpt-5.2 --scenario maze-1 --save-images
uv run python maze_navigation.py --model qwen3-235b
uv run python maze_navigation.py --model gpt-5.2 --save-images
```

### Free-Roam Navigation

**[free_roam_navigation.py](../../examples/carla_env/free_roam_navigation.py)** — LLM navigation in open traffic.
**[free_roam_navigation.py](https://github.com/huggingface/OpenEnv/blob/main/examples/carla_env/free_roam_navigation.py)** — LLM navigation in open traffic.

```bash
uv run python free_roam_navigation.py --model qwen3-235b
Expand All @@ -186,7 +186,7 @@ uv run python free_roam_navigation.py --model qwen3-235b --scenario free-roam-tr

### Autopilot Baseline (No LLM)

**[autopilot_navigation.py](../../examples/carla_env/autopilot_navigation.py)** — CARLA's built-in navigation agent.
**[autopilot_navigation.py](https://github.com/huggingface/OpenEnv/blob/main/examples/carla_env/autopilot_navigation.py)** — CARLA's built-in navigation agent.

```bash
uv run python autopilot_navigation.py --scenario maze-1
Expand All @@ -195,7 +195,7 @@ uv run python autopilot_navigation.py --scenario free-roam-default --behavior ca

### Rubric Reward Demo (No LLM)

**[rubric_autopilot_example.py](../../examples/carla_env/rubric_autopilot_example.py)** — Raw vs rubric rewards side-by-side.
**[rubric_autopilot_example.py](https://github.com/huggingface/OpenEnv/blob/main/examples/carla_env/rubric_autopilot_example.py)** — Raw vs rubric rewards side-by-side.

```bash
uv run python rubric_autopilot_example.py --scenario free-roam-default
Expand All @@ -222,7 +222,7 @@ Hugging Face models use [Inference Providers](https://huggingface.co/docs/infere

## Rubrics for RL Training

The environment includes rubrics following the [OpenEnv rubric system](../../rfcs/004-rubrics.md). Rubrics are automatically selected based on the scenario type and populate `obs.rubric_reward` alongside the raw `obs.reward` on each step.
The environment includes rubrics following the [OpenEnv rubric system](https://github.com/huggingface/OpenEnv/blob/main/rfcs/004-rubrics.md). Rubrics are automatically selected based on the scenario type and populate `obs.rubric_reward` alongside the raw `obs.reward` on each step.

**CarlaTrolleyRubric** — For trolley/action-bias scenarios. Returns 0.0 on intermediate steps, then the terminal reward at episode end. Supports temporal discounting (`gamma`) for credit assignment.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/environments/finrl.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,4 +347,4 @@ BSD 3-Clause License (see LICENSE file in repository root)

- [FinRL Paper](https://arxiv.org/abs/2011.09607)
- [FinRL GitHub](https://github.com/AI4Finance-Foundation/FinRL)
- [OpenEnv Documentation](README.md)
- [OpenEnv Documentation](https://github.com/huggingface/OpenEnv/blob/main/envs/finrl_env/README.md)
5 changes: 3 additions & 2 deletions docs/source/environments/git.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The Git Environment connects to a **shared external Gitea service** for optimal
from envs.git_env import GitAction, GitEnv

# Create environment from Docker image
git_env = GitEnv.from_docker_image("git-env:latest")
git_env = GitEnv.from_docker_image("git-env:latest").sync()

# Reset environment
result = git_env.reset()
Expand Down Expand Up @@ -178,7 +178,8 @@ docker run \

Then in your training code, environments automatically reset to the configured state.

See [`examples/local_git_env.py`](../../../examples/local_git_env.py) for complete working example.
See [`examples/local_git_env.py`](https://github.com/huggingface/OpenEnv/blob/main/examples/local_git_env.py) for a complete
synchronous example that resolves the Docker bootstrap handle with `.sync()`.

## Project Structure

Expand Down
1 change: 0 additions & 1 deletion docs/source/environments/opencode.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,4 +304,3 @@ opencode_env/
- [OpenEnv docs](https://huggingface.co/docs/openenv)
- [OpenCode CLI](https://opencode.ai/docs/cli/)
- [E2B Python SDK](https://e2b.dev/docs)
- [HF Inference Providers logprob matrix](../../../DOCS/HF/hf_inference_providers_logprobs.md)
26 changes: 13 additions & 13 deletions docs/source/environments/repl.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,14 @@ Inside the REPL, the model can:

Main modules:

- [`client.py`](client.py): remote async OpenEnv client
- [`local.py`](local.py): explicit in-process local env helper
- [`runner.py`](runner.py): local RLM orchestration loop
- [`recursive_backends.py`](recursive_backends.py): direct and recursive backend implementations
- [`recursive_controller.py`](recursive_controller.py): server-side backend/broker composition
- [`rubrics.py`](rubrics.py): reward rubrics (OpenEnv RFC 004)
- [`server/repl_environment.py`](server/repl_environment.py): server-side execution environment
- [`server/app.py`](server/app.py): OpenEnv HTTP server app and env factory
- [`client.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/client.py): remote async OpenEnv client
- [`local.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/local.py): explicit in-process local env helper
- [`runner.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/runner.py): local RLM orchestration loop
- [`recursive_backends.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/recursive_backends.py): direct and recursive backend implementations
- [`recursive_controller.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/recursive_controller.py): server-side backend/broker composition
- [`rubrics.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/rubrics.py): reward rubrics (OpenEnv RFC 004)
- [`server/repl_environment.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/server/repl_environment.py): server-side execution environment
- [`server/app.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/server/app.py): OpenEnv HTTP server app and env factory

## What Works Today

Expand Down Expand Up @@ -256,7 +256,7 @@ Run the local server:
PYTHONPATH=src:envs uvicorn envs.repl_env.server.app:app --host 127.0.0.1 --port 8000
```

The server uses a proper OpenEnv environment factory in [`server/app.py`](server/app.py).
The server uses a proper OpenEnv environment factory in [`server/app.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/server/app.py).

## API Surface

Expand Down Expand Up @@ -355,7 +355,7 @@ result = env.execute("answer['ready'] = True")

## Prompt Utilities

[`prompts.py`](prompts.py) contains the current message-building and parsing helpers used by the examples and runner.
[`prompts.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/prompts.py) contains the current message-building and parsing helpers used by the examples and runner.

Important exports:

Expand All @@ -371,14 +371,14 @@ These prompts were updated to reflect the actual helper surface the environment

## Examples

- [`examples/repl_with_llm.py`](../../examples/repl_with_llm.py)
- [`examples/repl_oolong_simple.py`](../../examples/repl_oolong_simple.py)
- [`examples/repl_with_llm.py`](https://github.com/huggingface/OpenEnv/blob/main/examples/repl_with_llm.py)
- [`examples/repl_oolong_simple.py`](https://github.com/huggingface/OpenEnv/blob/main/examples/repl_oolong_simple.py)

Default hosted model in the examples is currently `Qwen/Qwen3.5-9B`, but real hosted inference still depends on provider availability and token access.

## Environment Variables

Server-side configuration in [`server/app.py`](server/app.py):
Server-side configuration in [`server/app.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/server/app.py):

- `LLM_MODEL`
- `HF_TOKEN`
Expand Down
2 changes: 1 addition & 1 deletion docs/source/environments/wildfire.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Agents must contain spreading fires using **water**, **firebreaks**, and **timin
[![Docker](https://img.shields.io/badge/docker-ready-blue)](https://hub.docker.com/)
[![Python](https://img.shields.io/badge/python-3.10+-green)](https://www.python.org/)
[![FastAPI](https://img.shields.io/badge/backend-fastapi-teal)](https://fastapi.tiangolo.com/)
[![License](https://img.shields.io/badge/license-MIT-lightgrey)](LICENSE)
[![License](https://img.shields.io/badge/license-BSD--3--Clause-lightgrey)](https://github.com/huggingface/OpenEnv/blob/main/LICENSE)

---

Expand Down
25 changes: 16 additions & 9 deletions docs/source/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,26 +25,33 @@ A few integrations ship as optional extras. Install them with

## Try an Environment

Use `AutoEnv` and `AutoAction` when you want OpenEnv to find the matching client
and action classes for an installed or discoverable environment.
Use `AutoEnv` to load the Echo client from its Hugging Face Space. It asks for
confirmation before installing the environment package, then connects to the
running Space. Echo exposes its actions as MCP tools.

```python
from openenv import AutoAction, AutoEnv
from openenv import AutoEnv

Comment thread
cursor[bot] marked this conversation as resolved.
Resolved
env = AutoEnv.from_env("echo")
EchoAction = AutoAction.from_env("echo")
env = AutoEnv.from_env(
"openenv/echo_env",
base_url="https://openenv-echo-env.hf.space",
)

with env.sync() as client:
result = client.reset()
print(result.observation.echoed_message) # "Echo environment ready!"
print(result.observation.metadata["message"]) # "Echo environment ready!"

result = client.step(EchoAction(message="Hello, OpenEnv!"))
print(result.observation.echoed_message) # "Hello, OpenEnv!"
message = client.call_tool("echo_message", message="Hello, OpenEnv!")
print(message) # "Hello, OpenEnv!"
```

`AutoEnv.from_env()` accepts the common name forms:
For installed environment packages, `AutoEnv.from_env()` also accepts the common
name forms below. Without a `base_url`, these start a local Docker container and
require Docker and the environment image:

```python
from openenv import AutoEnv

AutoEnv.from_env("echo")
AutoEnv.from_env("echo-env")
AutoEnv.from_env("echo_env")
Expand Down
5 changes: 3 additions & 2 deletions docs/source/getting_started/environment-builder.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,9 +200,10 @@ app = create_app(create_my_environment, MyAction, MyObservation, env_name="my_en
# client.py
from openenv.core.env_client import EnvClient
from openenv.core.client_types import StepResult
from .models import MyAction, MyObservation, MyState
from openenv.core.env_server.types import State
from .models import MyAction, MyObservation

class MyEnv(EnvClient[MyAction, MyObservation, MyState]):
class MyEnv(EnvClient[MyAction, MyObservation, State]):
def _step_payload(self, action: MyAction) -> dict:
return {"command": action.command, "parameters": action.parameters}

Expand Down
2 changes: 1 addition & 1 deletion docs/source/guides/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ with env.sync() as client:
with env.sync() as client:
result = client.reset()

while not result.terminated:
while not result.done:
obs = result.observation
action = decide_action(obs)
result = client.step(action)
Expand Down
9 changes: 2 additions & 7 deletions docs/source/reference/cli.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CLI

The `openenv` CLI provides a set of commands for building, validating, and pushing environments to Hugging Face Spaces or a custom Docker registry. For an end-to-end tutorial on building environments with OpenEnv, see the [building an environment](../getting_started/environment-builder.md) guide.
The `openenv` CLI provides a set of commands for building, validating, and pushing environments to Hugging Face Spaces or a custom Docker registry. For an end-to-end tutorial on building environments with OpenEnv, see the [building an environment](../getting_started/environment-builder) guide.

## `openenv init`

Expand All @@ -27,12 +27,7 @@ openenv import path/to/source --name my_env --output-dir ./envs
openenv import path/to/source --name my_env --output-dir ./envs --env-class MyEnv
```

```{eval-rst}
.. automodule:: openenv.cli.commands.import_env
:members:
:undoc-members:
:show-inheritance:
```
[[autodoc]] openenv.cli.commands.import_env.import_env

## `openenv build`

Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/evaluation-inspect.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Evaluating agents with Inspect AI
# Evaluating with Environments

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/OpenEnv/blob/main/examples/evaluation_inspect.ipynb)

Expand Down
Loading
Loading