Feat/fresh grid world

docs/environments.md

@@ -338,6 +338,25 @@ The OpenEnv community has built a catalog of ready-to-run environments that cove
   </div>
 
 </div>
+<div class="environment-card">
+  <div class="environment-card__body">
+    <span class="environment-card__tag">Grid World</span>
+    <p class="environment-card__description">
+      A simple 5x5 grid world environment for testing and learning the OpenEnv framework.
+    </p>
+  </div>
+  <div class="environment-card__links">
+    <a class="environment-card__icon" href="https://github.com/yuvrajpant56/OpenEnv/blob/feat/fresh-grid-world/envs/grid_world_env/README.md" target="_blank" rel="noreferrer noopener" aria-label="Grid World docs">
+      <svg viewBox="0 0 24 24" aria-hidden="true" focusable="false">
+        <path d="M6 3c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h12c1.1 0 2-.9 2-2V9l-6-6H6zm8 1.5L18.5 9H14V4.5z" fill="currentColor"/>
+      </svg>
+    </a>
+    <a class="environment-card__icon environment-card__icon--hf" href="https://huggingface.co/spaces/yuvrajpant56/grid_world_env" target="_blank" rel="noreferrer noopener" aria-label="Grid World on Hugging Face">
+      <img src="https://huggingface.co/front/assets/huggingface_logo_noborder.svg" alt="" aria-hidden="true" />
+    </a>
+  </div>
+</div>
+
 
 <div class="environment-grid">
   <div class="environment-card">
@@ -360,4 +379,6 @@ The OpenEnv community has built a catalog of ready-to-run environments that cove
   </div>
 </div>
 
+
+
 > Want to publish your own environment? Head over to the [Build Your Own Environment](environment-builder.md) guide for a step-by-step walkthrough.

envs/grid_world_env/README.md

@@ -0,0 +1,264 @@
+---
+title: Grid World Env
+emoji: 🌍
+colorFrom: blue
+colorTo: indigo
+sdk: docker
+pinned: false
+---
+
+# Grid World Environment
+
+[Hugging Face Space](https://huggingface.co/spaces/yuvrajpant56/grid_world_env)
+
+This directory contains the implementation of a simple 5x5 Grid World environment, designed to serve two primary purposes within the OpenEnv ecosystem:
+
+1.  **A basic Reinforcement Learning (RL) testbed:** Providing a straightforward, deterministic environment for quick prototyping and testing of RL agents.
+2.  **A detailed "How-To" guide for building new OpenEnv environments:** Demonstrating the architectural patterns, best practices, and core components required to integrate a custom environment into the OpenEnv framework.
+
+---
+
+## 🚀 Environment Overview
+
+The Grid World environment features:
+
+* **Grid Size:** A 5x5 square grid.
+* **Agent:** Starts at position `(0,0)` (top-left).
+* **Goal:** Fixed at `(4,4)` (bottom-right).
+* **Actions:** `UP`, `DOWN`, `LEFT`, `RIGHT`.
+* **Dynamics:** Deterministic. An action always moves the agent one step in the chosen direction, unless it would move off the grid, in which case the agent stays in its current cell.
+* **Reward Function (Sparse):**
+    * `-0.1` for every step taken (a "living cost" or "step penalty").
+    * `+1.0` for reaching the goal at `(4,4)`. This also terminates the episode.
+* **Episode Termination:** The episode ends when the agent reaches the goal.
+
+### Example Gameplay
+
+Imagine the agent trying to find the goal:
+
+1.  **Reset:** Agent at `(0,0)` → `Obs(x=0, y=0, reward=0.0, done=False)`
+2.  **Step DOWN:** Agent moves to `(1,0)` → `Obs(x=1, y=0, reward=-0.1, done=False)`
+3.  **Step RIGHT:** Agent moves to `(1,1)` → `Obs(x=1, y=1, reward=-0.1, done=False)`
+4.  ...
+5.  **Step RIGHT (from 4,3):** Agent moves to `(4,4)` → `Obs(x=4, y=4, reward=1.0, done=True)`
+
+---
+
+## 🛠️ How to Build an OpenEnv Environment: A Detailed Guide
+
+This section explains the structure and key design choices of the Grid World environment.
+
+### 1. Scaffolding and Configuration
+
+This environment supports **multi-mode deployment**. It uses `pyproject.toml` for modern local development (via `uv`) and a `Dockerfile` for containerized deployment.
+
+### Directory Structure
+
+```text
+envs/grid_world_env
+├── server/
+│   ├── __init__.py           # Package initializer for the server side
+│   ├── app.py                # The FastAPI application entry point
+│   ├── Dockerfile            # Container definition (uses requirements.txt)
+│   ├── grid_world_environment.py # The core environment logic
+│   └── requirements.txt      # Dependencies for the Docker build
+├── __init__.py               # Package initializer for the client side
+├── client.py                 # Python client for interacting with the env server
+├── models.py                 # Pydantic data structures (Action, Observation)
+├── openenv.yaml              # OpenEnv metadata
+├── pyproject.toml            # Project configuration for local dev (uv)
+├── uv.lock                   # Exact dependency versions (Generated by uv)
+├── README.md
+└── test_grid_world.sh        # Integration test script (Docker based)
+
+
+# Core Components Explained
+
+This section dives into the specific code files that power the **Grid World**, explaining how the **OpenEnv** framework connects the data, logic, and server layers.
+
+---
+
+## 1. `models.py` — *The Data Contract*
+
+This file defines the strict “language” used for communication between the **Client (RL Agent)** and the **Server**. It relies on **Pydantic** to enforce type safety.
+
+### Key Components
+
+- **`MoveAction(str, Enum)`**  
+  Defines the allowed vocabulary for movement: `UP`, `DOWN`, `LEFT`, `RIGHT`.  
+  Using an `Enum` prevents *magic string* errors (e.g., sending `"up"` instead of `"UP"`).
+
+- **`GridWorldAction(Action)`**  
+  Wraps the movement enum in a standardized **OpenEnv** action structure.  
+  When the server receives a request, **FastAPI** automatically validates that the incoming JSON payload matches this schema.
+
+- **`GridWorldObservation(Observation)`**  
+  Defines exactly what the agent observes from the environment:
+  - `x`, `y`: Integer coordinates representing the agent’s position
+  - `reward`: Floating-point value (e.g., `-0.1`, `1.0`)
+  - `done`: Boolean flag indicating episode termination
+
+> **Note:**  
+> By inheriting from `pydantic.BaseModel` (via `Observation`), these classes automatically handle JSON serialization and deserialization.
+
+---
+
+## 2. `server/grid_world_environment.py` — *The Logic*
+
+This file contains the “physics engine” and rules of the environment. It translates abstract actions into concrete state transitions.
+
+### Core Responsibilities
+
+- **Inheritance**  
+  `GridWorldEnvironment` inherits from `openenv.core.env_server.Environment`, providing the standardized interface required by the OpenEnv server.
+
+- **`__init__` Method**  
+  - Sets static configuration:
+    - Grid size: `5 × 5`
+    - Goal location: `[4, 4]`
+  - Initializes the persistent state container.
+
+- **State Persistence (`self._state`)**  
+  - HTTP requests are stateless, so the environment instance must remember the agent’s position between calls.
+  - `self._state` (an instance of `openenv...State`) tracks:
+    - `step_count`
+    - `episode_id`
+    - `agent_x`, `agent_y`
+
+- **`step()` Logic**
+  - **Input:** Receives a validated `GridWorldAction`
+  - **Dynamics:** Applies movement rules and clamps coordinates using  
+    `max(0, min(..., grid_size - 1))` to prevent the agent from leaving the grid
+  - **Feedback:** Computes a sparse reward:
+    - `1.0` if `(x, y) == goal`
+    - `-0.1` otherwise  
+  - Returns a `GridWorldObservation`
+
+---
+
+## 3. `server/app.py` — *The API*
+
+This file is the “glue” that turns the environment logic into a running web service.
+
+### Key Elements
+
+- **`create_app` Utility**  
+  Instead of manually defining FastAPI routes, this file uses  
+  `openenv.core.env_server.create_app`.
+
+  It:
+  - Binds the environment logic (`GridWorldEnvironment`)
+  - Connects the data models (`GridWorldAction`, `GridWorldObservation`)
+  - Automatically generates standard endpoints:
+    - `/reset`
+    - `/step`
+    - `/state`
+    - `/health`
+
+- **`main()` Entry Point**  
+  Defines a `main()` function that calls `uvicorn.run`.  
+  This is what enables the `server = "..."` script in `pyproject.toml` to start the server.
+
+---
+
+## 4. `server/Dockerfile` — *The Container*
+
+This file defines how the environment is packaged for production or remote deployment.
+
+### Container Setup
+
+- **Base Image**  
+  Builds on `envtorch-base`, ensuring compatible system libraries.
+
+- **Dependencies**  
+  Copies and installs `server/requirements.txt`.  
+  This keeps the Docker image lightweight and focused only on server-side requirements.
+
+- **Execution**  
+  - Exposes port `8000`
+  - Defines the `CMD` to launch `uvicorn`  
+  The container is ready to accept HTTP requests immediately upon startup.
+
+---
+
+## 5. `pyproject.toml` — *Local Development*
+
+This file enables a modern local development workflow using **uv**.
+
+### Key Sections
+
+- **Project Metadata**
+  - Package name: `grid_world_env`
+  - Version information
+
+- **Dependencies**
+  Lists libraries required for local execution:
+  - `fastapi`
+  - `uvicorn`
+  - `gymnasium`
+  - `numpy`
+
+- **`[project.scripts]`**
+  Defines a shortcut command:
+
+  ```toml
+  server = "grid_world_env.server.app:main"
+
+
+# 🚀 Getting Started
+
+You can run the environment using **uv** (fastest for development) or **Docker** (best for deployment).
+
+---
+
+## Option 1: Local Development with `uv` (Recommended)
+
+Since this project is configured with `pyproject.toml`, you can run the server instantly.
+
+### Steps
+
+1. **Navigate to the environment folder**
+   ```bash
+   cd envs/grid_world_env
+   uv run server
+
+2. ** Visit the live Swagger UI in your Browser
+   ```bash
+   http://localhost:8000/docs
+
+
+ ## Option 2: Docker Integration Test
+
+To build the full container and run the integration test suite (simulating a production deployment):
+
+---
+
+### Steps
+
+1. **Navigate to the root OpenEnv directory**
+
+2. **Run the test script**
+   ```bash
+   ./envs/grid_world_env/test_grid_world.sh
+
+
+Builds the Docker image
+
+Starts the container
+
+Runs a series of curl requests to verify functionality
+
+Cleans up containers and images after completion
+
+
+## Conclusion
+
+This Grid World environment serves as the reference implementation for building environments in OpenEnv. By following this pattern, custom environments remain:
+
+Portable across local and containerized setups
+
+Strictly typed through Pydantic models
+
+Deployment-ready for development, testing, and production workflows
+---
+

envs/grid_world_env/__init__.py

@@ -0,0 +1,13 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Grid World Environment - A simple test environment for HTTP server."""
+
+from .client import GridWorldEnv
+from .models import GridWorldAction, GridWorldObservation
+
+__all__ = ["GridWorldAction", "GridWorldObservation", "GridWorldEnv"]
+

envs/grid_world_env/client.py

@@ -0,0 +1,62 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+
+from __future__ import annotations
+
+try:
+    from openenv.core.env_client import EnvClient
+    from openenv.core.client_types import StepResult
+    from openenv.core.env_server.types import State
+except ImportError:
+    from core.env_client import EnvClient
+    from core.client_types import StepResult
+    from core.env_server.types import State
+
+
+from .models import GridWorldAction, GridWorldObservation, MoveAction
+
+
+
+class GridWorldEnv(EnvClient[GridWorldAction, GridWorldObservation, State]):
+    """
+    A WebSocket-based client for interacting with the GridWorld environment.
+
+    This client inherits from EnvClient and is configured with the
+    GridWorld Pydantic models for automatic (de)serialization.
+    """
+
+
+    def step_move(self, move: MoveAction) -> StepResult[GridWorldObservation]:
+        """
+        Helper method to send a simple move action.
+
+        Args:
+            move: The MoveAction enum (e.g., MoveAction.UP)
+        """
+        action_payload = GridWorldAction(action=move)
+        # 'super().step' comes from the base HTTPEnvClient
+        return super().step(action_payload)
+
+    # --- REQUIRED ABSTRACT METHODS (The Missing Pieces) ---
+
+    def _step_payload(self, action: GridWorldAction) -> dict:
+        """Convert the Pydantic action model to a dictionary."""
+        # Uses Pydantic v2 'model_dump'. If this fails, try 'action.dict()'
+        return action.model_dump()
+
+    def _parse_result(self, data: dict) -> StepResult[GridWorldObservation]:
+        """Convert the raw dictionary response into a typed StepResult."""
+        return StepResult(
+            observation=GridWorldObservation(**data["observation"]),
+            reward=data["reward"],
+            done=data["done"],
+            info=data.get("info", {})
+        )
+
+    def _parse_state(self, data: dict) -> State:
+        """Convert the raw state dictionary into a State object."""
+        return State(**data)