Feat/fresh grid world

[yuvrajpant56]

Summary

This PR introduces grid_world_env, a canonical 5x5 Grid World environment. It serves two primary purposes:

1. RL Testbed: A lightweight, deterministic environment (Start: [0,0], Goal: [4,4]) for verifying agent logic and server-client communication.
2. Reference Implementation: A "How-To" example for building OpenEnv environments using modern pyproject.toml configuration and uv dependency management, while retaining full Docker compatibility for deployment.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• New environment
• Refactoring

Alignment Checklist

Before submitting, verify:

• I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
• I have checked .claude/docs/INVARIANTS.md and no invariants are violated
• I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

• Not required (bug fix, docs, minor refactoring)
• RFC exists: #___
• RFC needed (will create before merge)

Test Plan

This environment supports multi-mode testing. I have verified it using the following methods:

1. Local Server (uv)
Verified the server starts and responds to API requests locally:

cd envs/grid_world_env
uv run server
# Verified endpoints via Swagger UI at http://localhost:8000/docs

##Docker Integration Test Ran the full container build and integration test suite script:

```bash

./envs/grid_world_env/test_grid_world.sh
Result: Passed all checks (Health, Reset, Step, State).

## Unit Tests Added and verified Python unit tests for the client-server interaction:

```bash

pytest tests/envs/test_grid_world.py

[yuvrajpant56]

@burtenshaw, thank you for the detailed review. I have addressed all the feedback provided in the previous review cycle.

Summary of Changes:

Directory Structure: Moved the environment from src/envs/ to envs/grid_world_env/ to match the repo standard.

Architecture: Switched to EnvClient (WebSocket) and create_app factory pattern as requested.

Imports: Fixed all import paths to use openenv.core with try/except blocks for standalone support.

Standards: Converted data models to Pydantic, added Meta license headers, and removed debug print statements.

Testing: Added tests/envs/test_grid_world.py (pytest) and verified the Docker integration script works with the corrected paths.

All local and Docker integration tests are passing. Ready for re-review!

[greptile-apps]

Greptile Summary

This PR adds grid_world_env, a clean 5x5 Grid World environment that serves as both an RL testbed and reference implementation. The environment correctly implements the OpenEnv architecture with proper client-server separation, Pydantic models for type safety, and dual deployment modes (uv for local development, Docker for production).

Key Changes:

• Added complete Grid World environment with deterministic navigation (start: [0,0], goal: [4,4])
• Implements sparse reward function (-0.1 per step, +1.0 for goal)
• Properly uses Environment base class with standard reset(), step(), and state API
• WebSocket-based client using EnvClient generic pattern
• Comprehensive README serving as "How-To" guide for new environments
• Integration test script and unit tests

Issues Found:

• Commented-out code: Extensive blocks of old implementation attempts left in models.py, client.py, server/grid_world_environment.py, server/app.py, and server/Dockerfile
• Debugging comments: Comments like "FIX 1:", separator lines in production code
• Unnecessary dependencies: ale-py and pandas in requirements.txt not used by Grid World
• Missing newlines: End-of-file newline missing in 3 Python files

Alignment Status:

• ✅ Follows Gymnasium API pattern (reset, step, state)
• ✅ Proper client-server separation (no violations)
• ✅ Rewards computed inside environment boundary
• ✅ Type-safe with Pydantic models
• ✅ Uses standard State from framework (not custom state model)
• ✅ No security issues or credential exposure

Confidence Score: 4/5

• Safe to merge after cleaning up commented-out code and debug comments
• The core implementation is solid and follows OpenEnv patterns correctly. All architectural invariants are satisfied, and the environment logic is straightforward and correct. Score reduced from 5 to 4 due to code cleanliness issues (extensive commented-out code, debugging comments, unused dependencies) that should be addressed before merge.
• Focus on server/grid_world_environment.py, models.py, and client.py to remove commented-out code blocks. Also clean up server/Dockerfile and server/requirements.txt.

Important Files Changed

Filename	Overview
envs/grid_world_env/models.py	Defines action and observation models correctly; contains commented-out code and missing newline
envs/grid_world_env/client.py	Implements WebSocket client correctly; contains commented-out imports and missing newline
envs/grid_world_env/server/grid_world_environment.py	Core environment logic is correct; contains extensive commented-out code blocks and debugging comments
envs/grid_world_env/server/Dockerfile	Docker configuration is correct; large commented-out code block at the top
envs/grid_world_env/server/requirements.txt	Contains unnecessary dependencies (ale-py, pandas) not used by Grid World

Sequence Diagram

sequenceDiagram
    participant Client as GridWorldEnv (Client)
    participant Server as FastAPI Server
    participant Env as GridWorldEnvironment
    participant State as Internal State

    Note over Client,State: Environment Initialization
    Client->>Server: WebSocket Connect
    Server->>Env: Create Environment Instance
    Env->>State: Initialize State (episode_id, step_count=0)
    Env->>Env: Set agent position (0,0) and goal (4,4)
    
    Note over Client,State: Reset Episode
    Client->>Server: POST /reset
    Server->>Env: reset()
    Env->>State: Reset step_count=0, new episode_id
    Env->>Env: Reset agent_x=0, agent_y=0
    Env-->>Server: GridWorldObservation(x=0, y=0, reward=0.0, done=False)
    Server-->>Client: Observation response
    
    Note over Client,State: Step Action
    Client->>Client: Create GridWorldAction(action=MoveAction.DOWN)
    Client->>Server: POST /step {action: "DOWN"}
    Server->>Env: step(GridWorldAction)
    Env->>State: Increment step_count
    Env->>Env: Update position based on action
    Env->>Env: Clamp to grid boundaries [0,4]
    Env->>Env: Check if goal reached
    Env->>Env: Calculate reward (-0.1 or +1.0)
    Env-->>Server: GridWorldObservation(x, y, reward, done)
    Server-->>Client: StepResult with observation
    
    Note over Client,State: Query State
    Client->>Server: GET /state
    Server->>Env: state property
    Env->>State: Return State(episode_id, step_count)
    State-->>Server: State object
    Server-->>Client: State response

Loading

[greptile-apps]

_{13 files reviewed, 14 comments}

_{Edit Code Review Agent Settings | Greptile}

[yuvrajpant56]

@burtenshaw, I have updated docs/environments.md with the new environment card and link to the live Hugging Face Space.

The Hugging Face deployment is verified and running (Green). This PR now includes:

• Complete, clean environment code in envs/grid_world_env
• Passing Docker integration tests
• Live Demo on HF
• Documentation entry in the gallery

Ready for final review!