docs(tutorials): add Inspect AI evaluation tutorial and notebook

[sergiopaniego]

Summary

Adds docs/source/tutorials/evaluation-inspect.md and a matching examples/evaluation_inspect.ipynb Colab notebook covering how to evaluate OpenEnv agents with Inspect AI's InspectAIHarness. The tutorial walks through defining a Task (dataset + solver + scorer), using asyncio.Semaphore to serialise concurrent env connections, and running a structured EvalResult via InspectAIHarness. Also adds model_base_url and max_connections forwarding to InspectAIHarness.run(), discovered as needed while writing the tutorial.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• New environment
• Refactoring

Alignment Checklist

Before submitting, verify:

• I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
• I have checked .claude/docs/INVARIANTS.md and no invariants are violated
• I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

• Not required (bug fix, docs, minor refactoring)
• RFC exists: #___
• RFC needed (will create before merge)

Test Plan

• Build docs locally and confirm the page renders: cd docs && make html, then open docs/_build/html/tutorials/evaluation-inspect.html
• Verify the Next Steps links resolve to https://meta-pytorch.org/OpenEnv/tutorials/
• Open examples/evaluation_inspect.ipynb in Jupyter (or Colab); run all cells with a valid OpenAI or Anthropic API key — expected result: {'accuracy': 1.0} printed after the harness run
• Run the existing harness unit tests: PYTHONPATH=src:envs uv run pytest tests/core/test_evals/test_inspect_harness.py -v — all 23 pass
• Confirm no lint regressions in touched files: uv run ruff format src/openenv/core/evals/inspect_harness.py --check && uv run ruff check src/openenv/core/evals/inspect_harness.py

Claude Code Review

Automated Checks

• Lint: PASS — inspect_harness.py was reformatted as part of this PR; pre-existing failures in chat_env, repl_env, textarena_env are not in scope
• Debug code: CLEAN — no debug statements in any files touched by this PR

Open RFCs Context

• RFC 005 (agentic-harnesses) covers the EvalHarness infrastructure this tutorial builds on. Changes here are additive and compatible — no conflicts.

Tier 1: Fixes Required

None.

Tier 2: Alignment Discussion

None identified. The tutorial uses MCPToolClient (correct for MCP envs), keeps all reward logic inside the environment, and does not expose reset controls to the agent. The harness change is a transparent parameter pass-through to inspect_ai.eval().

Summary

• 0 mechanical issues
• 0 alignment points for human review
• 0 RFC conflicts

[greptile-apps]

Greptile Summary

This PR adds a new Inspect AI evaluation tutorial (evaluation-inspect.md) and matching Colab notebook (evaluation_inspect.ipynb), and extends InspectAIHarness with two transparent pass-through parameters (model_base_url, max_connections) forwarded directly to inspect_ai.eval(). The changes are additive, align with all stated invariants (rewards stay inside the environment, agents cannot call reset, MCP is used for tool interaction), and no RFC is required.

Confidence Score: 4/5

Safe to merge; only a P2 style concern in the notebook's API-key cell.

All code changes are correct and minimal. The single P2 finding (hardcoded API key placeholder pattern in the notebook) is a best-practice concern, not a functional defect. P2s alone cap at 4/5.

examples/evaluation_inspect.ipynb — API key cell pattern.

Important Files Changed

Filename	Overview
src/openenv/core/evals/inspect_harness.py	Adds model_base_url and max_connections pass-through to inspect_ai.eval() — minimal, correct, well-documented change with no logic issues.
docs/source/tutorials/evaluation-inspect.md	New tutorial covering InspectAIHarness usage; content is accurate, aligns with principles, and follows existing tutorial structure.
examples/evaluation_inspect.ipynb	Companion Colab notebook; otherwise clean but the API-key cell pattern encourages hardcoding secrets in notebook cells (P2).
docs/source/tutorials/index.md	Adds the new tutorial entry to the toctree — trivial, correct change.

Sequence Diagram

sequenceDiagram
    participant User
    participant InspectAIHarness
    participant inspect_eval
    participant Solver
    participant MCPToolClient

    User->>InspectAIHarness: run_from_config(EvalConfig)
    InspectAIHarness->>inspect_eval: eval(task, model, max_connections, model_base_url, ...)
    loop For each Sample
        inspect_eval->>Solver: solve(state, generate)
        Solver->>Solver: generate(state)  [LLM call]
        Solver->>MCPToolClient: reset()
        Solver->>MCPToolClient: call_tool("echo_message", ...)
        MCPToolClient-->>Solver: echoed result
        Solver->>MCPToolClient: close()
        Solver-->>inspect_eval: updated TaskState
    end
    inspect_eval-->>InspectAIHarness: EvalLog[]
    InspectAIHarness->>InspectAIHarness: _extract_scores(logs[0])
    InspectAIHarness-->>User: EvalResult { scores: {accuracy: 1.0} }

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
examples/evaluation_inspect.ipynb:60
**Hardcoded API key placeholder encourages insecure notebook hygiene**

Setting the key as a string literal in a cell trains users to put their real secret at the same position — notebooks get accidentally committed or shared, leaking credentials. Using `getpass` prompts for the key at runtime and avoids storing it in the cell output or notebook JSON.

```suggestion
import getpass, os
os.environ.setdefault("OPENAI_API_KEY", getpass.getpass("OpenAI API key: "))
MODEL = "openai/gpt-4o-mini"
```

_{Reviews (1): Last reviewed commit: "docs(tutorials): sync notebook with .md ..." | Re-trigger Greptile}

[burtenshaw]

Approved per maintainer merge request after required checks passed.

[burtenshaw]

Approved per maintainer merge request after required checks passed.