fix: gate task-tool tasksets on harness support (SUPPORTS_TASK_TOOLS)

[mikasenghaas]

Summary

• A harness without an MCP client (e.g. rlm) silently ignored a task's tool servers instead of refusing the run.
• Add a SUPPORTS_TASK_TOOLS ClassVar on Harness (default True). RLMHarness sets False (rlm drives its own tools, no MCP client); DefaultHarness and CompactingHarness support task tools (compact wires mcp_urls into its bundled MCP client).
• Environment.__init__ now raises an informative ValueError when a taskset that declares tools (overrides Taskset.tools) is paired with a harness whose SUPPORTS_TASK_TOOLS is False — so the incompatible combo fails fast at env construction instead of dropping the tools mid-run.

Verification

Before: uv run eval wikispeedia-v1 -n 1 --harness.id rlm ran with the task's MCP tools silently dropped.

After: it fails fast with

Harness 'rlm' does not support task tools, but taskset 'wikispeedia-v1' exposes tool servers (MCP). Run it with a harness that supports task tools (e.g. --harness.id default), or use a taskset without tools.

Note

Gate task-tool tasksets on harness SUPPORTS_TASK_TOOLS flag

• Adds SUPPORTS_TASK_TOOLS: ClassVar[bool] = True to the base Harness class in harness.py, defaulting all harnesses to supporting task tools.
• Sets SUPPORTS_TASK_TOOLS = False on RLMHarness and SUPPORTS_TASK_TOOLS = True on CompactingHarness to declare their capabilities explicitly.
• Adds a validation in Environment.__init__ (env.py) that raises ValueError when a harness with SUPPORTS_TASK_TOOLS = False is paired with a taskset that overrides Taskset.tools.
• Behavioral Change: RLMHarness now rejects tasksets that expose MCP tool servers at environment initialization time rather than failing silently at runtime.

^{Macroscope summarized c7f60a2.}

---

Note

Low Risk
Configuration-time validation only; incompatible harness/taskset pairs that previously ran without tools now raise at env init.

Overview
Introduces SUPPORTS_TASK_TOOLS on the base Harness (default True) so each harness declares whether it can wire a task’s MCP tool servers into the model. RLMHarness sets it to False (no MCP client; rlm uses its own tools). CompactingHarness sets True explicitly.

Environment now fails at construction with a clear ValueError when a harness with SUPPORTS_TASK_TOOLS=False is paired with a taskset that overrides Taskset.tools (i.e. exposes MCP servers). Incompatible combos like rlm + a tool-bearing taskset no longer run with tools silently dropped.

^{Reviewed by Cursor Bugbot for commit c7f60a2. Bugbot is set up for automated code reviews on this repo. Configure here.}