feat: consolidate shared harness code in harnesses.utils - #2482
Conversation
The building blocks that standalone chat programs share were spread over verifiers.v1.mcp.client, harnesses/standalone.py, and a monolithic minimal/program.py. Gather them in verifiers.v1.harnesses.utils: - mcp.py: the bundleable MCP client helpers (from mcp/client.py) - compaction.py: tool-output bounding and the Compactor (from minimal/program.py) - core.py: the chat call, local tools, interception hook, and chat loop (from minimal/program.py) - launch.py: launch_chat_program and bundle_program, which generalizes inline_mcp_client to splice any utils modules into a PEP 723 program (from standalone.py) minimal/program.py shrinks to argument parsing and assembly. The verifiers.v1.mcp package keeps the host-side server infrastructure and no longer re-exports the client helpers (re-exporting would import the whole harnesses package during mcp's own import). Drops a duplicated CONTEXT_OVERFLOW_MARKERS tuple left by a merge, and narrows the .gitignore core-dump pattern to core.[0-9]* so it stops swallowing core.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mikasenghaas
changed the title
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This refactor changes the shared harness execution bundle and removes the existing You can add or adjust custom eligibility rules. Learn more. |
core.py now carries the argument parsing, assembly, and __main__ entry point - inert on package import, the entry point once the module ends the bundled script. The program text is reduced to the PEP 723 metadata block, built as CHAT_PROGRAM_SOURCE in launch.py, and the harnesses.minimal package is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
|
||
| Reply with the summary as plain text. Do not call any tools - summarize from the conversation as it stands.""" | ||
|
|
||
| POST_COMPACTION_FRAMING = """Another language model started to solve this problem and produced \ |
There was a problem hiding this comment.
not sure I like this. is this based on anything?
There was a problem hiding this comment.
think codex, let me double chec
There was a problem hiding this comment.
Yes - it's Codex's compaction template, taken verbatim. The framing here is https://github.com/openai/codex/blob/c4350b4ca2a5/codex-rs/prompts/templates/compact/summary_prefix.md#L1 (word-for-word identical), and CHECKPOINT_COMPACTION_PROMPT above is https://github.com/openai/codex/blob/c4350b4ca2a5/codex-rs/prompts/templates/compact/prompt.md#L1-L9 plus our trailing no-tool-call line.
| raise | ||
| return completion, messages | ||
|
|
||
| async def compact(self, messages: list[dict]) -> list[dict]: |
There was a problem hiding this comment.
so if there is an overflow during compaction, it just tries again COMPACTION_ATTEMPTS many times, yes?
wonder if we should progressively truncate the middle each time there is a failed compaction attempt
There was a problem hiding this comment.
yes, this is what it currently does. can decide this in diff pr - this one just moves file
2 participants
Summary
Follow-up to #2454: the building blocks that standalone chat programs share were spread over
verifiers.v1.mcp.client,harnesses/standalone.py, and a monolithicminimal/program.py. This PR gathers them in a newverifiers.v1.harnesses.utilspackage and removesharnesses.minimalentirely:mcp.py— the bundleable MCP client helpers (mcp_client,connect_mcp,call_mcp,mcp_content_to_chat_content,with_retry), moved fromverifiers/v1/mcp/client.py.compaction.py— tool-output bounding and context compaction (markers, thresholds,discover_threshold,truncate_tool_output,Compactor), extracted fromminimal/program.py.core.py— the chat call, local tools (bash/edit/search), the interception hook, the chat loop (run_chat_loop), plus argument parsing, assembly, and the__main__entry point (inert on package import; the entry point once the module ends the bundled script), extracted fromminimal/program.py.launch.py—launch_chat_program;bundle_program(program, *modules), which generalizesinline_mcp_clientto splice any utils modules into a PEP 723 program (moved fromstandalone.py); andCHAT_PROGRAM_SOURCE, the shared Null/Bash program built asbundle_program(<PEP 723 metadata>, mcp, compaction, core)— the program text is only the metadata block.The browser_use program keeps its own program file and bundles
mcponly. Cross-module references inside bundled code stayTYPE_CHECKING-guarded, resolved by the flat bundle at runtime — same pattern the programs already used.Housekeeping picked up along the way:
verifiers.v1.mcpkeeps the host-side server infrastructure (Toolset,serve*,SharedToolServer) and no longer re-exports the client helpers — re-exporting from the new location would import the wholeharnessespackage duringmcp's own import. The one runtime consumer (tasksets/nemo_gym/toolset.py) imports from the new home.CONTEXT_OVERFLOW_MARKERStuple that a merge left inminimal/program.py..gitignorecore-dump pattern fromcore.*tocore.[0-9]*(it swallowedharnesses/utils/core.py).run_chat_loopreturning (instead of the loop's old earlyreturnfrommain()) means the interceptiontool_clientis now closed on the null overflow path too.No behavior changes otherwise: the bundled program is the same logic, reordered.
Verification
CHAT_PROGRAM_SOURCEand the browser_use bundle compile.uv run --no-project bundled.py --helpexecutes all spliced module-level code against the declared script deps and reaches the__main__entry point).verifiers.v1andtasksets.nemo_gym.toolsetimport cleanly; ruff + unit tests pass.Note
Consolidate shared harness code into
harnesses.utilspackageverifiers/v1/harnesses/utilspackage, deleting the oldharnesses/minimalpackagebundle_programin launch.py to replaceinline_mcp_client; builds a sharedCHAT_PROGRAM_SOURCEconstant used by the Bash, Null, and Browser-use harnessesrun_chat_loopfrommainin core.py and moves all compaction logic into compaction.pyutilsmodules instead ofminimalandstandaloneverifiers/v1/mcp.__init__no longer re-exports client helpers (call_mcp,connect_mcp,mcp_client,mcp_content_to_chat_content,with_retry); any imports of those names fromverifiers.v1.mcpwill fail at runtime — update imports toverifiers.v1.harnesses.utils.mcpMacroscope summarized fa2e316.