feat: support RLM training over ACP

[hallerite]

Summary

• express the RLM runtime as typed harness configuration and send its strict ai.prime.rlm/runtime-v1 payload through ACP session/new
• carry opaque ACP session/close metadata through the generic runner and let the RLM harness validate its final ai.prime.rlm/session-v1 snapshot into training metrics
• preserve a canonical root ACP-visible reply independently from concurrently intercepted descendant branches
• honor standard non-streaming HTTP idempotency by coalescing in-flight retries, replaying bounded completed responses, and namespacing provider keys per rollout
• run live RLM E2Es directly against merged nano-rlm main

Verifiers does not model RLM lineage, compaction, subagents, or agent-specific ACP lifecycle artifacts. Those semantics stay inside nano-rlm. Verifiers transports standard ACP metadata and records every intercepted parent and descendant model call in the trace graph.

Contract

This intentionally requires the nano-rlm contract merged in PrimeIntellect-ai/nano-rlm#134. No compatibility path or temporary commit pin remains; the harness default and CI both use nano-rlm main.

Validation

• unpinned deterministic Prime VM E2E against merged nano-rlm main (c27f8ea): 1 passed in 44.89s
• forces two recursive child requests to be in flight simultaneously
• asserts three retained trace branches: root, child one, and child two
• persisted trace: reward 1.0, no errors, five sampled turns, sub_rlm_num_calls = 2, has_sub_rlm = 1
• asserts the parent resumes after both children and the canonical reply remains the root ACP reply
• validates MCP resume, kernel environment isolation, and strict terminal training metrics in the same rollout
• exact Hermes ACP resume/Docker case passes, including its leading presentation-whitespace edge
• full local Verifiers suite and focused trace, idempotency, and contract checks pass
• pre-push Markdown, Ruff, format, and Ty checks pass
• all commits are GitHub signature-verified

Note

Add RLM training support over ACP with session metadata and close metrics

• RLM harness now passes configuration to ACP via structured session_meta from RLMHarness._runtime_metadata instead of environment variables; prepare_acp only forwards resolved env and RLM_HOME.
• ACPSession.close now returns a metadata dict from the close-session response, and serve_stream includes it in shutdown responses; ACPHarnessSession._stop records harness metrics on the trace via acp_close_metrics and raises HarnessFinalizationError on shutdown/finalization failures.
• Trace and Segment gain a primary_reply field; last_reply prefers it so child responses do not leak as the primary reply.
• Interception server adds explicit Idempotency-Key handling for non-streaming calls: coalesces concurrent attempts, replays cached responses, rejects key reuse with different body/path, and applies TTL/size eviction via _prune_idempotent_requests.
• Adds HarnessFinalizationError to the public verifier API and new tests for idempotency and the RLM Prime contract.
• Risk: RLMHarness.summarize_threshold now returns int | None instead of str; any consumer expecting a string threshold will need updating. RLMHarnessConfig adds new policy fields and validation (e.g. max_output disallows 0, max_concurrent_subagents >= max_depth) that may reject previously accepted configs.

^{Macroscope summarized 762650a.}

---

Note

High Risk
Changes the model interception path (idempotency coalescing) and RLM configuration transport; close-time metric finalization now fails rollouts when artifacts are missing.

Overview
Adds RLM training over ACP by sending typed ai.prime.rlm/runtime-v1 config through ACP session_meta instead of RLM env vars, and reading training metrics from validated session/close metadata (ai.prime.rlm/session-v1) rather than scraping meta.json on disk. RLMHarnessConfig grows policy fields (limits, kernel env, SDK retries, etc.); default nano-rlm ref is main, overridable in E2E via NANO_RLM_E2E_VERSION.

Introduces Trace.primary_reply (and Segment/last_reply behavior) so ACP harnesses can expose the canonical user-visible answer when subagent branches pollute graph leaf order; ACP prompts set it from the session reply.

ACP shutdown now returns close metadata to the host; harnesses implement acp_close_metrics. Failures to finalize required close artifacts raise HarnessFinalizationError and fail the rollout (not best-effort teardown).

The interception server honors non-streaming Idempotency-Key: coalesce in-flight duplicates, replay completed responses with TTL/size bounds, reject key/body mismatches, and rewrite upstream keys per rollout session.

New tests cover idempotency, wire primary_reply, ACP resume primary_reply, and a deterministic Prime VM RLM contract E2E.

^{Reviewed by Cursor Bugbot for commit 762650a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR adds substantial RLM-over-ACP functionality while changing shared ACP lifecycle, close-time metric finalization, trace reply semantics, and model interception/idempotency behavior. The cross-process contract and broad runtime effects exceed the scope of a small isolated change and warrant human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e4afc7b. Configure here.}