feat: publish ACP snapshots with prompt results - #144
Merged
Conversation
hallerite
marked this pull request as ready for review
August 25, 2026 17:01
mikasenghaas
approved these changes
Aug 25, 2026
hallerite
added a commit
to PrimeIntellect-ai/verifiers
that referenced
this pull request
Aug 26, 2026
## Summary - run nano-RLM through the generic typed ACP prompt lifecycle - pass its runtime-v1 execution policy and credentials at session creation - consume cumulative session metrics from successful `PromptResponse._meta`, not `session/close` - validate only the session id and metrics Verifiers consumes while allowing additive snapshot fields - pin nano-RLM to the exact producer commit from [nano-RLM PR #144](PrimeIntellect-ai/nano-rlm#144) - retain MCP, resume, recursive branches, credential isolation, skills, and cumulative metrics ## Stack PR 2 of the three-PR Verifiers stack. This targets typed ACP base PR #2438. Its exact nano-RLM dependency is `11dcb9c353f1f7c89c8c6f1bc0ddea33b3cffa19` from nano-RLM #144. ## Validation - full Verifiers suite on the combined stack — 919 passed, 77 credential-gated skips - nano-RLM full suite — 123 passed - live Prime VM contract E2E against the exact pin — passed - two concurrent recursive children - three expected training branches - MCP after resume - kernel credential isolation - prompt-boundary recursive metrics - changed-file pre-commit hooks and push-hook CI-parity type check <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Moves API keys and execution policy from env vars into ACP session metadata and changes the metrics contract; mis-validation or metadata drift could break rollouts or drop training signals. > > **Overview** > The RLM harness stops wiring nano-RLM through **`RLM_*` environment variables** and instead supplies a typed **`ai.prime.rlm/runtime-v1`** payload on **`ACPConfig.session_meta`** at session creation (model, provider credentials, policy, skills, kernel env, etc.). **`prepare_acp`** now keeps only **`resolved_env`** plus **`RLM_HOME`** in subprocess env. > > **Compaction policy** is expressed as **`int | None`** in that metadata (including per-task random thresholds) instead of string env values. > > **Rollout metrics** no longer come from a **`@metric`** helper that shells out to read **`sessions/*/meta.json`**. **`acp_turn_result`** validates **`ai.prime.rlm/session-v1`** with **`_SessionSnapshot`**, checks **`session_id`** against the trace, and calls **`trace.record_metrics`**. > > The default **nano-rlm** install pin is updated to **`5ee1c34024a183bbbd3a38a6129995f5b982631d`**. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit a15fcc6. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
session/promptresponseTest plan
uv run ruff check src/rlm/acp.py tests/test_acp.pyuv run pytest tests/(123 passed)Note
Low Risk
Additive wire metadata on the happy path only; snapshot is already validated and credential-scrubbed via existing
_session_metadata.Overview
Successful ACP
session/promptresponses now includefield_metawith the validatedai.prime.rlm/session-v1snapshot (same shape as session close), so clients can read cumulative session id, turn count, usage, andlast_stop_reasonafter each completed turn without waiting for close.new_sessionand non-success paths (e.g. cancelled prompts) still omit this metadata; close continues to publish the final snapshot.Reviewed by Cursor Bugbot for commit 11dcb9c. Bugbot is set up for automated code reviews on this repo. Configure here.