Repository navigation
feat: add network_guidance policy to drop network-dependent prompt guidance - #240
Merged
Merged
Conversation
rasdani
marked this pull request as ready for review
November 2, 2026 21:30
rasdani
added a commit
to PrimeIntellect-ai/verifiers
that referenced
this pull request
Nov 2, 2026
…twork (#3011) Adds `RLMHarnessConfig.network_guidance` (`bool | None`), passed to nano-rlm's execution policy the same way as `allow_git`. - **When unset** (the default), the harness derives it from the rollout's effective network policy, i.e. the runtime config, which already carries the task's network rules: - a box that allows no destinations gets `network_guidance=false`, so nano-rlm stops suggesting `uv pip install` and online lookups (verifiers already appends "External network access is disabled." there); - any other box sends nothing and keeps nano-rlm's default. - **An explicit `true` or `false`** overrides the derived value. - The default nano-rlm `version` is bumped to the commit that adds the policy field (PrimeIntellect-ai/nano-rlm#240). If that PR is squash-merged, the ref should be re-pointed to the merged commit. The branch is based on `60290d4`. main is 19 commits ahead, none of them in the rlm harness, and a test merge is clean. Tests: new cases in `tests/unit/harnesses/test_rlm.py`: - `test_rlm_drops_network_guidance_for_boxes_with_no_network` - `test_rlm_leaves_network_guidance_to_nano_rlm_when_network_reachable` - `test_rlm_explicit_network_guidance_overrides_runtime_policy` 241 unit tests passed and ruff is clean. The rlm integration tests on Prime sandboxes were not run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Disable network guidance in RLM prompts on boxes with no network > - Adds `RLMHarness._network_guidance` in [harness.py](https://github.com/PrimeIntellect-ai/verifiers/pull/3011/files#diff-3955da24b5fd88f62482297bbda7ba0936f633927f15576db27fed6e58f668ec): an explicit `network_guidance` setting in `RLMHarnessConfig` takes priority; when unset, guidance turns off if the runtime config is a `NetworkPolicyConfig` with no allowed destinations, and stays unset otherwise > - Includes the resolved value in the RLM policy metadata built by `RLMHarness._runtime_metadata` > - Changes the default nano-rlm Git ref in `RLMHarnessConfig` > - Adds tests in [test_rlm.py](https://github.com/PrimeIntellect-ai/verifiers/pull/3011/files#diff-4fecc41bf5fdc3879310574181182559b4f5f61b9f556f47352ea3e79f207ab8) covering all three resolution cases > - Behavioral Change: RLM installs now pull a different default nano-rlm Git ref; check `RLMHarnessConfig` for the new default if pinning matters > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized ab28ece.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Prompt-behavior tuning for offline sandboxes plus a pinned dependency bump; no auth or data-path changes. > > **Overview** > Adds **`network_guidance`** on `RLMHarnessConfig` and wires it into the nano-rlm runtime **policy** metadata. When unset, the harness sets **`network_guidance=false`** if the rollout runtime is a `NetworkPolicyConfig` with an empty allow list (no outbound network); otherwise it omits the knob so nano-rlm keeps its default on-network prompt hints. Explicit **`true`/`false`** on the config always wins. > > Also bumps the default **nano-rlm git ref** to a commit that understands the new policy field. Unit tests cover no-network boxes, reachable-network/local runtimes, explicit overrides, and refactor stop-reason tests with shared **`_trace`** / **`_sent_policy`** helpers. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ab28ece. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…idance ExecutionPolicy.network_guidance (default true) controls the system prompt sentences that assume outbound network access: the `!uv pip install` hint in the kernel packages line and the online-solutions clause of the git history guard. Boxes with no network can turn it off so the prompt does not suggest actions that cannot work there. The default leaves the prompt unchanged. The flag is wired like allow_git through the engine, the ACP runtime contract and the session limits snapshot; sub-agents inherit it with the policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rasdani
force-pushed
the
feat/network-guidance
branch
from
November 2, 2026 22:10
15513ae to
ef195f8
Compare
hallerite
approved these changes
Nov 2, 2026
mannuch
added a commit
to mannuch/nano-rlm
that referenced
this pull request
Nov 2, 2026
Brings in the network_guidance policy (PrimeIntellect-ai#240). Conflicts were additive: the limits snapshot, engine init and test fixtures keep the fork's compaction and harness fields alongside network_guidance, and the kernel packages line keeps the fork's `--python {sys.executable}` install hint on upstream's split KERNEL_PREINSTALLED_PROMPT prefix. docs/runtime-contract.md gains a network_guidance policy row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rasdani
added a commit
to PrimeIntellect-ai/verifiers
that referenced
this pull request
Nov 2, 2026
Update the RLM harness's pinned nano-rlm commit from `15513ae8` to `48da1d9c43bdcf2ef20b7cff9e4c6df800810b30`. `15513ae8` was the head of PrimeIntellect-ai/nano-rlm#240 when #3011 pinned it. #240 has since been squash-merged as `48da1d9c`, so `15513ae8` is on no branch and will be garbage collected. `48da1d9c` has the same tree as #240's final head. [Upstream changes](PrimeIntellect-ai/nano-rlm@15513ae...48da1d9) also bring in nano-rlm main since #240's original base: - PrimeIntellect-ai/nano-rlm#218: separates persistent tool execution from the agent loop. Local mode is affected too: bash background jobs outlive the kernel, and output keeps the first and last 512 KiB. - PrimeIntellect-ai/nano-rlm#246: spent limits are reported as ACP stop reasons. - PrimeIntellect-ai/nano-rlm#244 and PrimeIntellect-ai/nano-rlm#245: tests and docs. `tests/unit/harnesses/test_rlm.py` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Default dependency bump changes runtime agent/tool behavior for harnesses using the pinned ref, though unit tests for the harness still pass. > > **Overview** > Updates the **RLM harness** default `version` git ref for **nano-rlm** from `15513ae8…` to **`48da1d9c…`**, so installs track the squash-merged commit from upstream PR #240 instead of a ref that is no longer on any branch. > > Rollouts that rely on the default pin will pull the newer nano-rlm tree, which (per upstream) includes **persistent tool execution outside the agent loop** (e.g. bash background jobs surviving the kernel with head/tail output truncation), **ACP stop reasons when spend limits hit**, and related test/doc updates. Custom `version` overrides are unchanged. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit cea2226. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Bump default nano-rlm ref in `RLMHarnessConfig` to `48da1d9c` > Updates the default `version` for nano-rlm from `15513ae8` to `48da1d9c` in [harness.py](https://github.com/PrimeIntellect-ai/verifiers/pull/3044/files#diff-3955da24b5fd88f62482297bbda7ba0936f633927f15576db27fed6e58f668ec). Applies only when no version is explicitly configured. Behavioral Change: runs without an explicit version now use nano-rlm at ref `48da1d9c43bdcf2ef20b7cff9e4c6df800810b30`. > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized cea2226.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
ExecutionPolicy.network_guidance(defaulttrue). It controls the two system-prompt sentences that assume outbound network access:!uv pip install <pkg>hint in the kernel packages line;A box with no network can set it to
false, so the prompt doesn't suggest actions that can't work there. The default leaves the prompt byte-identical.Rendered diff with
network_guidance=false:The flag is wired like
allow_git: through the engine, the ACP runtime contract and the session limits snapshot. Sub-agents inherit it with the policy.Tested: 380 passed. The one failure,
tests/test_skills.py::test_valid_bash_skill_metrics, also fails on main. ruff check and format are clean.🤖 Generated with Claude Code
Note
Low Risk
Default-on policy keeps existing behavior; changes only affect optional prompt strings and runtime metadata fields.
Overview
Adds
ExecutionPolicy.network_guidance(defaulttrue) so air-gapped or no-network runs can omit system-prompt lines that assume outbound access.When
network_guidanceisfalse,build_system_promptswaps the kernel packages blurb for a pre-installed-only line (drops!uv pip install) and uses an offline git history guard that keeps local-history rules but drops the “online solutions” clause. With the defaulttrue, composed prompts stay unchanged.The flag is threaded like
allow_git: stored on the engine, passed into prompt construction, exposed on the ACPlimitssnapshot and_LimitsSnapshot, and inherited with the rest ofExecutionPolicyfor sub-agents. Tests cover config/ACP fixtures and a dedicated prompt test for online vs offline wording.Reviewed by Cursor Bugbot for commit ef195f8. Bugbot is set up for automated code reviews on this repo. Configure here.