Skip to content

fix(loop): make routine delivery steering deterministic under progressive disclosure - #7390

Merged
serrrfirat merged 1 commit into
mainfrom
fix/deterministic-routine-delivery-steering
Aug 8, 2026
Merged

serrrfirat merged 1 commit into
mainfrom
fix/deterministic-routine-delivery-steering

Conversation

@BenKurrek

Copy link
Copy Markdown
Collaborator

Summary

  • A live web-created routine ("look at my open issues… and send me a Slack message summarizing them") stored a prompt instructing the fire to use the vendor send-message tool to reach the requester; an identical retry pinned the correct builtin__outbound_deliver step. This PR removes both sources of that nondeterminism.
  • Disclosure tier: builtin.outbound_deliver + builtin.outbound_delivery_targets_list were Discoverable, so on catalogs past the defer threshold the bridged surface (default since feat(reborn): enable progressive tool disclosure by default #6958) drops them from visible_capabilities — and the delivery.md guidance block (which contains exactly the missing rule: "'Send it to me' is bot delivery via builtin__outbound_deliver, not an act-as-user send") renders only while both are visible (delivery_tools_visible, loop driver host). trigger_create is Core, so a creation turn could write routines while blind on the delivery lane; whether steering existed depended on whether an earlier tool_search happened to disclose the pair. Both tools are now Core-tier, the same reasoning the trigger lifecycle already uses.
  • Guidance precedence: TRIGGER_CREATE_DESCRIPTION and the trigger_create prompt-field schema said "never call builtin__outbound_deliver in a web-app-created routine". Correct for the unnamed-destination default, but over-applied — the qa_8d canary creation reasoned verbatim "I'm in the web app, so there's no outbound delivery target to pin — let me use the Slack extension's tools for the send step" before recovering. Both texts now scope the no-delivery default to "no external destination named" and state the named-destination rule explicitly (external-surface messages to the requester go through builtin__outbound_deliver, never integration messaging tools).
  • The wide-catalog schema-token benchmark (test(disclosure): pin wide-catalog schema-token reduction floor and make drift visible #7372) is unchanged at 82.9% — its synthetic 91-tool fixture contains no outbound tools, so the pinned baseline needs no move. Real deployments advertise two more small schemas (outbound_deliver: content+target_id; targets_list: no args).

Change Type

  • Bug fix

Linked Issue

Companion to #7389 (live-canary two-lane delivery contract), which documents the live incident evidence. The same root causes explain the qa_8d marker-in-final-answer-only delivery that PR fixes on the canary side.

Validation

  • cargo fmt -p ironclaw_loop_host -p ironclaw_host_runtime -- --check
  • cargo clippy -p ironclaw_loop_host -p ironclaw_host_runtime --all-targets --all-features -- -D warnings
  • cargo test -p ironclaw_loop_host --lib — 630 passed (core-name census, disclosure caps fit, token-reduction benchmark unchanged)
  • cargo test -p ironclaw_host_runtime --lib first_party_tools — 125 passed (new description/schema pins)
  • cargo test -p ironclaw_architecture_tests — full suite green, including the extension-specificity gate (first draft named "slack" in generic kernel text; the gate caught it and the wording is now extension-neutral)

Test Strategy

User behavior: routine creation turns deterministically receive delivery steering and pin builtin__outbound_deliver steps for named external destinations; fire-time runs can always call the delivery pair directly.

Risk areas:

  • Model behavior
  • Cross-component behavior

Tests added or updated:

  • Unit or contract: core_builtin_names_are_backed_by_known_capability_ids now pins both outbound tools with their capability ids (exhaustive census — a name missing from CORE_TOOL_NAMES fails); trigger_create_description_teaches_prompt_owned_delivery_with_no_stored_target gains the scoped-clause/named-destination assertions; new trigger_create_prompt_description_scopes_web_app_no_delivery_to_unnamed_destinations pins the schema sibling.
  • Reborn integration: Not applicable — production_default_defers_wide_catalog_to_bridge_meta_tools (tests/integration/tool_disclosure.rs) pins bridge membership and vendor-tool deferral, which this change does not alter; CI runs it.
  • Recorded fixture / Browser E2E / Backend or runtime: Not applicable — no wire, persistence, or transport change.
  • Live canary: the reborn-webui-v2-live-qa delivery cases (qa_3d/8d/9b/9d, no-retry) exercise creation-time pinning end-to-end every 3 hours; fix(live-qa): verify triggered Slack delivery through the two-lane contract #7389 additionally hard-fails a delivered message missing its marker.

What the tests prove: the delivery pair cannot silently drop out of the Core tier; the categorical never-clause cannot come back in either guidance home; the named-destination steering exists in both.

Commands run: the five above.

Security Impact

None. Tier changes affect only which already-authorized tools are advertised versus deferred — authorization, gating (outbound_deliver remains Allow + standing-grant as reviewed in #7157), and the capability surface policy are untouched. Guidance text changes steer the model toward the bot-identity delivery lane and away from act-as-user vendor sends for self-directed messages — a strictly safer default; the vendor lane itself remains available by design (spec decision 2, #7157).

Database Impact

None.

Blast Radius

ironclaw_loop_host tool-disclosure Core tier (advertised-token cost +2 small schemas on wide catalogs); ironclaw_host_runtime trigger_create description + input schema text. Worst case: slightly larger advertised surface; steering text regressions are pinned by tests.

Rollback Plan

Single commit; git revert restores Discoverable tier and the previous wording. No persisted state involved.

Review Follow-Through

  • The disclosure benchmark's synthetic fixture contains no outbound tools, so the 82.9% pin did not move; if reviewers want the fixture to model the delivery pair, that is a deliberate fixture change with a baseline re-pin (per the constant's own instructions), best done separately.
  • The deeper "hard deny vendor self-sends" product setting remains the owner-tabled follow-up from feat: explicit channel delivery tool — two lanes, notification channels, delivery heuristics deleted #7157 decision 2; this PR only makes the existing steering deterministic.

Review track: B

🤖 Generated with Claude Code

A web-created routine asking for GitHub-issue summaries "in a Slack
message" was created with a stored prompt instructing the fire to use
the vendor send-message tool instead of a pinned
builtin__outbound_deliver step; an identical retry produced the correct
pinned step. Two compounding causes, both observed live:

1. builtin.outbound_deliver and builtin.outbound_delivery_targets_list
   were Discoverable-tier, so on a catalog past the defer threshold the
   bridged disclosure surface (default since #6958) drops them from
   visible_capabilities — and the delivery guidance block renders only
   while both are visible (delivery_tools_visible). trigger_create is
   Core, so the model could create routines while blind on the delivery
   lane and without the "'Send it to me' is bot delivery via
   builtin__outbound_deliver" steering. Whether the steering existed
   depended on whether an earlier tool_search happened to disclose the
   pair. Both tools are now Core, restoring #7157's guidance-iff-tools
   coupling as a deterministic fact. The wide-catalog reduction
   benchmark is unchanged (82.9%): its synthetic fixture carries no
   outbound tools.

2. The trigger_create description and its prompt-field schema said
   "never call builtin__outbound_deliver in a web-app-created routine".
   The clause is correct for the no-named-destination default, but
   creation turns over-apply it — the qa_8d canary creation verbatim
   reasoned "I'm in the web app, so there's no outbound delivery target
   to pin — let me use the Slack extension's tools for the send step"
   before recovering. Both texts now scope the no-delivery default to
   "no external destination named" and state the named-destination rule
   explicitly: reaching the user or anyone else on an external surface
   goes through builtin__outbound_deliver with a pinned target id,
   never through integration messaging tools (concrete extension names
   kept out per the specificity gate).

Regression tests: the core-name census pins both tools with their
capability ids; the description tests pin the scoped clause, the
absence of the categorical never-clause, and the named-destination
steering on both the tool description and the prompt-field schema.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@railway-app

railway-app Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-7390 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Aug 8, 2026 at 4:30 am

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-7390 August 8, 2026 04:22 Destroyed
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Improved trigger delivery guidance for named external destinations and web-app run threads.
    • External destinations now use pinned outbound delivery targets.
    • Added consistent availability of outbound delivery and destination-listing tools.
  • Bug Fixes

    • Prevented integration messaging tools from being used for requester delivery.
    • Clarified behavior when no external destination is specified.

Walkthrough

Trigger guidance now routes named external destinations through pinned builtin__outbound_deliver steps. Unnamed web-app triggers reply through their run thread. Outbound delivery tools are always included in the core tool catalog.

Changes

Trigger delivery

Layer / File(s) Summary
Core outbound tool catalog
crates/loop/ironclaw_loop_host/src/tool_disclosure.rs
The core catalog now always advertises outbound_deliver and outbound_delivery_targets_list, with regression coverage for their builtin capability IDs.
Trigger delivery guidance
crates/kernel/ironclaw_host_runtime/src/first_party_tools/schemas.rs, crates/kernel/ironclaw_host_runtime/src/first_party_tools/trigger_management.rs, crates/kernel/ironclaw_host_runtime/src/first_party_tools/trigger_management/tests.rs
Trigger descriptions and tests distinguish unnamed web-app replies from named external destinations, require pinned outbound delivery for external destinations, and reject integration messaging tools.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: serrrfirat

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is detailed but omits the required Reborn Trust-Boundary Checklist for this runtime change. Add the Reborn Trust-Boundary Checklist and complete each item, or state N/A with a reason where applicable.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title uses Conventional Commits style and accurately summarizes the deterministic routine-delivery change.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size: M 50-199 changed lines risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Aug 8, 2026
@ironloopai

ironloopai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

🧭 IronLoop Run · Review

This comment updates in place as the Run moves through its stages.

🟩 Final result · Completed

🟨 Queued → 🟦 Working → 🟦 Posting results → 🟩 Completed

Automatic trigger · attempt 1 of 3 · completed in 11m 33s

IronLoop completed the review and posted it to GitHub.

🔗 Result

Open submitted review →

Run details

Run: f570b939-bef6-4032-8884-468094f1eab2
Base: main at cae1a04
Head: fix/deterministic-routine-delivery-steering at 7655512
Created: 2026-08-08 04:23 UTC
Updated: 2026-08-08 04:34 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@crates/kernel/ironclaw_host_runtime/src/first_party_tools/trigger_management.rs`:
- Line 43: Create one crate-owned multiline delivery-guidance prompt fragment
under prompts/ and load it with include_str!(). In
crates/kernel/ironclaw_host_runtime/src/first_party_tools/trigger_management.rs:43-43,
replace the inline delivery-policy section in TRIGGER_CREATE_DESCRIPTION with
that fragment; in
crates/kernel/ironclaw_host_runtime/src/first_party_tools/schemas.rs:910-910,
load the same fragment and retain only the schema-specific surrounding text so
both locations share identical guidance.

In `@crates/loop/ironclaw_loop_host/src/tool_disclosure.rs`:
- Around line 58-67: Update representative_tool_fixture() to include fixtures
for outbound_deliver and outbound_delivery_targets_list, ensuring the
wide-catalog benchmark measures both newly Core-disclosed tools. Run the
benchmark and replace the recorded baseline and history values with the measured
result.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3a5540fd-fa0d-4d85-9436-c355843b445e

📥 Commits

Reviewing files that changed from the base of the PR and between cae1a04 and 7655512.

📒 Files selected for processing (4)
  • crates/kernel/ironclaw_host_runtime/src/first_party_tools/schemas.rs
  • crates/kernel/ironclaw_host_runtime/src/first_party_tools/trigger_management.rs
  • crates/kernel/ironclaw_host_runtime/src/first_party_tools/trigger_management/tests.rs
  • crates/loop/ironclaw_loop_host/src/tool_disclosure.rs

Comment thread crates/loop/ironclaw_loop_host/src/tool_disclosure.rs

@ironloopai ironloopai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 IronLoop review

🟢 No actionable findings

No actionable findings in the four-file normal PR diff.

Validation

  • ✅ Formatting — cargo fmt -p ironclaw_loop_host -p ironclaw_host_runtime -- --check
  • ✅ Loop-host tests — cargo test -p ironclaw_loop_host --lib — 630 passed.
  • ✅ Host-runtime tests — cargo test -p ironclaw_host_runtime --lib first_party_tools — 125 passed.
  • ✅ Clippy — cargo clippy -p ironclaw_loop_host -p ironclaw_host_runtime --all-targets --all-features -- -D warnings
  • ✅ Diff integrity — git diff --check refs/ironloop/merge-base refs/ironloop/head passed.
Review details
  • Run: f570b939-bef6-4032-8884-468094f1eab2
  • Workflow: Review
  • Attempts: 1

@serrrfirat
serrrfirat added this pull request to the merge queue Aug 8, 2026
Merged via the queue into main with commit 9f71cbb Aug 8, 2026
46 checks passed
@serrrfirat
serrrfirat deleted the fix/deterministic-routine-delivery-steering branch August 8, 2026 07:13
Kampouse pushed a commit to Kampouse/ironclaw that referenced this pull request Aug 13, 2026
…ure (nearai#7390)

A web-created routine asking for GitHub-issue summaries "in a Slack
message" was created with a stored prompt instructing the fire to use
the vendor send-message tool instead of a pinned
builtin__outbound_deliver step; an identical retry produced the correct
pinned step. Two compounding causes, both observed live:

1. builtin.outbound_deliver and builtin.outbound_delivery_targets_list
   were Discoverable-tier, so on a catalog past the defer threshold the
   bridged disclosure surface (default since nearai#6958) drops them from
   visible_capabilities — and the delivery guidance block renders only
   while both are visible (delivery_tools_visible). trigger_create is
   Core, so the model could create routines while blind on the delivery
   lane and without the "'Send it to me' is bot delivery via
   builtin__outbound_deliver" steering. Whether the steering existed
   depended on whether an earlier tool_search happened to disclose the
   pair. Both tools are now Core, restoring nearai#7157's guidance-iff-tools
   coupling as a deterministic fact. The wide-catalog reduction
   benchmark is unchanged (82.9%): its synthetic fixture carries no
   outbound tools.

2. The trigger_create description and its prompt-field schema said
   "never call builtin__outbound_deliver in a web-app-created routine".
   The clause is correct for the no-named-destination default, but
   creation turns over-apply it — the qa_8d canary creation verbatim
   reasoned "I'm in the web app, so there's no outbound delivery target
   to pin — let me use the Slack extension's tools for the send step"
   before recovering. Both texts now scope the no-delivery default to
   "no external destination named" and state the named-destination rule
   explicitly: reaching the user or anyone else on an external surface
   goes through builtin__outbound_deliver with a pinned target id,
   never through integration messaging tools (concrete extension names
   kept out per the specificity gate).

Regression tests: the core-name census pins both tools with their
capability ids; the description tests pin the scoped clause, the
absence of the categorical never-clause, and the named-destination
steering on both the tool description and the prompt-field schema.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Kampouse pushed a commit to Kampouse/ironclaw that referenced this pull request Aug 13, 2026
…ure (nearai#7390)

A web-created routine asking for GitHub-issue summaries "in a Slack
message" was created with a stored prompt instructing the fire to use
the vendor send-message tool instead of a pinned
builtin__outbound_deliver step; an identical retry produced the correct
pinned step. Two compounding causes, both observed live:

1. builtin.outbound_deliver and builtin.outbound_delivery_targets_list
   were Discoverable-tier, so on a catalog past the defer threshold the
   bridged disclosure surface (default since nearai#6958) drops them from
   visible_capabilities — and the delivery guidance block renders only
   while both are visible (delivery_tools_visible). trigger_create is
   Core, so the model could create routines while blind on the delivery
   lane and without the "'Send it to me' is bot delivery via
   builtin__outbound_deliver" steering. Whether the steering existed
   depended on whether an earlier tool_search happened to disclose the
   pair. Both tools are now Core, restoring nearai#7157's guidance-iff-tools
   coupling as a deterministic fact. The wide-catalog reduction
   benchmark is unchanged (82.9%): its synthetic fixture carries no
   outbound tools.

2. The trigger_create description and its prompt-field schema said
   "never call builtin__outbound_deliver in a web-app-created routine".
   The clause is correct for the no-named-destination default, but
   creation turns over-apply it — the qa_8d canary creation verbatim
   reasoned "I'm in the web app, so there's no outbound delivery target
   to pin — let me use the Slack extension's tools for the send step"
   before recovering. Both texts now scope the no-delivery default to
   "no external destination named" and state the named-destination rule
   explicitly: reaching the user or anyone else on an external surface
   goes through builtin__outbound_deliver with a pinned target id,
   never through integration messaging tools (concrete extension names
   kept out per the specificity gate).

Regression tests: the core-name census pins both tools with their
capability ids; the description tests pin the scoped clause, the
absence of the categorical never-clause, and the named-destination
steering on both the tool description and the prompt-field schema.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
l3ocifer pushed a commit to l3ocifer/frick-ironclaw that referenced this pull request Sep 3, 2026
…ure (nearai#7390)

A web-created routine asking for GitHub-issue summaries "in a Slack
message" was created with a stored prompt instructing the fire to use
the vendor send-message tool instead of a pinned
builtin__outbound_deliver step; an identical retry produced the correct
pinned step. Two compounding causes, both observed live:

1. builtin.outbound_deliver and builtin.outbound_delivery_targets_list
   were Discoverable-tier, so on a catalog past the defer threshold the
   bridged disclosure surface (default since nearai#6958) drops them from
   visible_capabilities — and the delivery guidance block renders only
   while both are visible (delivery_tools_visible). trigger_create is
   Core, so the model could create routines while blind on the delivery
   lane and without the "'Send it to me' is bot delivery via
   builtin__outbound_deliver" steering. Whether the steering existed
   depended on whether an earlier tool_search happened to disclose the
   pair. Both tools are now Core, restoring nearai#7157's guidance-iff-tools
   coupling as a deterministic fact. The wide-catalog reduction
   benchmark is unchanged (82.9%): its synthetic fixture carries no
   outbound tools.

2. The trigger_create description and its prompt-field schema said
   "never call builtin__outbound_deliver in a web-app-created routine".
   The clause is correct for the no-named-destination default, but
   creation turns over-apply it — the qa_8d canary creation verbatim
   reasoned "I'm in the web app, so there's no outbound delivery target
   to pin — let me use the Slack extension's tools for the send step"
   before recovering. Both texts now scope the no-delivery default to
   "no external destination named" and state the named-destination rule
   explicitly: reaching the user or anyone else on an external surface
   goes through builtin__outbound_deliver with a pinned target id,
   never through integration messaging tools (concrete extension names
   kept out per the specificity gate).

Regression tests: the core-name census pins both tools with their
capability ids; the description tests pin the scoped clause, the
absence of the categorical never-clause, and the named-destination
steering on both the tool description and the prompt-field schema.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-7390 — 76555121 Deployed Aug 8, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules size: M 50-199 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants