Skip to content

fix(agent-loop): recover disabled capability calls - #6840

Merged
serrrfirat merged 2 commits into
mainfrom
codex/recover-disabled-capability-call
Jul 29, 2026
Merged

serrrfirat merged 2 commits into
mainfrom
codex/recover-disabled-capability-call

Conversation

@serrrfirat

@serrrfirat serrrfirat commented Jul 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Give outside_capability_surface model-output failures an immediate typed recovery observation instead of spending generic blind retries first.
  • Invert the disabled spawn_subagent integration pin so the run completes after a corrective model turn while proving zero dispatch and zero successful capability results.

Change Type

  • Bug fix
  • New feature
  • Refactor
  • Documentation
  • CI/Infrastructure
  • Security
  • Dependencies

Linked Issue

Closes #5583
Related #6284

Validation

  • cargo fmt --all -- --check
  • cargo clippy --all --benches --tests --examples --all-features -- -D warnings — Not run: targeted clippy was selected for the changed crate and integration binary.
  • cargo build — Not run separately: both test suites compiled the affected production and integration paths.
  • Relevant tests pass: ironclaw_agent_loop and reborn_integration_tool_call
  • cargo test --features integration if database-backed or integration behavior changed — Not applicable: no database or feature-gated backend behavior changed.
  • Manual testing — Not applicable: behavior is covered through the hermetic whole-turn Reborn harness.
  • If a coding agent was used and supports it, review-pr or pr-shepherd --fix was run before requesting review — Equivalent multi-agent review completed across eight lenses; both findings were fixed and revalidated.

Test Strategy

User behavior: A hallucinated call to a disabled capability receives a precise model-visible observation, gets one corrective model turn, and completes without reporting or executing the rejected call.

Risk areas:

  • Model behavior
  • Browser
  • Side effect
  • Persistence
  • Security or permissions
  • External provider
  • Cross-component behavior

Tests added or updated:

  • Unit or contract: Added recovery-policy coverage proving OutsideCapabilitySurface observes immediately and aborts if repeated.
  • Reborn integration: Inverted the existing tool_call.rs scenario through the real product, runner, loop, gateway, and capability-surface chain.
  • Recorded fixture: Not applicable: model tool choice is scripted; the host recovery contract is under test.
  • Browser E2E: Not applicable: no browser-visible surface changed.
  • Backend or runtime: Not applicable: no database, WASM, Docker, or external runtime behavior changed.
  • Live canary: Not applicable: deterministic host recovery behavior does not require provider drift coverage.

What the tests prove:

  • The second model request contains reason=outside_capability_surface.
  • The repaired assistant reply is finalized and the run survives.
  • The disabled capability is neither dispatched nor recorded as a successful result.
  • Other invalid-output reasons retain their existing bounded retry policy.

Commands run:

cargo fmt --all -- --check
cargo test -p ironclaw_agent_loop
cargo test -p ironclaw_reborn_integration_tests --test reborn_integration_tool_call
cargo clippy -p ironclaw_agent_loop --all-targets -- -D warnings
cargo clippy -p ironclaw_reborn_integration_tests --test reborn_integration_tool_call -- -D warnings
git diff --check

Security Impact

Capability-surface enforcement is unchanged and still rejects the disabled call before registration or dispatch. This changes only the model recovery policy after that rejection, using a typed host-authored observation with no provider-supplied detail.

Reborn Trust-Boundary Checklist

  • Public policy/evidence/trust-bearing types: Not applicable; no public types or constructors changed.
  • Untrusted content enters prompts only through an envelope/escaping primitive: the new path reuses the existing typed ModelErrorRecoveryObservation control message.
  • Hashes declare purpose: Not applicable; no hashes changed.
  • New/changed status, exit, policy, runtime, or error variants: Not applicable; no variants changed. Sibling search: rg -n "OutsideCapabilitySurface|outside_capability_surface" crates tests/integration.
  • Security/durability serde(default) fields fail closed or have migration tests: Not applicable; no serialized fields changed.
  • Queues/maps/buffers/counters have bounds and overflow-safe arithmetic: the existing single observation-attempt budget remains the bound.
  • Driver/operator-visible errors have stable class semantics: InvalidOutput(OutsideCapabilitySurface) remains unchanged.
  • Sandbox/native/host names accurately describe trust boundary: Not applicable; no sandbox or host naming changed.

Database Impact

None.

Blast Radius

Limited to agent-loop recovery for the already typed OutsideCapabilitySurface invalid-output reason and its existing whole-turn integration scenario. Gateway filtering, capability registration, dispatch, result persistence, and other invalid-output recovery classes are unchanged.

Rollback Plan

Revert this PR (commits 8ad550fcb and 1a91968a2). This restores the prior generic invalid-output retry policy and the old terminal integration pin without schema or compatibility work.

Review Follow-Through

Reviewer judgment requested on the deliberate one-attempt policy: a repeated outside-surface violation remains terminal after the model has received the precise correction once. No known follow-up is required in this scope.


Review track: C (runtime error-recovery behavior)

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6840 July 29, 2026 08:15 Destroyed
@github-actions github-actions Bot added size: M 50-199 changed lines risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Jul 29, 2026
@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Disabled-capability model calls now produce a model-visible OutsideCapabilitySurface observation, allowing a follow-up response without dispatching the rejected capability. Repeated identical errors abort as InvalidModelOutput, with unit and integration coverage.

Changes

Capability rejection recovery

Layer / File(s) Summary
Immediate observation and repeat abort
crates/ironclaw_agent_loop/src/strategies/recovery.rs
OutsideCapabilitySurface bypasses generic repair retry, emits one observation without an alteration, and aborts on repetition.
Disabled capability integration flow
tests/integration/tool_call.rs
The disabled builtin.spawn_subagent test verifies model-visible recovery, continued response, no invocation, and zero successful capability results.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Model
  participant Gateway
  participant DefaultRecoveryStrategy
  Model->>Gateway: invoke disabled builtin.spawn_subagent
  Gateway->>DefaultRecoveryStrategy: outside_capability_surface invalid output
  DefaultRecoveryStrategy-->>Model: model-visible error observation
  Model->>Gateway: repaired follow-up response
Loading

Possibly related issues

Possibly related PRs

Suggested reviewers: henrypark133

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed PR #5583 is satisfied: disabled capability calls now get a model-visible denial, can self-correct, and still stop after one repeated violation.
Out of Scope Changes check ✅ Passed The diff stays within recovery policy and the pinned integration test, with no unrelated subsystems or public surface changes.
Title check ✅ Passed Conventional-commits style title accurately summarizes the agent-loop recovery change for disabled capability calls.
Description check ✅ Passed Mostly matches the template: summary, change type, linked issue, validation, test strategy, and impact sections are present.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ironloopai

ironloopai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

🔎 Review · PR #6840

🔴 Failed

GitHub request failed

IronLoop could not complete a required GitHub request.

Automatic · PR opened · attempt 1 of 3 · failed after 2m 9s

Failure details
  • Repository: nearai/ironclaw
  • Base: main at a0e91d1
  • Head: codex/recover-disabled-capability-call at 8ad550f
  • Created: Jul 29, 2026, 8:20 AM UTC
  • Updated: Jul 29, 2026, 8:22 AM UTC
  • Run: 858d275f-744f-4be8-b56c-1e5bebee51b9
  • Latest attempt: 1 · Completed · d3fe70c3-8720-434c-a17e-b9bf92bca03d
  • Failed during: GitHub writeback
  • Retryable: No
  • Failure: 50e0bb05-6cba-486f-89c4-5bb1108054ea

@github-actions

github-actions Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Coverage ratchet

Ratchet mode: ENFORCING

RATCHET PASS: global
  observed: 85.73% (313881 / 366127 lines)
  floor:    80.81% (tolerance 0.5pp -> effective floor 80.31%)
  denominator: 366127 lines now vs 377084 at floor capture (-10957 lines, -2.91%) — not a material change

⚠️ 2 Reborn crate(s) have 0 int-tier coverage (target: 0) — ironclaw_prompt_envelope, ironclaw_scripts

Reborn integration-tier coverage

Line coverage (Reborn crates): 85.73% — 313881 / 366127 lines

Per-crate breakdown (60 crates, lowest-covered first)
Crate Line % Covered / Total
ironclaw_prompt_envelope 0% 0 / 88
ironclaw_scripts 0% 0 / 345
ironclaw_process_sandbox 33.91% 118 / 348
ironclaw_host_ingress 42.5% 17 / 40
ironclaw_event_projections 43.71% 684 / 1565
ironclaw_observability 61.54% 16 / 26
ironclaw_authorization 62.98% 609 / 967
ironclaw_memory 64.41% 959 / 1489
ironclaw_telegram_v2_adapter 69.69% 731 / 1049
ironclaw_trust 73.21% 664 / 907
ironclaw_wasm_limiter 74.6% 47 / 63
ironclaw_filesystem 74.64% 4829 / 6470
ironclaw_extractors 74.72% 538 / 720
ironclaw_capabilities 75.41% 2879 / 3818
ironclaw_projects 76.48% 400 / 523
ironclaw_mcp 76.6% 779 / 1017
ironclaw_reborn_cli 78.36% 10782 / 13760
ironclaw_wasm 79.72% 735 / 922
ironclaw_llm 80.04% 22094 / 27603
ironclaw_memory_native 81.02% 3299 / 4072
ironclaw_auth 81.88% 6679 / 8157
ironclaw_first_party_extensions 82.38% 6682 / 8111
ironclaw_events 82.56% 1600 / 1938
ironclaw_processes 83.3% 933 / 1120
ironclaw_host_api 83.67% 9698 / 11591
ironclaw_reborn_identity 83.8% 450 / 537
ironclaw_operator 84.41% 5561 / 6588
ironclaw_telegram_extension 84.54% 1230 / 1455
ironclaw_secrets 84.56% 2798 / 3309
ironclaw_reborn_config 85.23% 2101 / 2465
ironclaw_skills 85.27% 4493 / 5269
ironclaw_extension_host 85.34% 19110 / 22393
ironclaw_run_state 85.77% 458 / 534
ironclaw_reborn_composition 85.81% 24634 / 28709
ironclaw_triggers 85.92% 2783 / 3239
ironclaw_network 85.97% 913 / 1062
ironclaw_reborn_event_store 86.17% 1246 / 1446
ironclaw_webui 86.37% 11171 / 12934
ironclaw_hooks 86.72% 9949 / 11472
ironclaw_common 86.99% 1772 / 2037
ironclaw_approvals 87.07% 1542 / 1771
ironclaw_extensions 87.24% 4798 / 5500
ironclaw_threads 87.36% 4912 / 5623
ironclaw_product 87.42% 20309 / 23232
ironclaw_reborn_traces 88.13% 11987 / 13601
ironclaw_turns 88.53% 14468 / 16342
ironclaw_slack_extension 88.53% 1969 / 2224
ironclaw_host_runtime 88.93% 19947 / 22429
ironclaw_reborn_openai_compat 89.32% 3780 / 4232
ironclaw_conversations 90.01% 3164 / 3515
ironclaw_resources 90.84% 4474 / 4925
ironclaw_event_streams 91.24% 1063 / 1165
ironclaw_runner 91.34% 17354 / 19000
ironclaw_loop_host 91.9% 16572 / 18032
ironclaw_attachments 93.06% 630 / 677
ironclaw_outbound 93.91% 4101 / 4367
ironclaw_agent_loop 94.57% 10026 / 10602
ironclaw_safety 95.28% 3858 / 4049
ironclaw_first_party_extension_ports 95.62% 3672 / 3840
ironclaw_runtime_policy 96.56% 814 / 843

This table itself is informational and never gates the PR on its own — not the percentage, not the per-crate holes, not the 0-coverage callout. A separate coverage ratchet (dry-run until enforce=true; see tests/integration/coverage-floor.toml) can fail the build on specific configured floors.

Exemptions (3 entry/entries excluded from the accounting above)
Module / Crate Reason Issue
crate: ironclaw_embeddings v1-only: consumed only by root ironclaw (src/app.rs, src/tools/builtin/memory.rs, src/workspace/mod.rs, src/config/{mod,embeddings}.rs); no crates/* dependents. Covered by "Tests (Legacy)". #5657
crate: ironclaw_gateway v1-only: consumed only by root ironclaw (src/channels/web/platform/static_files.rs, src/channels/web/handlers/frontend.rs); no crates/* dependents. Covered by "Tests (Legacy)". #5657
crate: ironclaw_tui v1-only: consumed only by root ironclaw (src/main.rs, src/channels/tui.rs); no crates/* dependents. Crate's own doc comment confirms it bridges INTO v1, not Reborn. Covered by "Tests (Legacy)". #5657

@railway-app

railway-app Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-6840 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Jul 29, 2026 at 11:05 am

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6840 July 29, 2026 10:54 Destroyed
@serrrfirat
serrrfirat merged commit 6da2243 into main Jul 29, 2026
62 checks passed
@serrrfirat
serrrfirat deleted the codex/recover-disabled-capability-call branch July 29, 2026 11:46
@coderabbitai coderabbitai Bot mentioned this pull request Jul 29, 2026
18 of 29 tasks
l3ocifer pushed a commit to l3ocifer/frick-ironclaw that referenced this pull request Sep 3, 2026
* fix(agent-loop): recover disabled capability calls

* test(agent-loop): address recovery review

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-6840 — 1a91968a Deployed Jul 29, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules size: M 50-199 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reborn: hallucinated call to a disabled capability fails the run as model_error instead of a model-visible denial

1 participant