fix(reborn): discourage disabled tool workarounds

[italic-jinxin]

Summary

• Adds a model-visible capability surface policy telling Reborn loops to use only visible capabilities and not route disabled/unavailable user-requested capabilities through another tool.
• Includes the policy in the instruction bundle fingerprint so prompt materialization changes are tracked deterministically.
• Adds caller-level prompt materialization coverage for the disabled-tool workaround policy.
• Adds a Reborn WebUI v2 HTTP-level regression scenario for builtin.echo = disabled, guarding against fallback builtin.shell workaround requests.

Linked Issue

Closes #5197

Validation

• cargo fmt --package ironclaw_turns
• cargo test -p ironclaw_turns loop_prompt_port_materializes_memory_surface_and_safety_as_host_owned_refs --test agent_loop_host_contract
• cargo test -p ironclaw_turns
• git diff --check
• ./.venv/bin/python -m py_compile mock_llm.py scenarios/test_reborn_webui_v2_smoke.py
• Direct mock-LLM branch check for the issue 5197 regression trigger
• Reborn v2 E2E pytest was attempted locally, but the local fixture timed out while waiting for mock LLM /v1/models readiness before reaching the test assertion.

Security Impact

Low. This tightens model-visible capability guidance so disabled or unavailable capabilities are less likely to be bypassed through broader tools such as shell. Runtime authorization remains the final enforcement layer.

Database Impact

No schema or migration changes.

Blast Radius

Limited to Reborn instruction bundle capability-surface prompt materialization and related Reborn WebUI v2/mock E2E regression coverage.

Rollback Plan

Revert this PR to remove the extra capability-surface policy and the associated regression tests.

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added an explicit visible-capabilities usage policy to prompt construction, including deterministic prompt/fingerprint updates.
  • Suppressed provider tool execution when a user requests a capability that isn’t available on the currently visible capability surface.

• Bug Fixes

  • Improved capability descriptor rendering (including consistent formatting when no descriptors exist) and rejected blank policy text.
  • Prevented “unavailable” capability requests from being rerouted through shell/echo workarounds.

• Tests

  • Expanded unit, gateway, and end-to-end coverage, including new scenarios for disabled tool routing and updated mock LLM state resets.

Walkthrough

Adds visible capability-policy text, gateway suppression for unavailable tools, and E2E coverage for disabled echo requests.

Changes

Disabled tool workaround path

Layer / File(s)	Summary
Visible capability policy crates/ironclaw_turns/prompts/capability_surface_usage_policy.md, crates/ironclaw_turns/src/run_profile/instruction_bundle.rs, crates/ironclaw_turns/tests/agent_loop_host_contract.rs	Adds the policy text, injects it into the visible surface prompt, hashes it into the visible-surface fingerprint, and asserts the rendered prompt contains the policy block and structured capability descriptors.
Gateway unavailable capability guard crates/ironclaw_reborn/src/model_gateway.rs, crates/ironclaw_reborn/tests/llm_gateway.rs	Detects unavailable capability requests, suppresses provider tool calls, and covers the suppression and non-match cases in gateway tests.
E2E disabled tool coverage tests/e2e/mock_llm.py, tests/e2e/conftest.py, tests/e2e/scenarios/test_reborn_webui_v2_smoke.py	Adds mock trigger handling, per-test tool-permission setup and reset, and a v2 smoke test for the disabled-echo path.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

• BenKurrek
• serrrfirat

Poem

A policy bloomed in the surface light,
and tools stayed honest, visible, right.
No shell detour, no sneaky substitute,
just a clear unavailable salute.
The tests hummed soft; the timeline stood tight.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	Conventional Commits style is followed and the title matches the PR’s disabled-tool workaround fix.
Description check	✅ Passed	The description covers summary, linked issue, validation, security, blast radius, and rollback; only some template checklists are left blank.
Linked Issues check	✅ Passed	The changes address #5197 by making disabled capabilities unavailable and preventing fallback tool workarounds, with regression coverage.
Out of Scope Changes check	✅ Passed	The added policy, gateway guard, prompt materialization, and E2E tests all support the stated disabled-tool workaround objective.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/ironclaw_turns/src/run_profile/instruction_bundle.rs`:
- Around line 24-28: `CAPABILITY_SURFACE_USAGE_POLICY` is inline prompt text in
Rust and should be moved to a prompt file loaded with `include_str!` to match
the repo invariant for prompt templates in `*.rs` files. Update the
`instruction_bundle.rs` usage to read from a shared prompt asset under
`crates/ironclaw_turns/prompts/` (or the engine’s shared prompts directory),
then align the duplicated literals in `tests/mock_llm.py` and
`tests/agent_loop_host_contract.rs` to the same source text so the policy stays
synchronized across all three locations.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e7376564-a169-4778-9d22-4a080f2ad3d7

📥 Commits

Reviewing files that changed from the base of the PR and between 54ca210 and 31be313.

📒 Files selected for processing (4)

• crates/ironclaw_turns/src/run_profile/instruction_bundle.rs
• crates/ironclaw_turns/tests/agent_loop_host_contract.rs
• tests/e2e/mock_llm.py
• tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

[railway-app]

🚅 Deployed to the ironclaw-pr-5307 environment in ironclaw-ci-preview

Service	Status	Web	Updated (UTC)
ironclaw	✅ Success (View Logs)	Web	Jun 26, 2026 at 3:30 pm

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

tests/e2e/mock_llm.py (1)

884-888: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Route this case through CANNED_RESPONSES.

The new disabled-tool behavior was added as bespoke match_response / match_tool_call branches, which creates a second mock-dispatch path instead of extending the file’s canonical last-user-message matcher. Please fold this into CANNED_RESPONSES and keep the policy check as data for that canned case.

As per coding guidelines, Add new canned LLM responses to CANNED_RESPONSES in mock_llm.py by pattern-matching against the last user message; unmatched messages return a default response.

Also applies to: 1131-1138, 1333-1342

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/mock_llm.py` around lines 884 - 888, Route the new disabled-tool
behavior through CANNED_RESPONSES instead of bespoke
match_response/match_tool_call branches. Update mock_llm.py so the canonical
last-user-message matcher handles this case, with
_conversation_has_disabled_tool_workaround_policy used as the policy/data check
for that canned response. Keep the existing match dispatch unified in
CANNED_RESPONSES and let unmatched messages fall back to the default response.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/scenarios/test_reborn_webui_v2_smoke.py`:
- Around line 316-326: The fixture disabled_echo_shell_ask_policy needs safer
cleanup around the permission mutations. Move the setup calls to
_set_tool_permission for builtin.echo and builtin.shell inside the try block so
cleanup always starts even if the second mutation fails. In the finally block,
reset builtin.echo and builtin.shell independently using separate cleanup
attempts so one restore failure does not prevent the other from running.

---

Outside diff comments:
In `@tests/e2e/mock_llm.py`:
- Around line 884-888: Route the new disabled-tool behavior through
CANNED_RESPONSES instead of bespoke match_response/match_tool_call branches.
Update mock_llm.py so the canonical last-user-message matcher handles this case,
with _conversation_has_disabled_tool_workaround_policy used as the policy/data
check for that canned response. Keep the existing match dispatch unified in
CANNED_RESPONSES and let unmatched messages fall back to the default response.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6aebbeaa-93c3-46a3-ade1-9bf1afdb45ae

📥 Commits

Reviewing files that changed from the base of the PR and between 31be313 and 75e4d2b.

📒 Files selected for processing (5)

• crates/ironclaw_turns/prompts/capability_surface_usage_policy.md
• crates/ironclaw_turns/src/run_profile/instruction_bundle.rs
• crates/ironclaw_turns/tests/agent_loop_host_contract.rs
• tests/e2e/mock_llm.py
• tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/mock_llm.py`:
- Around line 113-117: The premature fallback in match_tool_call is caused by
REQUESTED_UNAVAILABLE_TOOL_TRIGGER being too broad and matching generic “use
<dotted-token> to” phrases before the specific capability handlers run. Tighten
that regex in mock_llm.py to only catch the intended issue 5197 echo workaround
path, such as requiring the exact disabled echo workaround wording or
builtin.echo-style prompts, and keep the UNAVAILABLE_CAPABILITY_POLICY_TEXT
check aligned with that narrow case. If needed, reorder the match_tool_call
branches so gmail/pikastream and other specific handlers are evaluated before
this fallback returns the builtin_shell echo "disabled-test" response.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3e66fbbf-9dae-4e55-aae4-e0a61c17e3ab

📥 Commits

Reviewing files that changed from the base of the PR and between 75e4d2b and bb46c4f.

📒 Files selected for processing (4)

• crates/ironclaw_turns/prompts/capability_surface_usage_policy.md
• crates/ironclaw_turns/tests/agent_loop_host_contract.rs
• tests/e2e/mock_llm.py
• tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/mock_llm.py`:
- Around line 113-116: The disabled-tool mock case is bypassing the standard
canned-response flow and should be moved into CANNED_RESPONSES. Add the
issue-5197 prompt match to the existing last-user-message pattern table in
mock_llm.py, then key the tool-call suppression from that matched canned
response instead of the standalone REQUESTED_UNAVAILABLE_TOOL_TRIGGER path. Keep
the existing match_response/match_tool_call behavior aligned with the
CANNED_RESPONSES entry so unmatched messages still fall through to the default
response.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2d7e10e7-044e-4f95-b511-4d6db46ae568

📥 Commits

Reviewing files that changed from the base of the PR and between bb46c4f and 210ab79.

📒 Files selected for processing (1)

• tests/e2e/mock_llm.py

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/ironclaw_reborn/src/model_gateway.rs`:
- Around line 1155-1168: The unavailable-capability guard in
model_gateway::unavailable_requested_capability_guard is still using visible
namespace membership as a proxy, so explicit requests like gmail.send can slip
through when only unrelated tools such as builtin.shell are visible. Update the
guard to detect explicit capability-request phrasing directly from
latest_user.content and suppress substitute tool calls without relying on
visible_namespaces, using visible_capability_ids and
extract_explicit_capability_ids as the key symbols to adjust. Add a regression
test at the gateway caller that exercises the full path through the model
gateway so the side effect is validated where it is triggered.

In `@tests/e2e/mock_llm.py`:
- Around line 1323-1330: The synthetic builtin_shell fallback in the mock LLM
should also honor the existing deduplication guard so it stays deterministic. In
the REQUESTED_UNAVAILABLE_TOOL_TRIGGER branch, check the same
recent_tool_results state used by the generic fallback before returning the
synthetic builtin_shell call, and suppress the repeat if that tool was already
emitted. Keep the fix within the mock LLM flow around
REQUESTED_UNAVAILABLE_TOOL_TRIGGER and recent_tool_results so the behavior
matches the rest of the tool-selection logic.

In `@tests/e2e/scenarios/test_reborn_webui_v2_smoke.py`:
- Around line 413-415: The test function
test_reborn_v2_disabled_tool_does_not_route_through_shell uses
disabled_echo_shell_ask_policy only for fixture side effects, so Ruff flags it
as an unused argument. Rename that parameter in the test signature to a
leading-underscore variant, or otherwise reference it inside the test, to
satisfy ARG001 while preserving the current behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 52a66966-2211-4021-906f-8a7b4e6e35d6

📥 Commits

Reviewing files that changed from the base of the PR and between b503337 and 5647f39.

📒 Files selected for processing (4)

• crates/ironclaw_reborn/src/model_gateway.rs
• crates/ironclaw_reborn/tests/llm_gateway.rs
• tests/e2e/mock_llm.py
• tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/ironclaw_reborn/src/model_gateway.rs`:
- Around line 1156-1159: The explicit capability filtering in
model_gateway::extract_explicit_capability_request_ids currently keeps only the
first hidden ID, which causes UnavailableCapabilityGuard to block visible
fallback tool calls too. Update the gateway flow to preserve the full explicit
request set, or at minimum the explicitly requested visible capability IDs, and
make UnavailableCapabilityGuard suppress only substitute tools that were not
directly requested. Adjust the caller that constructs the guard so it passes the
richer request information through. Add a regression test at the real gateway
call site that verifies a prompt like “builtin.echo, or builtin.shell if echo is
unavailable” still allows the directly requested visible fallback.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d06718da-592e-4203-94c1-d8e6a68c619c

📥 Commits

Reviewing files that changed from the base of the PR and between 3c1de9a and b8976eb.

📒 Files selected for processing (4)

• crates/ironclaw_reborn/src/model_gateway.rs
• crates/ironclaw_reborn/tests/llm_gateway.rs
• tests/e2e/mock_llm.py
• tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

[rpelevin]

I would make this fail-closed at the boundary between visible capability inventory and execution routing, not only in the prompt wording.

If a requested capability is disabled or absent, the loop can explain that the capability is unavailable, but it should not satisfy the same intent through a broader executable capability unless a fresh visible-capability selection path authorizes that alternate route.

The regression fixture I would keep is:

1. Disable a narrow harmless capability while leaving a broader shell or browser capability available.
2. Ask for the disabled capability's exact effect.
3. Assert the materialized instruction bundle includes the current visible-capability fingerprint.
4. Assert no alternate tool call is emitted to accomplish the same effect through the broader capability.
5. Assert the terminal record distinguishes unavailable capability from tool failure or model refusal.
6. Negative check: after a capability is disabled, a stale materialized bundle without the new fingerprint cannot be reused.

For the mock path, the unavailable-capability response should be accepted because it is the policy-compliant terminal outcome, not because the policy prose happened to appear in prior message history.

Boundary: architecture and regression-test feedback only; no claim about running this branch, validating implementation behavior, merge readiness, security review, production readiness, partnership, or customer interest.