feat(skills): activation feedback pipeline + install idempotence - #2530
Conversation
Add an optional `feedback: Vec<String>` field to the SkillActivated event so the engine and selector can surface human-readable activation notes (chain-load reasons, marker exclusions, scoring summaries) to the UI. Wire the field through the StatusUpdate, the SSE bridge, and the gateway's activity timeline; serialize-skip empty vectors so the wire format stays backwards compatible.
When the LLM force-activates a persona via `/ceo-setup` it sometimes
follows up with a redundant `skill_install("ceo-setup")` call. The
`execute` path was already idempotent (returns `already_installed`
without touching the catalog), but `requires_approval` still gated
the call behind a confirmation prompt — pure friction on a guaranteed
no-op.
Mirror the idempotent shortcut in `requires_approval`: when a skill
with the requested name is already loaded (bundled, user, workspace,
or previously installed), return `ApprovalRequirement::Never`. The
shortcut wins even when `install_dependencies=true` because the
top-level execute is still a no-op (companions get reconciled by their
own activation paths). Regression test covers all three cases.
There was a problem hiding this comment.
Code Review
This pull request introduces a feedback field to skill activation events to display human-readable notes in the UI and updates the SkillInstallTool to skip approval prompts for already-loaded skills. Review feedback identifies a security vulnerability where the idempotency check incorrectly bypasses approval even when install_dependencies is true, potentially allowing unvetted code. Additionally, a contradiction was noted in the regression test where the assertion for dependency installation did not match the intended security behavior.
henrypark133
left a comment
There was a problem hiding this comment.
Review: idempotence shortcut weakens approval on dependency installs
The activation-feedback pieces look fine, but the new skill_install approval shortcut introduces a verified security regression.
Critical: install_dependencies=true now bypasses approval when the top-level skill is already loaded
File: src/tools/builtin/skill_tools.rs:864
requires_approval() returns ApprovalRequirement::Never as soon as registry.has(name) is true, before it checks install_dependencies. That means a request like {"name":"ceo-setup","install_dependencies":true} skips approval entirely even though execution can still fetch and install additional companion skills. The new regression test at the bottom of the file locks in that unsafe behavior rather than catching it.
Suggested fix: only take the no-op shortcut when install_dependencies is false; dependency installs still need the existing approval gate.
Summary:
- Recommended verdict: Request changes
- Prior feedback status: partially unresolved; the prior blocker about
install_dependencies=truebypassing approval still applies - Residual risk: this path changes a trust boundary, so I would want the regression test updated alongside the fix.
zmanian
left a comment
There was a problem hiding this comment.
Summary
Adds a feedback: Vec<String> field to SkillActivated events (wire-compatible via skip_serializing_if), renders skill activation cards in the web UI, and short-circuits skill_install's approval prompt when the named skill is already loaded. Clean, small change — but the approval-skip path for install_dependencies=true deserves a closer look.
Findings
Blocking
None strictly, but see the first suggestion.
Suggestions
- Approval bypass with
install_dependencies=true(src/tools/builtin/skill_tools.rs:854-875+ test at:1407). The shortcut returnsApprovalRequirement::Neverwhenever the top-level skill name is already loaded, INCLUDING wheninstall_dependencies=true. The test asserts this as correct with the comment "known-name shortcut wins over install_dependencies (execute is a no-op)". This relies onexecute()returningalready_installedand refusing to touch companions even wheninstall_dependencies=trueis set. If that contract ever changes — or if a future refactor makesinstall_dependencies=truewalk the companion list when the top skill is loaded — we have a silent approval bypass that pulls in additional prompt-injection surface. Two ways to harden:- Pin the contract with a caller-level test that drives
SkillInstallTool::executewith{"name": "loaded-skill", "install_dependencies": true}and asserts no companion was installed. Right now onlyrequires_approvalis tested in this scenario. - Or: don't take the shortcut when
install_dependencies=true— fall through to the existing gated path. The prompt cost is minimal and the safety property is easier to reason about.
- Pin the contract with a caller-level test that drives
has(name)vs name+source check. If an attacker's skillname: code-reviewis already loaded (say, an unrelated user-placed skill with the same name), an LLM can nowskill_install {name: code-review}from a different source without approval. The dedup is by name only. IfSkillRegistry::hasis purely name-based, this is an identity-spoofing lane. Worth confirming — ifhas()checks name+source/version, disregard.- Feedback strings are unbounded. No length or count cap. SSE payloads are small today, but a skill that dumps 50 lines of "gated out because X" notes becomes ugly on narrow viewports. Consider truncating each note to ~200 chars and capping the list to ~10 entries on the server side.
- Comment drift on
src/bridge/router.rs:3328-3333. "The engine event doesn't carry feedback yet; callers in the v1 path populate it directly onStatusUpdate" — this is a coordination seam (v1 populatesStatusUpdate.feedback, v2 doesn't yet). When v2 gains feedback support, this TODO will be silent unless somebody searches forVec::new()in this file. Consider a// TODO(engine-v2-feedback):tag so a future grep finds it.
Nits
crates/ironclaw_gateway/static/app.js:909-914— theJSON.parse(e.data)is unguarded. Existing handlers follow the same pattern, so no worse than before; flagging as a wider-pattern hygiene note.addSkillActivationCardappends directly togetOrCreateActivityGroup()without a max-cards guard. A flurry of activations will grow the DOM without bound. Low priority.
Verdict
COMMENT — ship after confirming the install_dependencies=true + already-loaded path doesn't install companions, or gate the shortcut on install_dependencies != true for defense in depth. The rest are minor.
henrypark133
left a comment
There was a problem hiding this comment.
Review: loaded-skill shortcut still defeats install_dependencies=true
The activation-feedback pieces look fine, but the idempotence shortcut still breaks the dependency-install path in the current head.
Concerning: install_dependencies=true is still ignored when the top-level skill is already loaded
File: src/tools/builtin/skill_tools.rs:664 and src/tools/builtin/skill_tools.rs:856
Both execute() and requires_approval() still short-circuit on registry.has(name) before honoring install_dependencies. That means a request like {"name":"ceo-setup","install_dependencies":true} returns already_installed immediately and never walks companions, even though the caller explicitly asked for dependency installation. The regression test at src/tools/builtin/skill_tools.rs:1405 still encodes the opposite expectation in its comment, which makes the contract even harder to reason about.
Suggested fix: only take the idempotent shortcut when install_dependencies is false, and update the regression test to pin that behavior.
Summary:
- Recommended verdict: Request changes
- Prior feedback status: partially unresolved; the loaded-skill shortcut still wins over the dependency-install path
- Residual risk: this also keeps the approval contract coupled to a no-op assumption, so the behavior is brittle around future refactors.
|
Addressed the dependency-install approval feedback in f3adc5e. What changed:
Verified with:
@henrypark133 this should address the change-requested comments about both approval and execution short-circuiting. @zmanian I took the defense-in-depth route you suggested: the shortcut no longer applies when |
henrypark133
left a comment
There was a problem hiding this comment.
Review: activation notes never reach the emitted events
The dependency-install approval regression looks fixed, but the new activation-feedback feature still does not make it to any emitted skill-activation event.
Concerning: feedback is added to the event schema, but every emission path still sends an empty vector
File: src/agent/agent_loop.rs:554, src/bridge/router.rs:3329, src/bridge/router.rs:3434
The new UI card expects AppEvent::SkillActivated.feedback, but select_active_skills() still returns only (skills, rewritten_message) and never computes or returns any notes. Both bridge mappings then hard-code feedback: Vec::new(), so the gateway can only ever render the skill names. In practice that means the advertised notes such as "chain-loaded from ..." or "excluded by setup marker" are never surfaced on either the channel-status path or the SSE event path.
Suggested fix: Thread the activation notes out of skill selection (or the engine event) and populate StatusUpdate::SkillActivated / AppEvent::SkillActivated from that source, then add a caller-level test that asserts non-empty feedback reaches the web event.
Code ReviewOverviewAdds Strengths
Concerns1. PR body overclaims — curly-quote / em-dash normalization isn't in this diff. 2. Feedback pipeline has no producer — scaffolding-only.
The comment at
3. 4. Test-plan checkboxes still unchecked. At minimum confirm Style / ConventionsMatches CLAUDE.md: no Risk Summary
RecommendationRequest changes, small ones. Fix the |
ilblackdragon
left a comment
There was a problem hiding this comment.
Overview
Cherry-pick of two independent changes from #2504:
- Plumb
feedback: Vec<String>throughStatusUpdate::SkillActivated→AppEvent::SkillActivated→ SSE → web UI activity card. - Make
skill_installidempotent when a skill is already loaded: no approval prompt, no 404 against ClawHub.install_dependencies=truestill walks companions.
🔴 Blocking
Build is broken — all 3 Clippy CI jobs fail. crates/ironclaw_common/src/event.rs:523 constructs AppEvent::SkillActivated { skill_names, thread_id } in the variant-enumeration list without the new feedback field:
error[E0063]: missing field `feedback` in initializer of `event::AppEvent`
--> crates/ironclaw_common/src/event.rs:523
The enum definition was updated in the same file but this sibling constructor was missed.
Pattern matches in non-updated files also break (likely masked by compiler stopping at first error):
tests/e2e_github_dev_workflow.rs:165—StatusUpdate::SkillActivated { skill_names }tests/support/test_rig.rs:409—StatusUpdate::SkillActivated { skill_names }tests/support/live_harness.rs:400— existing arm still{ skill_names }(the diff adds a new arm at line ~158 but leaves the old one untouched)
All need , .. added.
⚠️ Correctness / Design
Feedback pipeline has no producer. The field is threaded end-to-end, but every construction site passes Vec::new():
src/bridge/router.rs:3716— v1 path (comment claimsagent_loop::select_active_skillspopulates it, but that change isn't in this PR)src/bridge/router.rs:3438— v2 engine path
So the UI card renders, but never with notes. Either land the producer in the same PR, or be explicit in the body that this is plumbing-only.
PR description claim not in the diff. Body says "Skill selector normalizes typographic punctuation (curly quotes, em-dashes) during matching" — no changes to crates/ironclaw_skills/src/selector.rs are present. Either the feature was dropped during cherry-pick or the description is stale.
✅ What Works Well
append_chain_install_report_fieldsextraction cleanly deduplicates report-to-JSON rendering.- Two-phase lock pattern (
loaded_required_skillscollected under read guard,.awaitafter drop) correctly avoids holdingRwLockReadGuardacross await. requires_approvalshortcut mirrors theexecuteno-op path; theinstall_dependencies=truecarve-out is correctly gated.#[serde(default, skip_serializing_if = "Vec::is_empty")]onfeedbackpreserves SSE wire-format back-compat.- Regression tests for both approval-skip and dependency-install-when-already-loaded are tight and well-commented.
Minor
feedback: Vec<String>is unstructured — long-term, an enum of reasons (ChainLoaded,ExcludedBySetupMarker,BudgetExhausted) would align with.claude/rules/types.md. Not blocking.- No test covers the SSE → UI feedback rendering; acceptable given there's no producer yet, but add one when producers land.
Recommendation
Fix the event.rs:523 constructor + the three unupdated pattern sites, then either (a) wire up a real feedback producer or (b) re-scope the PR title/body to "plumbing + idempotence". Also remove the stale punctuation-normalization bullet.
…s list The variant-enumeration constructor in event.rs:501 was missed when the new `feedback` field was added to AppEvent::SkillActivated, breaking the build with E0063. All three Clippy CI jobs failed on this. Regression: covered by `cargo build --all-features`, which fails to compile if any variant in this list is constructed with missing fields.
|
Pushed 30d575c to fix the immediate build break (
There are also pre-existing clippy warnings ( The blocking-feedback items from the review (no producer for |
Resolves conflicts and updates the three call sites that now need `, ..` on `StatusUpdate::SkillActivated` patterns because of the new `feedback` field: - `tests/e2e_github_dev_workflow.rs:165` - `tests/support/test_rig.rs:409` - `tests/support/live_harness.rs:400` Also reworks the web UI skill-activation card to the new activity controller model that landed on staging (`_chatToolActivity` with `getOrCreateGroup` / `removeThinking`) — the old `_activityThinking` module-level var is gone, so `addSkillActivationCard` now delegates through the public helpers. Test fixes: - `skill_install_skips_approval_when_already_loaded` - `skill_install_execute_honors_dependencies_when_already_loaded` split `install_skill` into `prepare_install_to_disk` + `commit_install` to avoid holding the `std::sync::RwLock` write guard across `.await` (clippy `await_holding_lock`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Merged staging (dca5fc6). Resolved the two conflicts and fixed the post-merge fallout: Conflict resolutions:
Pattern updates: added New clippy fix: the two Full The two review items that aren't mechanical — no producer for |
The `SkillActivated` event carried an empty `feedback` field because
nothing populated it. This adds the producer end of the pipeline.
**Selector:**
- `prefilter_skills` now returns `SelectionOutcome { selected, notes }`.
- `try_select` returns a reason enum (`Selected`, `BudgetFull`,
`CandidateLimit`, `MarkerSatisfied`, `AlreadySelected`) so callers
can render distinct notes instead of opaque "skipped".
- Notes generated for:
- `<companion>: chain-loaded from <parent>`
- `<companion>: chain-load skipped (budget full)`
- `<companion>: chain-load skipped (max active skills reached)`
- `<companion>: chain-load skipped (setup already complete)`
- `<skill>: skipped (skill context budget exhausted)` for parents
that scored but didn't fit.
**Agent loop:**
- `select_active_skills` returns the notes alongside selected skills
and prepends a `<skill>: force-activated via /mention` note for each
explicit mention.
**Dispatcher:**
- Emits `StatusUpdate::SkillActivated { skill_names, feedback }` via
`channels.send_status` whenever something activated or notes exist
(so "nothing loaded because budget exhausted" surfaces too).
- Silent when nothing activated and no notes — no UI noise.
**Stale comment:**
- Router's v2-bridge comment no longer claims v1 callers populate
feedback "directly on `StatusUpdate`"; the v1 dispatcher now emits
its own event, and v2 remains empty until the Python orchestrator
is updated.
Regression: existing selector test `test_chain_load_respects_budget`,
`test_chain_load_skips_companion_with_satisfied_marker`, and
`test_chain_load_is_non_transitive` now also assert that the
corresponding note is in `outcome.notes`. The 42 selector tests and
503 agent-module tests all pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rai#2530) * feat(events): SkillActivated carries activation feedback notes Add an optional `feedback: Vec<String>` field to the SkillActivated event so the engine and selector can surface human-readable activation notes (chain-load reasons, marker exclusions, scoring summaries) to the UI. Wire the field through the StatusUpdate, the SSE bridge, and the gateway's activity timeline; serialize-skip empty vectors so the wire format stays backwards compatible. * fix(skills): skill_install never prompts when skill is already loaded When the LLM force-activates a persona via `/ceo-setup` it sometimes follows up with a redundant `skill_install("ceo-setup")` call. The `execute` path was already idempotent (returns `already_installed` without touching the catalog), but `requires_approval` still gated the call behind a confirmation prompt — pure friction on a guaranteed no-op. Mirror the idempotent shortcut in `requires_approval`: when a skill with the requested name is already loaded (bundled, user, workspace, or previously installed), return `ApprovalRequirement::Never`. The shortcut wins even when `install_dependencies=true` because the top-level execute is still a no-op (companions get reconciled by their own activation paths). Regression test covers all three cases. * fix(skills): preserve approval for dependency installs * fix(events): include feedback in AppEvent::SkillActivated all-variants list The variant-enumeration constructor in event.rs:501 was missed when the new `feedback` field was added to AppEvent::SkillActivated, breaking the build with E0063. All three Clippy CI jobs failed on this. Regression: covered by `cargo build --all-features`, which fails to compile if any variant in this list is constructed with missing fields. * feat(skills): wire up v1 feedback producer for SkillActivated The `SkillActivated` event carried an empty `feedback` field because nothing populated it. This adds the producer end of the pipeline. **Selector:** - `prefilter_skills` now returns `SelectionOutcome { selected, notes }`. - `try_select` returns a reason enum (`Selected`, `BudgetFull`, `CandidateLimit`, `MarkerSatisfied`, `AlreadySelected`) so callers can render distinct notes instead of opaque "skipped". - Notes generated for: - `<companion>: chain-loaded from <parent>` - `<companion>: chain-load skipped (budget full)` - `<companion>: chain-load skipped (max active skills reached)` - `<companion>: chain-load skipped (setup already complete)` - `<skill>: skipped (skill context budget exhausted)` for parents that scored but didn't fit. **Agent loop:** - `select_active_skills` returns the notes alongside selected skills and prepends a `<skill>: force-activated via /mention` note for each explicit mention. **Dispatcher:** - Emits `StatusUpdate::SkillActivated { skill_names, feedback }` via `channels.send_status` whenever something activated or notes exist (so "nothing loaded because budget exhausted" surfaces too). - Silent when nothing activated and no notes — no UI noise. **Stale comment:** - Router's v2-bridge comment no longer claims v1 callers populate feedback "directly on `StatusUpdate`"; the v1 dispatcher now emits its own event, and v2 remains empty until the Python orchestrator is updated. Regression: existing selector test `test_chain_load_respects_budget`, `test_chain_load_skips_companion_with_satisfied_marker`, and `test_chain_load_is_non_transitive` now also assert that the corresponding note is in `outcome.notes`. The 42 selector tests and 503 agent-module tests all pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
SkillActivatedevent now carries feedback notes explaining why skills were selected/rejectedStatusUpdate::SkillActivatedwith feedback produced by the selector (chain-load decisions, budget exhaustion, setup-marker skips, explicit/mentionforce-activations)skill_installtool no longer prompts for approval when a skill is already loaded (idempotence fix)Cherry-picked from #2504, with the v1 producer wired up in this PR.
Test plan
cargo testpassesskill_installwith already-loaded skill returns success without approval prompt🤖 Generated with Claude Code