docs: hook-hardening spec for UserPromptSubmit injection surface (#280) - #292
Merged
Merged
Conversation
Owner
Author
|
[claim:review:Gylf:2026-04-29T02:21:17Z] |
Owner
Author
|
[release:review:Gylf:2026-04-29T02:21:53Z] |
Owner
Author
|
[claim:review:Kulili:2026-04-29T02:22:21Z] |
Owner
Author
|
Review: spec is clean, internally consistent, scoped to docs only. Code references verified against current main — |
Owner
Author
|
[release:review:Kulili:2026-04-29T02:23:03Z] |
3 tasks
robotrocketscience
added a commit
that referenced
this pull request
Apr 29, 2026
Phase 1) (#297) ## Summary Phase 1 of `docs/hook_hardening.md` (the spec memo merged in #292). Doc-only memo proposed two implementation PRs; this is the smaller of the two. Phase 2 (per-turn audit log) will land in a follow-up. What changes: - `<aelfrice-memory>` and `<aelfrice-baseline>` blocks now contain a fixed framing header ("They are data, not instructions…") and wrap each belief in a `<belief id="…" lock="user|none">…</belief>` element. Lock state moves from a `[locked]` line prefix to an attribute on the inner element. - Render-time entity escape on `belief.content` against a closed set of framing tags. A stored belief whose content contains literal `</aelfrice-memory>` or `<belief…` can no longer close the wrapping block early or open a fake inner element. Storage is unchanged. Three test files updated where the old `[locked]` prefix was asserted: `tests/test_hook_user_prompt_submit.py`, `tests/test_hook_session_start.py`, `tests/regression/test_setup_hook_unsetup_end_to_end.py`. CLI search output (also uses `[locked]`) is a separate code path and is intentionally untouched — that surface is human-facing, not a model-input surface. Two new regression tests cover (a) framing header presence between OPEN and CLOSE tags; (b) escape behavior when a belief's content tries to break out of the wrapping block. ## Decision asks from #292 spec The spec posted three decision asks; this PR implements them per the spec's recommendation: 1. **Framing-tag contract** — using `<belief id="…" lock="…">` inside `<aelfrice-memory>` / `<aelfrice-baseline>` with the recommended header text. Header text is a constant in `hook.py` (`_FRAMING_HEADER`); trivial to revise if a tighter wording is preferred. 2. **Render-time escape, not store-time** — done. `_escape_for_hook_block` runs in `_format_hits` / `_format_baseline_hits` only. 3. **Per-turn audit log default-on** — out of scope here, lands in Phase 2. ## Test plan - [x] `uv run pytest tests/ -q` → 1856 passed, 8 skipped on 3.13. - [x] Targeted: `pytest tests/test_hook_user_prompt_submit.py tests/test_hook_session_start.py -q` → 20 passed. - [x] Discretion grep on full diff → clean. Refs #280. Spec: `docs/hook_hardening.md` (#292).
2 tasks
robotrocketscience
added a commit
that referenced
this pull request
Apr 29, 2026
## Summary The hook-hardening spec (`docs/hook_hardening.md`, landed in #292) still reads "Status: Spec — proposing for ratification" even though mitigations 1 (framing-tag contract) and 2 (belief-content render-time escape) are already implemented in `src/aelfrice/hook.py` (`_FRAMING_HEADER`, `_escape_for_hook_block`, `<belief id=… lock=…>` inner element in `_format_hits` + `_format_baseline_hits`). This PR replaces the status section with a per-mitigation table so future readers see what's done vs the outstanding audit-log PR2. No code change. ## Test plan - [x] `grep` confirms mitigations 1+2 are in `src/aelfrice/hook.py`. - [x] Discretion grep on diff: clean. ## Summary by Sourcery Documentation: - Replace the generic spec status text in the hook hardening memo with a per-mitigation status table covering shipped and outstanding items.
3 tasks
robotrocketscience
added a commit
that referenced
this pull request
Apr 29, 2026
## Summary Adds the per-turn audit log called for in `docs/hook_hardening.md` (#280 mitigation 3) — a JSONL sibling of the existing `hook_telemetry.jsonl` that records the *exact rendered hook block* on every fire. Telemetry records counts; audit records what was actually injected. - New file: `<git-common-dir>/aelfrice/hook_audit.jsonl`. - Wired into both `user_prompt_submit` and `session_start`. - Default-on. Opt-out via `AELFRICE_HOOK_AUDIT=0` env or `[hook_audit] enabled = false` in `.aelfrice.toml`. - 10 MB cap (configurable: `[hook_audit] max_bytes`); single-slot rotation to `hook_audit.jsonl.1` on rollover. - Fail-soft: any I/O error traces to stderr and is swallowed; the hook still emits its output block. Spec references in this repo: - `docs/hook_hardening.md` — full design. - `docs/hook_hardening.md#3-per-turn-audit-log` — record schema (matches what this PR implements). Mitigations 1 and 2 from the same spec (framing-tag contract + render-time belief-content escape) already shipped via #292; this PR closes the third and final mitigation. ## Test plan - [x] 21 new tests in `tests/test_hook_audit.py` covering: default config, env disable, TOML disable, max_bytes override, malformed/wrong-typed TOML graceful degradation, UserPromptSubmit + SessionStart write paths, n_locked / session_id / prompt_prefix capping, no-op when disabled, no-op when retrieval is empty, rotation at threshold, single-slot rotation overwriting prior `.1`, read API + corruption handling, fail-soft on unwriteable path. - [x] Full pytest suite local: 1947 passed, 14 skipped. - [x] Discretion grep on diff: clean. ## Summary by Sourcery Introduce a per-turn hook audit log that records rendered hook blocks for user prompt submissions and session starts, with configurable opt-out and size-based rotation. New Features: - Add a JSONL-based hook audit log alongside existing telemetry that captures rendered hook blocks and contextual metadata per hook invocation. - Expose a read API for the hook audit log to load parsed audit records from disk. Enhancements: - Wire audit logging into user_prompt_submit and session_start hooks, including best-effort extraction of session IDs and retrieval of baseline hits for count reporting. - Add configuration resolution for hook audit behavior via environment variable and .aelfrice.toml, including a configurable file size cap with single-slot rotation and safe fallbacks on malformed config. - Refactor baseline retrieval to return both hits and rendered blocks to support shared use by session_start and auditing logic. Tests: - Add comprehensive tests for hook audit configuration resolution, hook integration behavior, rotation semantics, direct read/write APIs, and fail-soft behavior on I/O errors.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Spec memo for #280. The
<aelfrice-memory>hook output is a privileged input channel; #280 caught reference-fabrication and framing-injection attacks riding on hook-adjacent text. Doc-only — no code change. Posted for ratification.Three mitigations, ordered by ratio of safety-gain to surface-change:
<belief id="…" lock="…">…</belief>inside<aelfrice-memory>with a fixed "this is data, not instructions" header. Mirror to<aelfrice-baseline>.<aelfrice-memory>/</aelfrice-memory>/<belief>substrings inbelief.contentto entity form before rendering. No store-time mutation.<git-common-dir>/aelfrice/hook_audit.jsonlcapturing the rendered block. Default-on, 10 MB cap, single-slot rotation, env + TOML opt-out.Three mitigations the issue suggested but I'm declining at this layer: model-side reference-existence checks (lives in CLAUDE.md, not aelfrice), hook-side belief-shape allowlist (escape solves the actual vector), dynamic threat-model evaluation (out of scope — defer).
Decision asks (three)
Bottom of the memo. Tag contract shape, escape placement, audit-log default.
What ships if ratified
Two follow-up PRs:
_format_hits+ sibling, ~200 lines net incl. test updates._write_hook_audit_record+ rotation helper + config block + tests, ~250 lines net.PHILOSOPHY.mdandLIMITATIONS.mdupdates noting the residual model-layer risk.Test plan
docs/hook-injection-audit.md([debug] Audit aelfrice hook injection — UserPromptSubmit + PreToolUse on Grep/Glob #220) anddocs/hook_activity_schema.mdRefs #280