Skip to content

docs: close out hook-hardening (#280) - #327

Merged
robotrocketscience merged 1 commit into
mainfrom
docs/issue-280-close-out
Apr 30, 2026
Merged

docs: close out hook-hardening (#280)#327
robotrocketscience merged 1 commit into
mainfrom
docs/issue-280-close-out

Conversation

@yoshi280

@yoshi280 yoshi280 commented Apr 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #280. Three docs-only changes that close out the hook-hardening campaign:

  • docs/hook_hardening.md — status table updated. Mitigation 3 (per-turn audit log) shipped via feat: per-turn hook audit log (#280 mitigation 3) #314; that row was still showing "Outstanding". The closing paragraph also explicitly records that the two remaining suggestions from the original hook hardening: UserPromptSubmit memory hook is an injection surface #280 body (model-side reference-existence check, hook-side allowlist) are declined per this same memo's reasoning.
  • docs/PHILOSOPHY.md — new section Trust boundary at the hook surface. The spec's implementation tracker (docs/hook_hardening.md lines 273-276) called for this paragraph alongside the residual-risk note. Frames what the hook layer can and cannot enforce so future readers don't re-litigate the declined mitigations.
  • docs/LIMITATIONS.md — Sharp-edges entry covering the residual model-layer risk: hook framing relies on the model honoring the trust boundary; the audit log is the recovery surface.

Why this shape

The original #280 report listed four suggested mitigations. The ratified spec memo (docs/hook_hardening.md, landed in #292) accepted three (framing-tag contract, render-time tag-substring escape, per-turn audit log) and explicitly declined two:

  • Reference-existence check protocol inside the model — declined because the hook is a Python script that runs before the model thinks; it cannot enforce model verification of named session artifacts. That is a model-behavior contract and belongs in CLAUDE.md / AGENTS.md, not in aelfrice.
  • Hook-side allowlist for belief shapes — declined because belief.content is arbitrary user / ingest text by design. Any allowlist regex strict enough to block attacks would also drop legitimate beliefs; render-time escape (mitigation 2) is the right shape.

Both declinations are on lines 103-116 of the existing spec; this PR just makes the close-out structurally legible (status table reflects ship state, residual risk noted in LIMITATIONS.md, trust-boundary framing in PHILOSOPHY.md).

Test plan

  • Docs-only diff. No code under src/ touched. No test files touched.
  • Discretion grep on diff: clean.

Out of scope

Anything that would require re-opening the spec's declined-mitigation reasoning. If new evidence shows the declinations were wrong, that's a fresh spec re-ratification cycle, not a follow-up to this PR.

Summary by Sourcery

Document the shipped state and trust boundary of the UserPromptSubmit hook and its mitigations, and record the residual model-layer risk.

Documentation:

  • Update hook hardening status to mark all ratified mitigations as shipped and explicitly record declined suggestions from the original report.
  • Add philosophy documentation describing the trust boundary at the hook surface and clarifying which responsibilities belong to the hook layer versus the model.
  • Extend limitations documentation with a sharp-edges note on residual model-layer risk and the role of the hook audit log as the recovery surface.

…d trust-boundary notes

- docs/hook_hardening.md: status table updated to reflect that mitigation 3
  (per-turn audit log, hook_audit.jsonl) shipped via #314. Issue #280 is
  closed by this PR; the remaining suggestions in the original report
  (model-side reference-existence check, hook-side allowlist for belief
  shapes) remain declined per the spec's own reasoning, with the
  model-behavior side captured in the docs noted below.
- docs/PHILOSOPHY.md: new 'Trust boundary at the hook surface' section
  per hook_hardening.md lines 273-276 (the spec's implementation tracker
  called for it). Frames what the hook layer can and cannot enforce.
- docs/LIMITATIONS.md: residual-risk Sharp-edges entry — model-layer
  compliance with the framing tag is not enforceable from Python; the
  audit log is the recovery surface.
@sourcery-ai

sourcery-ai Bot commented Apr 30, 2026

Copy link
Copy Markdown

Reviewer's Guide

Documents the completion of the hook-hardening campaign by marking all ratified mitigations as shipped, explicitly recording the declined mitigations and their rationale, adding a philosophy section that defines the trust boundary at the hook surface, and capturing the remaining model-layer risk in the limitations doc.

Sequence diagram for per-turn hook processing and audit logging

sequenceDiagram
  actor User
  participant CLI as Aelfrice_CLI
  participant Hook as UserPromptSubmit_hook
  participant DB as Belief_Store_SQLite
  participant Model as LLM_Model
  participant Audit as Hook_Audit_Log

  User ->> CLI: Submit_prompt(text)
  CLI ->> Hook: OnUserPromptSubmit(text)

  Hook ->> DB: Retrieve_relevant_beliefs(text)
  DB -->> Hook: beliefs

  Hook ->> Hook: Escape_tag_substrings_in_beliefs()
  Hook ->> Hook: Format_belief_block_with_framing_tag()

  Hook ->> Model: Send(user_text, framed_belief_block)
  Model -->> CLI: Response

  Hook ->> Audit: Write_record(turn_id, user_text, framed_belief_block)
Loading

Flow diagram for hook-hardening mitigations and decisions

flowchart TD
  Start["Original_issue_280: four_suggested_mitigations"]

  Start --> M1["Mitigation_1:\nFraming_tag_contract"]
  Start --> M2["Mitigation_2:\nRender_time_escape"]
  Start --> M3["Mitigation_3:\nPer_turn_audit_log"]
  Start --> M4["Mitigation_4:\nReference_existence_check_in_model"]
  Start --> M5["Mitigation_5:\nHook_side_allowlist_for_belief_shapes"]

  M1 -->|"Ratified_in_spec_memo"| M1_impl["Implemented_in_hook_layer"]
  M2 -->|"Ratified_in_spec_memo"| M2_impl["Implemented_in_hook_layer"]
  M3 -->|"Ratified_in_spec_memo"| M3_impl["Implemented_in_hook_layer\n+ audit_log_config"]
  M4 -->|"Declined:\nmodel_behavior_contract"| M4_doc["Documented_as_model_layer_duty\nin PHILOSOPHY_and_LIMITATIONS"]
  M5 -->|"Declined:\nwould_block_legitimate_beliefs"| M5_doc["Documented_as_foot_gun;\nrender_time_escape_used_instead"]

  M1_impl --> Status["Hook_hardening_status:\nAll_ratified_mitigations_shipped"]
  M2_impl --> Status
  M3_impl --> Status
  M4_doc --> Residual["Residual_risk_is_model_layer"]
  M5_doc --> Residual
Loading

File-Level Changes

Change Details Files
Mark hook-hardening mitigations as fully shipped and explicitly document declined mitigations plus cross-references. docs/hook_hardening.md
Add a philosophy section describing the trust boundary at the UserPromptSubmit hook surface and what belongs in the model layer vs. hook layer.
  • Introduce a new 'Trust boundary at the hook surface' section explaining that the hook is a privileged channel injecting retrieved beliefs into the model input.
  • Describe the three structural defenses provided by the hook layer: fixed framing tag, render-time tag-substring escape, and per-turn audit log, with a pointer to hook_hardening.md.
  • Spell out what the hook cannot enforce (model-side verification of named artifacts, treatment of escaped content as data), and locate those obligations in model-behavior docs (CLAUDE.md / AGENTS.md).
  • Clarify that misuse of belief content as instructions is classified as a model-layer problem, not a hook-layer one.
docs/PHILOSOPHY.md
Document residual model-layer risk of the hook trust boundary and position the audit log as the recovery surface.
  • Add a sharp-edges bullet explaining that hook framing depends on the model honoring the trust boundary.
  • Summarize the three structural defenses again with links to hook_hardening.md and the new PHILOSOPHY section.
  • State that the hook cannot force verification of named session artifacts or guarantee the model treats belief content as data, classifying failures here as model-layer issues.
  • Note that hook_audit.jsonl exists as the recovery surface for post-hoc inspection of injected content.
docs/LIMITATIONS.md

Assessment against linked issues

Issue Objective Addressed Explanation
#280 Harden the UserPromptSubmit hook by structurally framing hook output as retrieved memory (not instructions) and escaping tag-like substrings so belief content cannot inject fake tags or instructions.
#280 Provide a per-turn audit log that records the exact hook-output block injected into the model input, to enable post-hoc inspection of what was injected on each turn.
#280 Add additional protections for the injection surface by (a) enforcing a model-side reference-existence check protocol for named session artifacts, and (b) implementing a hook-side allowlist for belief/tag content rendered by the hook. The PR explicitly documents that both the reference-existence check and the hook-side allowlist mitigations are declined, and instead only updates documentation to reflect the shipped mitigations and residual risk; it does not implement these two additional protections.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Apr 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (2)
  • docs:
  • release:

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f6e22080-ada5-4ae1-87e1-a0a61c86f3c7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/issue-280-close-out

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In hook_hardening.md, consider avoiding hard-coded line number references (e.g., “lines 240–252” and “lines 273–276”), since they will drift as the document evolves and become misleading over time.
  • In the status section of hook_hardening.md, you may want to rephrase time-relative language like “#280 closes on this PR” to something PR-agnostic so the docs read cleanly outside the context of this specific change.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `hook_hardening.md`, consider avoiding hard-coded line number references (e.g., “lines 240–252” and “lines 273–276”), since they will drift as the document evolves and become misleading over time.
- In the status section of `hook_hardening.md`, you may want to rephrase time-relative language like “#280 closes on this PR” to something PR-agnostic so the docs read cleanly outside the context of this specific change.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@robotrocketscience

Copy link
Copy Markdown
Owner

[claim:review:Setr:2026-04-30T22:22:58Z]

@robotrocketscience
robotrocketscience merged commit 94d6de3 into main Apr 30, 2026
10 of 16 checks passed
@robotrocketscience
robotrocketscience deleted the docs/issue-280-close-out branch April 30, 2026 22:24
@robotrocketscience

Copy link
Copy Markdown
Owner

[release:review:Setr:2026-04-30T22:24:08Z]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hook hardening: UserPromptSubmit memory hook is an injection surface

2 participants