Skip to content

feat(lock): inject reference-tier locks as a bounded manifest (#1016-B) - #1037

Merged
github-actions[bot] merged 2 commits into
mainfrom
feat/issue-1016-lock-manifest-injection
Jun 30, 2026
Merged

github-actions[bot] merged 2 commits into
mainfrom
feat/issue-1016-lock-manifest-injection

Conversation

@robotrocketscience

@robotrocketscience robotrocketscience commented Jun 30, 2026

Copy link
Copy Markdown
Owner

Part of #1016 — the second slice of #1016-B, building on the lock_tier field (PR #1036). This is the behavior-delivering slice: it actually bounds reference-lock injection and frees relevance budget. Does not close the umbrella.

What

At the two hook injection paths — per-prompt UserPromptSubmit and SessionStart — a reference-tier lock now renders as a one-line manifest entry instead of full text:

<aelfrice-locks-manifest note="bounded reference locks (#1016); read full text on demand via `aelf locked` / `aelf search`">
  ref <id>: "<topic>"
</aelfrice-locks-manifest>

retrieve() gains manifest_reference_locks (the hook paths pass it): reference locks then cost only their manifest line in the budget, freeing relevance budget that fat locks otherwise consume (the #1014/#1015 starvation the bound targets). The topic is a deterministic string transform (first sentence, or an 80-char cap with ellipsis — no ML, per the locked determinism philosophy #605).

Safety: byte-identical until demotion

lock_injection_tokens(frozen) == _belief_tokens, the manifest_reference_locks default is off, and every lock is frozen until the user demotes it. So existing stores, the rebuilder, aelf search, the search-tool hook, and all benchmarks are byte-identical — the new path only activates for locks explicitly demoted to reference.

Scope kept tight to the two always-injected formatters + retrieve(). The rebuild-block and search-tool formatters still render reference locks verbatim (their own retrieve calls keep the default-off budget, so they stay consistent — verbatim render + verbatim cost); folding them into the manifest is a follow-up.

Tests

tests/test_lock_manifest_injection.py: deterministic topic + cap, manifest-line shape, token accounting (reference < full when flagged; frozen always full), is_reference_lock, formatter manifestizes reference / byte-identical without reference locks, and a retrieve() budget-freed assertion under budget pressure.

Verification

  • Full suite: 5463 passed, 66 skipped, 75 xfailed (core retrieval/injection path — no regressions).
  • uvx vulture … --min-confidence 80 → clean. uvx typos → clean.

🤖 Generated with Claude Code

Summary by Sourcery

Render reference-tier locks as bounded manifest entries at hook injection paths and account for their reduced injection cost in retrieval budgeting, while keeping behavior byte-identical until locks are demoted.

New Features:

  • Introduce a locks manifest block that surfaces reference-tier locks as one-line ref <id>: "<topic>" entries in UserPromptSubmit and SessionStart hook outputs.
  • Add deterministic topic generation and manifest-line helpers for reference-tier locks, enabling manifest-only injection without ML dependence.
  • Extend retrieve() with a manifest_reference_locks flag to treat reference-tier locks as manifest-sized for budgeting in specific injection paths.

Enhancements:

  • Refactor hook-formatting logic to split verbatim belief lines from manifest entries and wrap reference locks in a dedicated manifest block only when present.
  • Adjust locked-belief token accounting to use manifest cost for reference-tier locks when enabled, freeing relevance budget under tight budgets.

Documentation:

  • Update the v3 changelog to document layered locks injection behavior and the new bounded manifest treatment of reference-tier locks.

Tests:

  • Add acceptance tests covering manifest-topic determinism and capping, manifest line shape, token accounting for frozen vs reference locks, formatter rendering behavior, and retrieval budget freeing behavior under pressure.

Second slice of layered locks: at the hook injection paths (UPS +
SessionStart) a reference-tier lock renders as a one-line manifest entry
(ref <id>: "<topic>") inside an <aelfrice-locks-manifest> block instead
of full text; the agent reads full text on demand via aelf locked/search.
retrieve() gains manifest_reference_locks (hook paths pass it): reference
locks then cost only their manifest line, freeing relevance budget fat
locks otherwise consume (#1014/#1015). Topic is a deterministic string
transform (first sentence or 80-char cap, no ML). Frozen locks unchanged;
byte-identical until a lock is demoted to reference (default flag off), so
existing stores, the rebuilder, aelf search, and benches are unaffected.
Rebuild-block + search-tool formatters still render reference verbatim
(follow-up).
@sourcery-ai

sourcery-ai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Reviewer's Guide

Implements bounded injection for reference-tier locks by rendering them as a one-line manifest in hook formatters and updating retrieval budgeting to treat reference locks as manifest-sized instead of verbatim, with behavior gated by a new manifest_reference_locks flag and covered by acceptance tests.

Sequence diagram for bounded manifest injection of reference-tier locks

sequenceDiagram
    actor User
    participant hook as hook
    participant retrieval as retrieval
    participant lock_tokens as lock_injection_tokens
    participant manifest_line as lock_manifest_line

    User->>hook: _format_hits / _format_hits_with_session_start
    hook->>retrieval: retrieve(store, prompt, token_budget, manifest_reference_locks=True)
    retrieval->>lock_tokens: lock_injection_tokens(belief, manifest_reference_locks=True)
    alt is_reference_lock(belief)
        lock_tokens->>manifest_line: lock_manifest_line(belief)
        lock_tokens-->>retrieval: token cost of manifest line
        retrieval-->>hook: hits with reference locks bounded
        hook->>hook: _split_belief_lines(hits)
        hook->>hook: _manifest_block_lines(manifest_lines)
        hook-->>User: memory/baseline with <aelfrice-locks-manifest>
    else not is_reference_lock(belief)
        lock_tokens-->>retrieval: _belief_tokens(belief)
        retrieval-->>hook: hits with full content
        hook-->>User: memory/baseline with verbatim <belief> lines
    end
Loading

File-Level Changes

Change Details Files
Render reference-tier locks as a one-line manifest block in hook injection paths instead of verbatim content, while keeping output byte-identical when no reference locks are present.
  • Introduce LOCKS_MANIFEST_OPEN_TAG and LOCKS_MANIFEST_CLOSE_TAG constants for the new wrapper block.
  • Factor belief-line rendering into _split_belief_lines that separates verbatim lines from reference-lock manifest entries using retrieval.is_reference_lock and retrieval.lock_manifest_line.
  • Add _manifest_block_lines helper to conditionally wrap manifest entries in the manifest block or omit it entirely when there are no reference locks.
  • Update _format_hits, _format_hits_with_session_start, and _format_baseline_hits to use _split_belief_lines and _manifest_block_lines so reference locks render as manifest entries and other beliefs remain verbatim.
src/aelfrice/hook.py
Adjust retrieval budgeting and lock helpers so reference-tier locks can be accounted as manifest-sized when requested, while keeping the default path identical to previous behavior.
  • Import LOCK_TIER_REFERENCE into retrieval to distinguish reference-tier locks.
  • Add _lock_topic to derive a deterministic, capped one-line topic string from lock content (first sentence when short, otherwise an 80-char cap with ellipsis, with whitespace collapsing and double-quote normalization).
  • Add lock_manifest_line to format a single manifest entry line for a reference-tier lock as ref <id>: "<topic>".
  • Add is_reference_lock helper to detect user locks demoted to the reference tier via lock_level and lock_tier.
  • Add lock_injection_tokens to compute injection token cost of locked beliefs, using manifest-sized cost for reference locks when manifest_reference_locks is True and full content otherwise.
  • Extend retrieve() with a manifest_reference_locks boolean parameter (default False) that threads into the internal cost computation.
  • Update _cost() to compute locked_used via lock_injection_tokens so that, when manifest_reference_locks is enabled, reference locks free relevance budget while frozen locks still cost full content.
src/aelfrice/retrieval.py
Enable manifest-based budgeting for reference-tier locks in the hook search path while preserving existing audit behavior.
  • Pass manifest_reference_locks=True from hook_search.search_for_prompt into retrieve so the search hook uses manifest-sized cost for reference locks in its budget.
  • Document that this path’s formatter renders reference-tier locks as manifest entries and that the behavior is byte-identical until a lock is demoted to reference.
src/aelfrice/hook_search.py
Document the new reference-lock manifest injection behavior in the v3 changelog.
  • Add a v3 changelog entry describing layered locks injection, the one-line manifest format ref <id>: "<topic>" in , the manifest_reference_locks flag on retrieve, and the byte-identical-until-demotion guarantee.
  • Clarify that rebuild-block and search-tool formatters still render reference locks verbatim and will be folded into the manifest in a follow-up.
CHANGELOG/v3.md
Add acceptance tests to cover manifest topics, manifest-line formatting, token accounting, formatter output, and retrieval-budget effects for reference-tier locks.
  • Introduce a helper _b() to construct Belief instances with various lock levels and tiers for tests.
  • Test _lock_topic determinism, capping behavior, and sentence selection for short content.
  • Test lock_manifest_line shape and quoting for reference locks.
  • Verify lock_injection_tokens charges manifest-sized cost when manifest_reference_locks is True for reference-tier locks and full content for frozen locks regardless of the flag.
  • Test is_reference_lock behavior for locked reference-tier beliefs, frozen-tier beliefs, and non-locked beliefs.
  • Check formatter behavior: reference locks are emitted as manifest entries with the manifest tags and do not include full content verbatim; outputs are byte-identical to pre-feat(locks): bound lock injection + fix lock framing — frozen/reference tiers, locks-file manifest, provenance-aware framing #1016 when no reference locks are present.
  • Add a retrieval test showing that enabling manifest_reference_locks under tight budgets yields at least as many non-locked hits while still returning the reference lock.
tests/test_lock_manifest_injection.py

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown

PR-size soft cap

This PR is over the advisory size threshold:

  • 338 changed lines (limit: 200)
  • 5 changed files (limit: 3)

Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated attn:merge-conflict cycles (see #602). When practical, split into smaller PRs that each touch a focused surface.

This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the size:override label and this comment will be removed on the next push.

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 6 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6e887603-27d6-47eb-8933-26f4daf67593

📥 Commits

Reviewing files that changed from the base of the PR and between ff2361b and e306ef5.

📒 Files selected for processing (5)
  • CHANGELOG/v3.md
  • src/aelfrice/hook.py
  • src/aelfrice/hook_search.py
  • src/aelfrice/retrieval.py
  • tests/test_lock_manifest_injection.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/issue-1016-lock-manifest-injection

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The token budgeting for reference locks only accounts for the manifest line itself; if you want tighter alignment with actual injection size, consider also including the manifest block wrapper tags in the locked token cost.
  • The _lock_topic sentence splitting is limited to '. ', '? ', and '! '; if lock content often uses different punctuation (e.g. line breaks, colon, semicolon) for sentence boundaries, you may want to broaden the delimiter handling to keep topics semantically sharper.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The token budgeting for reference locks only accounts for the manifest line itself; if you want tighter alignment with actual injection size, consider also including the manifest block wrapper tags in the locked token cost.
- The `_lock_topic` sentence splitting is limited to `'. '`, `'? '`, and `'! '`; if lock content often uses different punctuation (e.g. line breaks, colon, semicolon) for sentence boundaries, you may want to broaden the delimiter handling to keep topics semantically sharper.

## Individual Comments

### Comment 1
<location path="src/aelfrice/retrieval.py" line_range="562-570" />
<code_context>
+_LOCK_TOPIC_MAX: Final[int] = 80
+
+
+def _lock_topic(content: str) -> str:
+    """Deterministic one-line topic for a reference lock's manifest entry.
+
+    Whitespace-collapsed; the first sentence if it ends within the cap,
+    else a hard char-cap with an ellipsis. No ML — a pure string
+    transform so the manifest is reproducible. Internal double-quotes are
+    flattened to single so the `"<topic>"` wrapper stays unambiguous.
+    """
+    collapsed = " ".join(content.split()).replace('"', "'")
+    if not collapsed:
+        return ""
</code_context>
<issue_to_address>
**🚨 issue (security):** Manifest topic generation does not escape `<` / `&`, which may break the surrounding XML-like hook block.

Because `lock_manifest_line` is written directly into `<aelfrice-locks-manifest>` and `_lock_topic` only normalizes whitespace and quotes, any tag-like or entity-prefixed content (e.g. starting with `<script>` or `&...`) can be interpreted as markup rather than text. Please apply the same escaping as `_escape_for_hook_block` (or reuse that helper) when building `lock_manifest_line` so manifest entries are treated as literal content and cannot inject markup into the hook block.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread src/aelfrice/retrieval.py
…review)

Run the manifest line through _escape_for_hook_block like belief content,
so a reference lock cannot spoof the <aelfrice-memory>/<aelfrice-baseline>
envelope via a framing tag in its topic. Addresses Sourcery security note.
@robotrocketscience robotrocketscience added the ready-to-merge Trigger merge-train: FF main to this PR's head label Jun 30, 2026
@github-actions
github-actions Bot merged commit e306ef5 into main Jun 30, 2026
28 checks passed
@github-actions

Copy link
Copy Markdown

merge-train: merged e306ef5main via FF push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant