feat(retrieval): graduate HRR structural-query lane onto production retrieve() (#1107 phase 5, #152) by robotrocketscience · Pull Request #1117 · robotrocketscience/aelfrice

robotrocketscience · 2026-07-07T17:05:34Z

What

Graduates the HRR structural-query lane (use_hrr_structural, #152) onto
the production retrieve() path — the final staged lane in the #1107 cutover.
The shim now passes use_hrr_structural=None (resolver-driven: env → TOML →
default-ON) instead of hard-off, so a production host runs the lane the moment
its resolver says on. The resolver default has been True since v2.1
(#154/#437 gate), so this is a single-step graduation.

Why it's safe

The lane is marker-routed. retrieve_v2 calls _route_structural_query →
parse_structural_marker(query), which returns None before any index build
unless the query starts with an exact <KIND>:<target_id> marker (e.g.
CONTRADICTS:<id>). On any natural-language query — the only kind the
UserPromptSubmit hook, context_rebuilder, and mcp_server callers issue —
the lane is a pure pass-through: one fast-failing regex, then None, then the
byte-identical legacy path. The expensive HRR index build is gated behind a
marker hit.

Characterization evidence (epic #1107, 2026-07-07):

Byte-identical on NL queries: output diverged on 0/30 non-marker
queries (spine corpus, 10k beliefs).
Latency: one regex match on the non-marker hot path; measured Δp50 ≈ 0.
Lane is real when it fires: a TEMPORAL_NEXT:<id> marker returned 0 hits
with the lane off (textual match on the literal string) vs 50 hits on.

Changes

src/aelfrice/retrieval.py — shim flips use_hrr_structural False → None;
comments move it from held-off to graduated (four graduated lanes now, two
held).
tests/test_retrieve_v2_equivalence.py — SHIM_LANES mirrors the shim so
retrieve() == retrieve_v2(**SHIM_LANES) stays an identity by construction;
test_shim_runs_graduated_lanes_others_off gains a fourth graduated-lane
clause (structural marker over a CONTRADICTS edge), keeping origin
tie-break as the held-off proof.
benchmarks/hrr_structural_ablation.py — deterministic on-HEAD analogue of
the [v2.0] Reproducibility harness — benchmarks/results/v2.0.0.json is canonical, uv sync && aelf bench all #437 gate: synthetic edge-connected <KIND>:<target_id> marker corpus
whose answers share no vocabulary with the marker string. recall@k
0.000 → 1.000 (textual BM25 cannot reach a structural answer; the lane routes
it exactly). CI-safe, no live-store content.
docs/user/CONFIG.md, CHANGELOG/v3.md — record the graduation.

Epic disposition

This is Phase 5, the last staged pack-touching lane. With it merged, the
#1107 cutover is complete:

ON (resolver-driven, graduated): temporal-spine (Temporal spine: ingest-time chronological edges + dedicated retrieval lane (confirmed +14.6pp coverage on LoCoMo) #1064, Phase 2),
entity-persist demotion (Entity-persistence demotion prior: a deterministic organic sink for coordination junk (follow-up to #1086/#1081) #1096, Phase 3), intentional clustering ([v2.0] Intentional clustering — co-locate related beliefs for multi-fact coherence #436,
Phase 4), HRR structural ([retrieval] HRR structural-query lane (bind/probe over outgoing edges) #152, Phase 5).
OFF (intentional): origin tie-break (refuted feat(retrieval): default-off origin-tier rerank lane de-ranking doc chunks below user facts (#1011) #1013), HRR-expand
(recall-neutral eval: run the deferred +HRR-expand ablation arm on the #977 canonical sweep (gated on edge-type expansion) #1001).

Production retrieve() now converges on retrieve_v2 with every valuable lane
live and the refuted/neutral lanes explicitly held off.

Closes #1107.
Refs #152.

Summary by Sourcery

Graduate the HRR structural-query lane onto the production retrieve() path as the final phase of the #1107 cutover while keeping non-marker queries behavior byte-identical.

New Features:

Expose an on-HEAD benchmark (hrr_structural_ablation.py) that measures recall uplift of the HRR structural-query lane on synthetic marker queries.
Enable resolver-driven HRR structural-query routing on the production retrieve() path for <KIND>:<target_id> marker queries.

Enhancements:

Align the retrieve() shim and SHIM_LANES config to treat the HRR structural-query lane as graduated alongside the other live lanes.
Extend the equivalence test suite to assert that the HRR structural lane is load-bearing when on and remains distinct from the lane-off behavior.
Document the production graduation of the HRR structural-query lane and its marker-routed, no-op behavior on non-marker queries in CONFIG.md and CHANGELOG.

…phase 5 gate) Deterministic on-HEAD analogue of the #437 reproducibility-harness gate: a synthetic edge-connected corpus of <KIND>:<target_id> marker queries whose answers share no vocabulary with the marker string. recall@k 0.000 -> 1.000 (textual BM25 cannot reach a structural answer; the lane routes it exactly). CI-safe, no live-store content. Complements the existing tests/test_retrieve_v2_hrr_structural.py IT1-IT6 wiring tests.

…etrieve() (#1107 phase 5, #152) The #1107 shim now passes use_hrr_structural resolver-driven (None -> env->TOML->default-ON) instead of hard-off, so the marker-routed structural lane reaches the production retrieve() hook path. The resolver default was already True (v2.1, #154/#437 gate) so this is a single-step graduation. The lane is marker-routed: on any non-marker query parse_structural_marker returns None and the call falls through byte-identically, adding only one fast-failing regex to the non-marker hot path; the expensive HRR index build is gated behind a marker hit. SHIM_LANES mirrors the shim (four graduated lanes None, two held False) so retrieve() == retrieve_v2(**SHIM_LANES) stays a by-construction identity. test_shim_runs_graduated_lanes_others_off gains a fourth graduated-lane clause (structural marker over a CONTRADICTS edge) and keeps origin tie-break as the held-off proof. Full suite 5840 passed.

…1107 phase 5, #152) CONFIG.md: use_hrr_structural marked live on retrieve() via the #1107 Phase-5 cutover, noting the marker-routed no-op fall-through on non-marker queries. CHANGELOG: Unreleased Added entry for the Phase-5 graduation with the recall 0.000->1.000 ablation signal.

github-actions · 2026-07-07T17:05:51Z

PR-size soft cap

This PR is over the advisory size threshold:

232 changed lines (limit: 200)
5 changed files (limit: 3)

Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated attn:merge-conflict cycles (see #602). When practical, split into smaller PRs that each touch a focused surface.

This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the size:override label and this comment will be removed on the next push.

coderabbitai · 2026-07-07T17:06:41Z

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 35 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2bdc1d2b-c581-4779-84c1-79f33c95c5a8

📥 Commits

Reviewing files that changed from the base of the PR and between 3c5087f and 811140b.

📒 Files selected for processing (5)

CHANGELOG/v3.md
benchmarks/hrr_structural_ablation.py
docs/user/CONFIG.md
src/aelfrice/retrieval.py
tests/test_retrieve_v2_equivalence.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/1107-phase5-hrr-structural

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

sourcery-ai · 2026-07-07T17:07:41Z

Reviewer's Guide

Graduates the HRR structural-query lane from staged to production by wiring it through the retrieve() shim as resolver-driven (default-on) rather than hard-off, updates the equivalence tests and docs to reflect four graduated lanes and two held lanes, and adds a synthetic benchmark that demonstrates recall uplift of the structural lane on marker queries.

Sequence diagram for HRR structural-query lane on the production retrieve() path

sequenceDiagram
    actor User
    participant retrieve
    participant resolver
    participant retrieve_v2
    participant parse_structural_marker
    participant HRR_lane

    User->>retrieve: retrieve(store, query)
    retrieve->>resolver: resolve use_hrr_structural (env, TOML, default)
    resolver-->>retrieve: use_hrr_structural (True/False)

    retrieve->>retrieve_v2: retrieve_v2(store, query, use_hrr_structural)

    alt use_hrr_structural is True
        retrieve_v2->>parse_structural_marker: parse_structural_marker(query)
        alt marker query
            parse_structural_marker-->>retrieve_v2: marker
            retrieve_v2->>HRR_lane: structural routing
            HRR_lane-->>retrieve_v2: structural beliefs
        else non-marker query
            parse_structural_marker-->>retrieve_v2: None
            retrieve_v2-->>retrieve_v2: [fall through to textual stack]
        end
    else use_hrr_structural is False
        retrieve_v2-->>retrieve_v2: [skip structural lane]
    end

    retrieve_v2-->>retrieve: beliefs
    retrieve-->>User: beliefs

File-Level Changes

Change	Details	Files
Make the HRR structural-query lane live on the production retrieve() path via the shim configuration.	Change the `retrieve()` shim call to `retrieve_v2` so `use_hrr_structural` is passed as `None` (resolver-driven) instead of `False` (forced off). Update surrounding comments to list HRR structural as a graduated lane and adjust the count/description of remaining staged lanes. Ensure the lane is documented as resolver default-on and marker-routed with no-op behaviour on non-marker queries.	`src/aelfrice/retrieval.py`
Align the equivalence test shim configuration and coverage with the graduated HRR structural lane.	Update the `SHIM_LANES` dict so `use_hrr_structural` is `None` and listed alongside other graduated lanes, with only origin tie-break and HRR-expand left forced off. Extend the module docstring/comments to describe HRR structural as Phase 5 and to state that only two staged lanes remain off. Add a new HRR-structural clause in `test_shim_runs_graduated_lanes_others_off` that constructs a small edge-connected corpus with a `CONTRADICTS` edge and asserts that `retrieve()` matches `retrieve_v2` with structural on and diverges from structural off. Import `EDGE_CONTRADICTS` for use in the new structural test setup.	`tests/test_retrieve_v2_equivalence.py`
Document the graduation of the HRR structural lane and its behaviour on the production path.	Update the `[retrieval]` config documentation to mark `use_hrr_structural` as live on the production `retrieve()` path via the epic(retrieval): converge production hook onto retrieve_v2 — staged lanes are not on the live path #1107 Phase-5 cutover and emphasize its marker-routed, no-op behaviour on non-marker queries. Expand the dedicated `use_hrr_structural` section to explain that it is now live on `retrieve()` (not just `retrieve_v2`), describe the blast radius limitation to `<KIND>:<target_id>` queries, and reiterate the marker-routing pipeline. Add a CHANGELOG entry under v3 describing the structural lane graduation, opt-out mechanisms, and the new benchmark as a public on-head signal, while noting that equivalence now pins four graduated lanes and one held lane.	`docs/user/CONFIG.md` `CHANGELOG/v3.md`
Introduce a deterministic synthetic benchmark to validate recall uplift from the HRR structural-query lane.	Add `benchmarks/hrr_structural_ablation.py`, which builds a synthetic edge-connected corpus with `<KIND>:<target_id>` marker queries whose answers share no vocabulary with the marker string. Implement helper functions to generate content, build beliefs/edges, and compute recall@k with the structural lane off vs on using `retrieve_v2` and `HRRStructIndexCache`. Print a concise report including corpus statistics, recall values, uplift, and a simple ship-condition check (recall uplift > 0).	`benchmarks/hrr_structural_ablation.py`

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1107	Converge the production retrieval path `retrieve()` onto the `retrieve_v2` lane configuration by graduating the HRR structural-query lane (`use_hrr_structural`) from retrieve_v2-only into the live production path used by hooks/context_rebuilder/mcp_server.	✅
#1107	Update tests, benchmarks, and user-facing documentation to accurately reflect that the HRR structural-query lane is default-on in the resolver and now live on the production `retrieve()` path, clarifying the prior retrieve_v2-only behaviour and the completed cutover state.	✅

Possibly linked issues

#epic(retrieval) – converge production hook onto retrieve_v2: PR finalizes the epic(retrieval): converge production hook onto retrieve_v2 — staged lanes are not on the live path #1107 retrieve_v2 shim cutover, answering the epic’s retrieve()/retrieve_v2 migration disposition.

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey - I've reviewed your changes and they look great!

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

robotrocketscience · 2026-07-07T17:14:10Z

[claim:review:Setr:2026-07-07T17:14:08Z]

robotrocketscience · 2026-07-07T17:15:48Z

Review: approve. This is the Phase-5 graduation I was about to build; it correctly builds on the #1107 characterization I posted earlier today and completes the cutover.

Verified:

Core change is the clean single-step graduation: use_hrr_structural=False → None. Resolver default has been True since v2.1 ([retrieval] Pipeline composition tracker — unified retrieve() with feature-flag gate #154/[v2.0] Reproducibility harness — benchmarks/results/v2.0.0.json is canonical, uv sync && aelf bench all #437 gate cleared 11/11), so no default flip here.
Safety corroborated by the characterization. The lane is marker-routed: _route_structural_query → parse_structural_marker returns None before any index build on non-marker queries. My independent measurement: 0/30 divergence on NL queries (byte-identical), one regex on the hot path (Δp50 ≈ 0), and the lane provably fires on markers (TEMPORAL_NEXT:<id> → 0 hits off, 50 on). The UserPromptSubmit/rebuilder/mcp callers issue only NL queries, so production blast radius is confined to structured-tooling marker callers.
Equivalence net — non-vacuous structural-live clause. b_src -CONTRADICTS-> b_tgt, query CONTRADICTS:b_tgt: asserts b_src in prod ∧ prod == structural_on ∧ prod != structural_off, with NATO-vocab contents sharing no tokens with the marker so the arms are unambiguously distinct. Held-lane origin-tiebreak clause retained. Four graduated lanes now proven live.
Benchmark hrr_structural_ablation.py: deterministic, CI-safe (no live-store), 3 edge kinds × disjoint vocab, recall 0.000→1.000 — proves textual BM25 cannot reach a structural answer while the lane routes it exactly. Honestly framed as the on-HEAD analogue of the [v2.0] Reproducibility harness — benchmarks/results/v2.0.0.json is canonical, uv sync && aelf bench all #437 gate.
FF-clean, 3 commits SSH-signed, CI green, discretion grep clean.

Epic disposition is accurate — Closes #1107 is correct. All six staged lanes dispositioned: graduated = temporal-spine (#1114), entity-persist (#1115), clustering (#1116), HRR-structural (this PR); held-off intentional = origin-tiebreak (refuted #1013), HRR-expand (recall-neutral #1001). Production retrieve() fully converges on retrieve_v2 with every valuable lane live.

Adding ready-to-merge.

github-actions · 2026-07-07T17:16:31Z

merge-train: merged 811140b → main via FF push.

robotrocketscience · 2026-07-07T17:16:50Z

[release:review:Setr:2026-07-07T17:16:50Z]

robotrocketscience added 3 commits July 7, 2026 09:51

robotrocketscience added the author-Kulili PR coordination mutex label Jul 7, 2026

robotrocketscience added the attn:review Needs review (PR open, awaiting reviewer) label Jul 7, 2026

sourcery-ai Bot reviewed Jul 7, 2026

View reviewed changes

robotrocketscience added the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 7, 2026

github-actions Bot removed the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 7, 2026

github-actions Bot merged commit 811140b into main Jul 7, 2026
40 of 41 checks passed

github-actions Bot mentioned this pull request Jul 7, 2026

epic(retrieval): converge production hook onto retrieve_v2 — staged lanes are not on the live path #1107

Closed

Navigation Menu

feat(retrieval): graduate HRR structural-query lane onto production retrieve() (#1107 phase 5, #152) - #1117

feat(retrieval): graduate HRR structural-query lane onto production retrieve() (#1107 phase 5, #152)#1117
github-actions[bot] merged 3 commits into
mainfrom
feat/1107-phase5-hrr-structural

robotrocketscience commented Jul 7, 2026 •

edited by sourcery-ai Bot

Loading

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

coderabbitai Bot commented Jul 7, 2026

Review limit reached

Uh oh!

sourcery-ai Bot commented Jul 7, 2026 •

edited

Loading

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai Bot left a comment

Uh oh!

robotrocketscience commented Jul 7, 2026

Uh oh!

robotrocketscience commented Jul 7, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

robotrocketscience commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

robotrocketscience commented Jul 7, 2026 • edited by sourcery-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why it's safe

Changes

Epic disposition

Summary by Sourcery

Uh oh!

github-actions Bot commented Jul 7, 2026

PR-size soft cap

Uh oh!

coderabbitai Bot commented Jul 7, 2026

Review limit reached

Uh oh!

sourcery-ai Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence diagram for HRR structural-query lane on the production retrieve() path

File-Level Changes

Assessment against linked issues

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

robotrocketscience commented Jul 7, 2026

Uh oh!

robotrocketscience commented Jul 7, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

robotrocketscience commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

robotrocketscience commented Jul 7, 2026 •

edited by sourcery-ai Bot

Loading

sourcery-ai Bot commented Jul 7, 2026 •

edited

Loading