feat(ingest): subfloor noise-pattern filter + intra-turn edge-anchor demotion (#809) - #810
Conversation
robotrocketscience
added
the
author-sankara
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (2)
📝 WalkthroughWalkthroughSubfloor noise filtering is added to ChangesSubfloor Noise Filtering and Edge Anchoring
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related issues
Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
PR-size soft capThis PR is over the advisory size threshold:
Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the |
robotrocketscience
added
the
attn:review
|
[claim:review:chomsky:2026-05-14T20:06:31Z] |
Review: code APPROVED; deferring
|
| Spec letter | This PR | |
|---|---|---|
| Mechanism | MIN_BELIEF_CONTENT_CHARS: Final[int] = 80 length floor on triple S/O slots |
Three pattern matches (code-fence prefix, : suffix, bullet prefix) at sentence-level path |
| Drop set | All sentences < 80 chars | Only sentences matching one of three patterns |
| False-positive surface | Drops some legit short claims | Drops "He said:" / any prose ending in : |
| Layer | triple_extractor |
ingest._ingest_turn_ids |
The two layers don't produce the same dropped set. Engineering rationale for the divergence is sound on both axes:
- A literal 80-char floor would drop the existing fixture
"The default port is 8080."(25 chars) and ~26 other ingest-test fixtures — the spec letter is incompatible with the existing test contract. triple_extractor's NP slots are 10-40 chars typical; any length floor there would zero-fire.
But the deviation is large enough that I'd rather you OK it explicitly before the merge-train picks it up. If you're happy with the pattern-gate framing, drop a ratify comment (or just add ready-to-merge yourself) and the bot will take it from there.
Non-blocking observations
endswith(':')false-positive scope — flagged in code + body. Lab campaign named the pattern, trade-off accepted. The fact thatextract_sentencesdoesn't split on:means this fires mainly when a header is newline-separated, which limits real-prose damage in practice. Worth re-measuring once production data accumulates.#809audit-bot comment is a false-positive — it flaggedsrc/aelfrice/ingest.pyas already-shipped because feat(ingest): speaker-attribution gate excludes assistant rows from belief creation (#785 §1) #795/docs(ingest): spec for speaker-gate + sentiment-routing + SVO floor (#785) #786 also touched it, but the §3 surface is genuinely new. No action needed.- PR is over the 200-line soft cap (392 lines / 3 files). Advisory only. Splitting would just be churn at this point.
Approving the code. Holding off on ready-to-merge until you ratify the deviation framing.
|
[release:review:chomsky:2026-05-14T20:11:07Z] |
robotrocketscience
added
attn:decisions-needed
Recommended tightening before mergeI ran an empirical audit of FindingsOn the existing belief store (~13k rows total, 901 of them end in
Short stubs dominate the Length distribution of Recommended changeAdd the spec's own constant _SUBFLOOR_COLON_MAX_LEN: Final[int] = 80
def _looks_like_subfloor_noise(sentence: str) -> bool:
stripped = sentence.strip()
if not stripped:
return False
if stripped.startswith("```"):
return True
if (stripped.endswith(":")
and len(stripped) < _SUBFLOOR_COLON_MAX_LEN):
return True
if _SUBFLOOR_BULLET_PREFIX.match(stripped):
return True
return FalseEffects vs the PR as it stands:
Why
|
robotrocketscience
added
ready-to-merge
|
Flipping For the audit trail: the |
|
Follow-up filed as #818 capturing the |
Deeper check on the bullet + code-fence rulesFollowup to my earlier comment recommending the They don't — they're unreachable in this call path. Reading
Probe results on synthetic inputs:
The gate's regex ( Implication: the bullet and code-fence checks in
Either is defensible. The only behavior-relevant change for PR #810 A note on the 411 long-form bullet+code-fence rows I'd seen in |
robotrocketscience
added
the
attn:review
|
Applied the
Local pytest: 4206 passed, 62 skipped, 75 xfailed (was 4202; +4 matches the new boundary tests). All three commits on the branch are SSH-signed. Re-flagged |
|
[claim:review:bagheera:2026-05-14T21:24:42Z] |
robotrocketscience
left a comment
robotrocketscience
left a comment
There was a problem hiding this comment.
Reviewed — substantive logic is sound; CI gap blocks merge
Substantive review (logic + content) — pass:
- The pattern-gate-+-length-floor combination is well-argued and the empirical evidence cited in the prior PR comment (152
:-ending rows ≥80 chars on the live store; 263 long-form bullet and 112 long-form code-fence beliefs preserved) is convincing. The< 80 AND patternshape catches the named noise class without the standalone-floor collateral. - The deeper-check note (bullet + code-fence patterns are unreachable under
extract_sentencesupstream stripping) is correct; keeping them as defense-in-depth is fine. - Intra-turn
DERIVED_FROMdirection (src=later → dst=earlier) matches the inter-turn convention iningest_jsonl. - Anchor-text join (
" | ".join(between)[:ANCHOR_TEXT_MAX_LEN]) and theget_edgededup before insert make re-ingest idempotent on both beliefs and intra-turn edges. - All five commits SSH-signed (
%G? = G). - Operator-ratified scope deviations from the spec letter (pattern-gate vs length-floor;
_ingest_turn_idsvstriple_extractor) are explicitly documented in commit bodies and the PR description. - Together with PR #795's §1, this closes ~70% of the empirically-measured short-reinforced-bloat leak. Issue #818 is content-superseded by commits
a928888a+3f57f029here.
Blocker — CI not green on PR head:
gh run list --commit 3f57f029 --event pull_request returns zero rows; same for a928888a. Only label-docs (push-event, push-paths-filtered) reported on the new HEAD. The full required-check matrix (pytest (3.12), pytest (3.13), CodeQL, Staging Gate / secrets-scan, Staging Gate / pii-scan, Staging Gate / commit-history-audit, Bench Smoke, Replay Soak Gate, Eval Calibration, deadcode) only ran against the prior HEAD a5425f30. The pull_request synchronize event didn't fire on the two 21:07Z commits, so the merge-train will refuse (head SHA ≠ labeled-event SHA).
Removing ready-to-merge so the merge-train doesn't churn on the stale ref. Suggested re-trigger: push an empty commit to the branch (or close+reopen the PR via the GitHub UI) to force the pull_request synchronize event. Once pytest + CodeQL + Staging Gate green on 3f57f029, re-add ready-to-merge.
robotrocketscience
removed
the
ready-to-merge
|
[claim:review:feynman:2026-05-14T21:26:39Z] |
|
[release:review:bagheera:2026-05-14T21:26:43Z] |
|
[release:review:feynman:2026-05-14T21:26:43Z] |
|
[claim:review:oppenheimer:2026-05-14T21:27:58Z] |
robotrocketscience
left a comment
robotrocketscience
left a comment
There was a problem hiding this comment.
Review pass. Substantively sound; the scope deviations (pattern-gate over length-floor, sentence-level over triple-extractor) are well-defended in the PR body and on the _looks_like_subfloor_noise docstring. Atomic commit shape is clean. CHANGELOG entry under [Unreleased] / Fixed is present (commit a5425f3) and matches the deviations.
Two things to address before this lands, plus one question.
Blocking: rebase needed
mergeStateStatus: DIRTY. The #820 refactor (demote-path removal) merged at 21:24:44Z; the touch surface there is small but the existing CI history on this PR predates it, so full CI hasn't run on a current-main base — gh pr checks 810 shows only label, Sourcery review, CodeRabbit. Rebase onto github/main (HEAD fed650ca), push, and re-run the merge-train label cycle so pytest (3.12/3.13), calibration, pattern-scan, history-scan, etc. land green against current main.
Question: derived_belief_ids semantics — is the new ids[0]-only resolution intended?
The diff at src/aelfrice/ingest.py collapses the old "iterate derived_belief_ids and accumulate every new id" loop into "take ids[0] only, then accumulate that one." See:
Before (github/main):
for log_id in log_ids:
entry = store.get_ingest_log_entry(log_id)
if entry is None:
continue
ids = entry.get("derived_belief_ids") or []
if not isinstance(ids, list):
continue
for bid in ids:
if (isinstance(bid, str) and bid not in ids_before
and bid not in seen):
seen.add(bid)
inserted.append(bid)After (this PR):
for log_id in log_ids:
entry = store.get_ingest_log_entry(log_id)
bid: str | None = None
if entry is not None:
ids = entry.get("derived_belief_ids") or []
if isinstance(ids, list) and ids:
head = ids[0]
if isinstance(head, str):
bid = head
log_belief_ids.append(bid)
if bid is not None and bid not in ids_before and bid not in seen:
seen.add(bid)
inserted.append(bid)The doc-string on _ingest_turn_ids says "the per-sentence derived belief id (in input order, with duplicates dropped)" — singular — so reading only ids[0] is consistent with the contract as written. But the underlying schema column derived_belief_ids is a JSON array, and at least two test fixtures encode it as multi-element (tests/test_ingest_log.py:140 writes ["b1", "b2"], :227 writes ["b-1", "b-2"]). Those are schema round-trip tests, not _ingest_turn_ids exercisers, so they don't directly catch a regression — but they document that the storage layer supports multi-id, and the previous _ingest_turn_ids loop honoured that.
Two questions:
-
Does
run_workerever emit a log row with multiplederived_belief_ids(e.g., one sentence producing multiple beliefs via the derivation pipeline)? If yes, this PR silently drops all but the first from theinsertedreturn — which feeds the publicingest_turnreturn value. -
If the contract really is per-sentence-singular, the ingest_log schema is over-typed and could be tightened in a follow-up (separate issue) — but for this PR, an explicit assertion or test case ("multi-id derivation rows are unreachable from
_ingest_turn_ids" / "run_workeris single-id per sentence") would lock the assumption in.
If the answer to (1) is "no, never," a one-line comment on the bid: str | None = None resolution would prevent the next reader from asking the same question.
Minor nits (non-blocking)
_SUBFLOOR_BULLET_PREFIX = re.compile(r"^[-*+]\s")— fine for the listed three bullet markers. Numbered lists (1.,2.) and quote markers (>) are not covered; if those classes ever surface in the corpus-bloat data, follow up.- The acknowledged FP surface (
"He said:","Note:") plus the 80-char scoping is exactly the right trade-off given empirical data. Worth a sentence indocs/feature-ingest-speaker-gate.md§ 3 (the spec doc) saying the implementation realized the spec via pattern+length-cap rather than literal length-floor, with a pointer to the helper docstring — so the next reader of the spec doesn't re-derive the divergence from scratch. Follow-up issue is fine if it crowds this PR.
Net
LGTM after rebase + a clarifying note (or confirmation) on the ids[0] resolution. Holding the formal approve until I see the rebased branch + the multi-id answer.
|
[release:review:oppenheimer:2026-05-14T21:29:50Z] |
…demotion (#809) Adds a pattern-based subfloor gate at the sentence-level ingest path (`_ingest_turn_ids`). Sentences matching `_looks_like_subfloor_noise` do not become freestanding belief rows. When a matched sentence sits between two full-length-belief sentences in the same turn, it attaches as `anchor_text` on an intra-turn DERIVED_FROM edge between the surrounding beliefs; unanchored matches are silently dropped. Pattern set (defense-in-depth across the three noise classes named in the lab campaign): * Code-fence boundaries (```bash, ```) * Header stubs ending with `:` (Acceptance criteria:) * Markdown bullet stubs (`- foo`, `* bar`, `+ baz`) Code-fence and bullet patterns are already largely handled upstream by `extract_sentences` (paired-fence wholesale strip, line-leading list-marker strip). The gate is a backstop for edge cases that survive those strips. The header-ending-in-`:` pattern is NOT handled upstream and is the load-bearing pattern in the normal pipeline. Closes 19% of the short-reinforced-bloat leak documented in `retrieval-corpus-bloat` R0/R2 (header-stub class). Companion to "§1 speaker-attribution gate" shipped under #795 (51% of the same leak). Pattern-gate rather than length-floor per operator-ratified scope for #809: a strict length floor (spec literal: 80 chars) drops legitimate short factual claims ("The config file lives at /etc.") alongside the noise, breaking the conservative ingest contract the existing test suite encodes. Pattern-matching closes only the named noise classes; short legit claims survive. Acknowledged false-positive risk: "ends with `:`" can fire on real prose ("He said:", "The reasons are these:"). The lab campaign named this pattern explicitly; trade-off accepted at empirical scope. Re-measure if production data surfaces non-trivial miss rate. Architectural deviation from spec letter: spec describes the gate on "triple subject/object slots", but the noun-phrase-based `triple_extractor` produces slots typically far below any length floor. The observable leak is sentence-level (`_ingest_turn_ids` via `extract_sentences`), so the gate lives here. "Edge-anchor demotion" maps to intra-turn DERIVED_FROM edges between consecutive full-length sentences in the same turn — the natural codebase analogue of the spec's "surrounding full-length beliefs". Refs: - docs/feature-ingest-speaker-gate.md § 3 - PR #795 (§1 speaker-attribution gate, shipped 2026-05-14)
…anchor demotion
16 tests covering:
* `_looks_like_subfloor_noise` unit checks across all three pattern
classes (header, codefence, bullet); legit-claim non-firing;
whitespace strip; empty input.
* End-to-end filter via `_ingest_turn_ids`: header alone, all-
subfloor turn, unanchored-at-start, unanchored-at-end (the
silently-dropped paths).
* End-to-end demotion: header between two full sentences -> edge
anchor_text; multiple subfloor clauses joined; three-full-sentence
chain with subfloor between each consecutive pair.
* Negative-control regressions: legit short claim still ingests;
consecutive full sentences produce no spurious intra-turn edge;
re-ingest is idempotent on beliefs AND on intra-turn edges.
Test fixtures separate header stubs by double-newline so
`extract_sentences` emits them as standalone sentences (`:` is not
a sentence boundary in the splitter; without the newline a "header:"
merges with following prose and never reaches the gate).
The three subfloor noise patterns (`:`-suffix, code-fence prefix,
bullet prefix) are designed to catch short fragment markers that
carry no semantic claim. Apply that intent explicitly with a
length cap at the gate's top.
Empirical motivation:
- An audit of the full belief corpus shows 152 sentences ending in
`:` are >= 80 chars; these are real prose statements ("If you
look at the way the rebuilder picks beliefs, the order is always
the same:" — 81 chars) distributed across 38 sessions and
retrieved at corpus baseline rate. Without the length scope they
would all be dropped from ingest.
- 263 bullet-prefix beliefs and 112 code-fence-prefix beliefs are
long-form multi-sentence content (median ~250 chars for bullet,
~200 for code-fence) that happen to start with `- ` or ```. The
length scope preserves them.
- A standalone 80-char length floor (without the pattern check)
drops short legit claims like "The default port is 8080." The
combination (pattern AND < 80) catches the load-bearing noise
class while preserving both short legit claims and long-form
prose.
The 80-char constant matches the spec literal in
docs/feature-ingest-speaker-gate.md §3.
Updates the helper's module docstring + body docstring to describe
the length-scope contract. No behavior change for inputs <80 chars
that match the patterns; long-form pattern-matching inputs (>=80)
now correctly pass through ingest instead of being dropped.
Four new tests pin the length-floor behavior added in the preceding commit: - long-form prose ending in `:` (81 chars) does NOT trigger the gate - long-form bullet paragraph (>= 80 chars) does NOT trigger the gate - short header stubs (canonical 'Acceptance criteria:', '```bash', '- run tests') still trigger - boundary: 79-char :-suffix drops, 80-char :-suffix survives Plus a CHANGELOG note documenting the length-scope on the gate.
3f57f02 to
4eb7df4
Compare
|
Rebased onto current Answers to your (1) "Does (2) "Should the contract be documented?" Agree. Adding an inline comment on the The multi-id test fixtures you found ( Nits acknowledged:
Awaiting full CI on the new SHA. |
|
[claim:review:bagheera:2026-05-14T21:36:09Z] |
|
[claim:review:clarke:2026-05-14T21:37:18Z] |
|
[release:review:clarke:2026-05-14T21:37:22Z] |
robotrocketscience
left a comment
robotrocketscience
left a comment
There was a problem hiding this comment.
Independent review — rebase landed; multi-id audit answers owner's question
Picking this up after the prior owner-review pass (21:26Z + 21:29Z) flagged two blockers — rebase + the derived_belief_ids collapse semantics. Both are now addressable.
1. Rebase status — done
Branch tip 4eb7df4f is now based directly on fed650ca (current github/main HEAD, post-#820). git log --oneline github/feat/issue-809-svo-minlen-floor shows the five PR commits sitting directly on fed650ca with no merge commit. The owner's prior review was against 3f57f029 (the pre-rebase tip); the rebase happened between then and now.
CI re-fired on the new HEAD and is green on the staging-gate matrix (secrets-scan, pattern-scan, history-scan, release-docs-check, commit-msg-prefix, pr-title-prefix, pr-body-issue-link), deptry, vulture, bench-smoke, consecutive-green ≥ 7d, typos, CodeQL (python + actions), size-check. pytest (3.12) and pytest (3.13) are still in-flight from runtime 25886927066; merge-train is correctly SKIPPED while ready-to-merge is off.
2. derived_belief_ids multi-id audit — production is single-id by construction
The owner asked whether run_worker ever emits a log row with multiple derived_belief_ids, since the ids[0]-only collapse silently drops the tail if so. Grepping every production writer of that column in src/:
derivation_worker.py:257—store.update_ingest_derived_ids(log_id, derived_belief_ids=[])(orphan / no-derive case, empty array).derivation_worker.py:316-318—store.update_ingest_derived_ids(log_id, derived_belief_ids=[actual_id])(the canonical "successfully derived one belief" case — explicit single-element list).store.py:1124-1131— legacy-log-synth migration (#263), inserts one row per orphan belief withderived_belief_ids = [belief.id](single-element).
No production code path writes a multi-element derived_belief_ids array. The contract is "0 or 1 derived belief ids per ingest_log row." The previous for bid in ids loop in _ingest_turn_ids was defensive against a case that never arises in run_worker output. The schema column is over-typed (TEXT-encoded JSON array), but the empirical contract is singular — consistent with the _ingest_turn_ids docstring's "per-sentence derived belief id (in input order, with duplicates dropped)."
So the ids[0]-only collapse is safe at current head. The intra-turn edge wiring requires a positional single-id per sentence; the loop refactor is the natural shape for that.
3. Lock-the-assumption sub-ask is still unaddressed
The owner's #2 sub-ask — an explicit assertion or test locking the "multi-id rows are unreachable from _ingest_turn_ids" assumption — is not in the current diff. tests/test_ingest_subfloor_noise.py does not grep for derived_belief_ids. The over-typed schema means a future contributor could legitimately add a multi-id producer (e.g., a sentence that fans out to N beliefs through derivation) without realizing _ingest_turn_ids will silently swallow N-1.
Minimum: a one-line inline comment on head = ids[0] documenting "by contract derivation_worker emits at most one belief id per log row; if that contract ever broadens, this collapse drops the tail and inserted becomes incomplete" would be enough. Author's call whether to address inline here or open a follow-up. Not blocking on my read.
4. Logic spot-check on the edge-wiring loop
Read through for i in range(1, len(log_belief_ids)) in ingest.py. The prior_bid == curr_bid short-circuit correctly suppresses self-edges when two adjacent sentences canonicalize to the same belief id (e.g., duplicate prose in one turn). One consequence: a sub-floor clause sandwiched between two occurrences of the same belief is silently dropped rather than attached as anchor_text — natural reading since there's no relational edge to wire, but worth noting if anyone later expects subfloor demotion to be a strict invariant. Tests cover the typical case (full→subfloor→full with distinct beliefs); they do not cover the same-belief sandwich. Not blocking.
5. Discretion
Diff vs github/main is clean — no public-boundary issues.
Net
Rebase blocker is cleared by the new HEAD; the multi-id question is answered by the producer audit (single-id by construction in all three writers). Awaiting pytest (3.12/3.13) green on 4eb7df4f and an owner formal approve on the multi-id finding; once both, re-add ready-to-merge and the merge-train should ship.
|
[release:review:bagheera:2026-05-14T21:40:00Z] |
robotrocketscience
added
the
ready-to-merge
|
merge-train: merged 4eb7df4 → |
github-actions
Bot
removed
the
ready-to-merge
1 participant
Summary
Closes #809. Adds a pattern-based subfloor-noise gate at the sentence-level ingest path (
_ingest_turn_ids). Sentences matching the named noise patterns from theretrieval-corpus-bloatlab campaign — code-fence boundaries, header stubs ending with:, markdown bullet stubs — no longer become freestanding belief rows. When a matched sentence sits between two full-length-belief sentences within the same turn, it attaches asanchor_texton a new intra-turnDERIVED_FROMedge between the surrounding beliefs (src=later, dst=earlier, matching the inter-turn convention iningest_jsonl). Unanchored matches are silently dropped.This closes 19% of the short-reinforced-bloat leak documented in
retrieval-corpus-bloatR0/R2. Companion to PR #795's §1 speaker-attribution gate (51% of the same leak). Together they close ~70% of the empirically-measured input-side leak.Operator-ratified scope deviations
Two deviations from the spec letter in
docs/feature-ingest-speaker-gate.md§ 3, both flagged + approved during implementation:1. Pattern-gate, not length-floor
Spec literal:
MIN_BELIEF_CONTENT_CHARS: Final[int] = 80. Applied as a strict floor, this dropped 26 existing ingest-test fixtures (legitimate short factual claims:"The configuration file lives at /etc/aelfrice/conf.","The default port is 8080."). The lab campaign empirically calibrated 80 chars against the α+β ≥ 10 reinforced stratum — content already bumped repeatedly — not against all incoming sentences. A length-floor read of the spec over-applies that finding.Operator-ratified scope is pattern-based: only the three pattern classes the lab named (code-fence,
:-header, bullet stub) trigger the gate. Short legit claims pass. The acknowledged false-positive surface is"He said:"-style real prose ending in:; the lab named this pattern explicitly, trade-off accepted at empirical scope.2. Gate lives in
ingest._ingest_turn_ids, nottriple_extractorSpec letter: gate the "triple-extraction emission path" at "subject or object slots". But
triple_extractor.pybuilds Triples from noun phrases (_NP= 1-5 tokens, ~10-40 chars typical) — applying ANY length floor or content-pattern check there would over-fire to near-zero.The observable leak the campaign measured comes from the sentence-level path (
_ingest_turn_idsviaextract_sentences). That's where the gate lives. "Edge-anchor demotion" maps to intra-turnDERIVED_FROMedges between consecutive full-length sentences in the same turn — the natural codebase analogue of the spec's "surrounding full-length beliefs". Mirrors PR #795's deviation for §1 (moved gate fromtranscript_loggertoingest_jsonlfor the same architectural reason).Files changed
src/aelfrice/ingest.py— new_looks_like_subfloor_noise(sentence)helper checks for code-fence prefix /:-ending / markdown bullet marker._ingest_turn_idspartitions sentences into full-length belief candidates and pending-demotion sub-floor clauses, then wires intra-turn DERIVED_FROM edges withanchor_text = " | ".join(between)[:ANCHOR_TEXT_MAX_LEN]. Edge insert is deduped viaget_edge(...)check; re-ingest is idempotent on both beliefs AND on intra-turn edges.tests/test_ingest_subfloor_noise.py— 16 new tests across three layers:_looks_like_subfloor_noise(all three patterns; legit-claim non-firing; whitespace strip; empty input)._ingest_turn_ids(header-alone, all-subfloor-turn, unanchored-at-start, unanchored-at-end — the silently-dropped paths).CHANGELOG.md— Unreleased / Fixed entry.Pattern coverage relative to
extract_sentencesextract_sentencesalready handles two of the three pattern classes upstream::— NOT stripped; reach the gate as-is when newline-separated.The gate retains all three pattern checks as defense-in-depth for malformed / mid-line cases that survive
extract_sentences(unclosed code-fences, bullets that appear after.in a single line). Unit tests cover all three. End-to-end integration tests cover the:pattern (the load-bearing one in the normal pipeline). Test fixtures separate:-headers by\n\nbecause:is not a sentence boundary inextract_sentences.Test plan
uv run pytest -x -q→ 4202 passed, 62 skipped, 75 xfailed. No existing ingest test broken.github/main— clean (verified byaelf-pr-open.shpre-flight).%G? = G).Out of scope
feedback_historyrouting — held per the lab-sidealpha-gain-third-pathcampaign verdict ("not load-bearing for the top-stratum leak"). If/when re-prioritized, a separate follow-up.Refs
docs/feature-ingest-speaker-gate.md§ 3 — spec.retrieval-corpus-bloatR0/R2 — lab campaign empirical basis (2026-05-11, lab-side).alpha-gain-third-path— verdict that holds §2 (lab-side).Closes #809
Summary by CodeRabbit
New Features
Tests