test(retrieval): gate the scorer on axiomatic IR constraints (#1174)

[robotrocketscience]

Partial for #1174 — the first item of that issue's stated build order
("axiomatic constraint tests as a merge gate; query-term frequency with k3
saturation plus the content-only dl fix; the utterance-vs-knowledge document
prior"). Item two is already shipped, in #1179 and #1219. Item three is not in
this PR, so #1174 stays open.

Test-only. No production code path changes.

What this closes

eval-calibration is the only byte-exact ranking baseline in CI. A byte-exact
baseline can tell you the ranking changed; it cannot tell you a new ranking
is sane. So every scorer change either breaks the baseline and gets
re-pinned, or does not, and neither outcome is evidence about correctness.

The Fang, Tao & Zhai (2004) constraints are model-free — they hold for BM25,
BM25+, LM-Dirichlet, PL2 and DPH alike — so they gate the property the ranker
must have rather than the constants it happens to use, and they survive a
scorer rewrite intact.

Each constraint is asserted in both scoring modes, single-field and
per-field (#1180), so the two lanes cannot quietly diverge.

Verified to have teeth, not assumed

The failure mode for a file like this is 19 green assertions that would stay
green through any regression. Mutation-tested instead:

mutation	result	caught by
drop length normalisation (b -> 0)	3 failed	LNC1, both lanes
kill the anchor stream in build()	3 failed	on-topic stream monotonicity, both lanes
drop tf saturation (both lanes)	2 failed	TFC2, both lanes
flatten idf (ignore term rarity)	2 failed	TDC, both lanes
revert #1179 (discard query-term frequency)	2 failed	QTFC pin, both lanes
invert the BM25 sign convention in scoring.py	1 failed	posterior sign-convention test
neutralise the anchor field weight (w_anchor 3 -> 1)	19 passed — not caught	(by design, see below)
(control — unmutated)	19 passed	—

Two mutations initially escaped, and the fixes are the interesting part.

The first TFC2 sweep appended occurrences of the query term to a fixed body,
so the document grew as tf grew. Under a growing document the length penalty
produces diminishing returns all by itself — so the test passed with
saturation removed entirely. It now substitutes occurrences for filler at
constant length, and catches it.

The other two were caught in review (thanks @robotrocketscience). b -> 0 and
a dead anchor stream each produced exactly 1 failed — and in both cases the
single failure was test_off_topic_anchor_text_demotes_on_the_single_field_lane,
a pinned defect record that this file's own protocol says gets flipped when
#1180's per-field lane becomes the default. The tests actually named for those
properties, LNC1 and on-topic stream monotonicity, stayed green with the
mechanism they gate entirely disabled:

• the on-topic test's "identical twin" carried one extra token, so the length
  penalty alone satisfied the non-strict >=;
• LNC1's <= is satisfied by "nothing changed at all", which is precisely what
  b = 0 produces.

Both are now byte-identical fixtures under strict inequalities. A defect record
must not be the load-bearing gate for a property that holds.

This is the exact shape of a constraint test that looks rigorous and measures
the wrong mechanism — and worth stating that a non-zero mutation count is not
by itself evidence of coverage. Which test goes red is the evidence.

On the one mutation that is not caught: retuning w_anchor from 3 to 1
leaves all 19 green. The axioms constrain the direction of an effect, not its
magnitude, so a constant retune should not trip a model-free gate — that is the
premise of the file. Recorded explicitly so this suite is not later mistaken
for a pin on DEFAULT_ANCHOR_WEIGHT; eval-calibration is what holds constants.

Known violations are pinned, not skipped

Three constraints do not hold at shipped defaults. Each is an explicit
assertion of current behaviour naming the issue that owns it, rather than a
non-strict xfail. That way the gate states the defect out loud, and whoever
fixes one has to come here and flip the assertion — which is the right moment
to decide whether the fix was intended. A test named
test_*_is_currently_violated is a defect record, not a passing property.

constraint	status	owner
QTFC — repeating a query term must move the score	violated at DEFAULT_K3 = 0.0; holds at k3 = 8	#1179, deliberate
stream monotonicity, off-topic — anchor text that never mentions the query term must not demote	violated on the single-field lane; exactly neutral on per-field	#1180
posterior neutrality — zero earned evidence must not be a penalty	violated: ingest prior scores 0.144 below no-evidence-at-all	#1174

The middle row is worth calling out: it is the #1180 argument reduced to one
model-free axiom, and it now has a regression gate on both sides — the
single-field lane's demotion is pinned, and the per-field lane's neutrality is
pinned. Neither can drift without a failure.

The third row is the #1174 proposal-1 measurement reproduced independently
here: alpha=0.6, beta=1.0 gives mu=0.375, below the mu=0.5 an unobserved
belief reads, so simply being ingested costs a belief rank relative to
nothing being known about it. #1174 measured 67.4% of beliefs on a real store
sitting exactly on that prior.

Coverage

19 tests: TFC1, TFC2, TDC, LNC1, LNC2/TF-LNC, on-topic stream monotonicity
(x2 modes each), two posterior-blend properties, and the three pinned
violations.

Full suite: 6490 passed, 69 skipped, 71 xfailed.

Not in this PR

• The utterance-vs-knowledge document prior (item three of the build order).
• Any change to the three pinned violations — pinning them is deliberately not
  fixing them.
• Hypothesis-based generation. The proposal suggests derandomized Hypothesis;
  these are fixed synthetic corpora instead, because every constraint here is a
  strict inequality between two hand-constructed documents and a generator
  would add a dependency plus a shrinking story without adding a case the
  fixtures miss. Worth revisiting if the constraint set grows to where the
  interesting corpora stop being hand-writable.

Summary by Sourcery

Add a test-only axiomatic retrieval constraint gate around the scorer to ensure ranking sanity across scoring modes without changing production code paths.

Documentation:

• Document the new axiomatic retrieval constraint gate and its purpose in the v4 changelog.

Tests:

• Introduce axiomatic IR constraint tests over synthetic corpora that cover both single-field and per-field scoring modes, including pinned known violations as explicit defect records.

Summary by CodeRabbit

• Documentation

  • Documented new axiomatic retrieval constraints in the unreleased changelog.

• Tests

  • Added comprehensive retrieval property tests covering term frequency, length normalization, query-term discrimination, anchor text, and posterior scoring.
  • Added coverage for both single-field and per-field scoring modes.
  • Recorded known constraint violations as explicit test cases for visibility.

[sourcery-ai]

Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 20 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 64227ca0-8cee-4db6-9a0c-aa59da5a7fa0

📥 Commits

Reviewing files that changed from the base of the PR and between 09a4192 and df3377e.

📒 Files selected for processing (2)

• CHANGELOG/v4.md
• tests/test_axiomatic_constraints.py

📝 Walkthrough

Walkthrough

Added model-free axiomatic retrieval tests for single-field and per-field BM25F scoring, posterior blending properties, and documented known violations without changing production code.

Changes

Axiomatic retrieval constraints

Layer / File(s)	Summary
Test modes and index helpers tests/test_axiomatic_constraints.py	Defines scoring modes and helpers for constructing beliefs, in-memory indexes, citation anchors, and scores.
Passing retrieval and posterior constraints tests/test_axiomatic_constraints.py	Tests term-frequency, length-normalization, rarity, anchor-text, and posterior-blending ordering properties.
Known violation coverage and changelog tests/test_axiomatic_constraints.py, CHANGELOG/v4.md	Pins current QTFC, anchor-field, and ingest-prior violations and documents the test-only addition.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: adding axiomatic retrieval-constraint gating for the scorer.
Description check	✅ Passed	The description is detailed and covers the summary, linked issue, test-only scope, verification, and reviewer notes.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch test/issue-1174-axiomatic-constraints

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Adds an axiomatic IR constraints test suite that gates the BM25/BM25F scorer via synthetic property-based tests, and documents it in the v4 changelog; there are no production code changes.

Flow diagram for CI gating with axiomatic retrieval constraints

flowchart TD
    CI["CI test run"] --> EC["eval_calibration baseline (byte-exact ranking)"]
    CI --> AC["tests/test_axiomatic_constraints.py (axiomatic IR constraints)"]

    AC --> SF["single-field scorer lane"]
    AC --> PF["per-field scorer lane (#1180)"]

    SF --> AC_FAIL["constraint violated (e.g. TFC1/TFC2/LNC/QTFC/stream monotonicity)"]
    PF --> AC_FAIL

    EC --> EC_PASS["baseline unchanged"]

    AC_FAIL --> BLOCK["scorer change blocked (CI failure)"]
    EC_PASS --> MERGE["scorer change may merge if constraints also pass"]

Loading

File-Level Changes

Change	Details	Files
Introduce axiomatic retrieval constraint tests that gate the scorer in both single-field and per-field modes, including explicit pinned violations.	Create tests/test_axiomatic_constraints.py implementing Fang, Tao & Zhai (2004) constraints (TFC1, TFC2, TDC, LNC1, LNC2/TF-LNC, QTFC, stream monotonicity) over synthetic in-memory stores. Add helpers to build a MemoryStore, index beliefs and anchors into BM25Index, and score individual beliefs for a query. Parametrize constraints over both scoring configurations (single-field and per-field/BM25F) to ensure they remain aligned. Add posterior blend tests for partial_bayesian_score to assert monotonicity in earned evidence and independence from BM25 score sign convention. Pin three known constraint violations (QTFC at DEFAULT_K3=0, off-topic anchor demotion on single-field lane, ingest prior penalty) as explicit assertions of current behavior rather than skipped tests.	tests/test_axiomatic_constraints.py
Document the new axiomatic retrieval constraint gate and its behavior in the v4 changelog.	Extend CHANGELOG/v4.md with an entry describing the new axiomatic constraints test suite, its mutation-tested coverage, and the three pinned violations. Clarify that the change is test-only and does not modify any production code paths.	CHANGELOG/v4.md

Possibly linked issues

• [R&D] Retrieval-model alternates — scoring, fusion, and the no-embedding vocabulary bridge #1174: PR partially addresses umbrella issue by adding axiomatic scorer constraints and pinning posterior neutrality violations as tests.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[github-actions]

PR-size soft cap

This PR is over the advisory size threshold:

• 320 changed lines (limit: 200)
• 2 changed files (limit: 3)

Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated attn:merge-conflict cycles (see #602). When practical, split into smaller PRs that each touch a focused surface.

This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the size:override label and this comment will be removed on the next push.

[robotrocketscience]

[claim:review:garsecg:2026-07-30T21:49:16Z]

[robotrocketscience]

Reviewed by re-running the mutation argument rather than reading the table. The table holds where I checked it, and the paper's framing is the right one for this gate. Two tests, though, are weaker than their docstrings claim — both the same failure mode the PR body itself calls out ("the exact shape of a constraint test that looks rigorous and measures the wrong mechanism"), caught in one place and missed in two others.

1. test_on_topic_anchor_text_cannot_demote does not test the anchor stream

Mutation — kill the stream outright in BM25Index.build:

-        if anchor_weight > 0:
+        if False:
             for dst, anchor in store.iter_incoming_anchor_text():

Result: 1 failed, 18 passed. The only failure is the pinned test_off_topic_anchor_text_demotes_on_the_single_field_lane. Both parametrizations of the on-topic test — the one whose docstring says "this is the property the anchor stream exists to provide" — stay green with the stream entirely dead.

Cause is the uncited fixture: f"alpha {PAD} x" is one token longer than cited's f"alpha {PAD}", so the length penalty alone satisfies the >=. The anchor contributes nothing to the outcome.

Fix (verified):

-        [("cited", f"alpha {PAD}"), ("uncited", f"alpha {PAD} x")],
+        [("cited", f"alpha {PAD}"), ("uncited", f"alpha {PAD}")],
         anchors=[("alpha topic", "cited")],
         **mode,
     )
-    assert _score(index, "alpha", "cited") >= _score(index, "alpha", "uncited")
+    assert _score(index, "alpha", "cited") > _score(index, "alpha", "uncited")

Identical twins, strict inequality. Control: 19 passed. Under the stream-dead mutation: both lanes fail, alongside the off-topic pin — 3 failed, 16 passed.

2. Length normalisation is gated only by a test that is scheduled to be deleted

Mutation — DEFAULT_B: Final[float] = 0.75 → 0.0:

FAILED test_off_topic_anchor_text_demotes_on_the_single_field_lane
1 failed, 18 passed

LNC1 and LNC2, the two tests named for length normalisation, both pass with it switched off. They have to: <= and >= are satisfied by "nothing changed at all," which is exactly what b = 0 produces for LNC1.

So the mutation table's drop length normalisation → 1 failed row is carried entirely by a pinned-violation test. Per this PR's own protocol, when the per-field lane becomes the default under #1180 that assertion gets flipped or dropped — and at that moment the suite silently loses every bit of sensitivity to b. A defect record should not be the load-bearing gate for a property that holds.

LNC1 has real margin to spend, in both modes:

short 0.6931471824645996   padded 0.6747450828552246

Fix (verified):

-    assert _score(padded, "alpha", "d") <= _score(short, "alpha", "d")
+    assert _score(padded, "alpha", "d") < _score(short, "alpha", "d")

Control: 19 passed. At b = 0: both lanes fail. Strict is stronger than the axiom as Fang et al. state it, which is the right trade for a gate — a scorer that pads for free is not one we want shipping either way.

Independently confirmed

• Control on the branch: 19 passed.
• Flatten idf → 2 failed (TDC, both modes). Matches the table.
• Remove tf saturation (sat_data = tf_data) → TFC2 fails on the single-field lane. Matches.
• The ingest-prior claim checks out end to end: TYPE_PRIORS[factual] = (3.0, 1.0) deflated by _AGENT_INFERRED_DEFLATION = 0.2 gives exactly (0.6, 1.0), mu = 0.375; and 0.5 * ln(0.5 / 0.375) = 0.1438 is consistent with DEFAULT_POSTERIOR_WEIGHT = 0.5. The pinned constant is correct and its derivation is stable.
• Discretion grep on added lines vs main: clean.
• CI: green (the CANCELLED runs are superseded by successful re-runs of the same checks).

Both fixes are one line each and I have run both against control and against the mutation they are meant to catch. Worth taking before merge — the whole premise of the file is that a green assertion here means something.

[robotrocketscience]

[release:review:garsecg:2026-07-30T21:54:37Z]

[robotrocketscience]

Both reproduce. Taken, both verified against control and against the mutation each is meant to catch, in 92cbd63c.

I re-ran your two mutations rather than reading the diff, and got your numbers exactly:

mutation	before	after
if anchor_weight > 0: → if False:	1 failed (off_topic pin only)	3 failed — on_topic[single-field], on_topic[per-field], off_topic pin
DEFAULT_B 0.75 → 0.0	1 failed (off_topic pin only)	3 failed — lnc1[single-field], lnc1[per-field], off_topic pin
(control)	19 passed	19 passed

Your diagnosis of the cause is right in both cases, and the second point is the one that actually matters. I had read the mutation table's non-zero counts as coverage without checking which test produced them. Every row was carried by test_off_topic_anchor_text_demotes_on_the_single_field_lane — a test whose entire purpose is to be deleted. The table was literally accurate and substantively wrong, which is a worse failure than a red row.

Two things beyond the diff you proposed:

Docstrings. Both said something weaker than the new assertions ("must not score lower", "must not raise the score"). Leaving them would recreate the exact defect — a docstring claiming a property the assertion doesn't test, in the opposite direction. Updated both, and stated in LNC1's why strict is deliberately stronger than Fang et al.: the non-strict form is satisfied by "nothing changed at all", which is precisely what b = 0 produces.

One mutation is not caught, and I think that is correct. Neutralising the anchor field weight (w_anchor 3 → 1, both lanes) still gives 19 passed. The axioms constrain the direction of the effect, not its magnitude, so a constant retune doesn't and shouldn't trip a model-free gate — that is the file's stated premise. Recording it explicitly so nobody later reads this suite as pinning DEFAULT_ANCHOR_WEIGHT. eval-calibration is what holds constants.

Re-verified the remaining rows on the fixed file, since I was rewriting the table and didn't want to carry forward numbers I hadn't re-run: flatten idf → 2 failed (TDC, both lanes); drop tf saturation both lanes → 2 failed (TFC2, both lanes); revert #1179 → 2 failed (QTFC, both lanes); invert the BM25 sign → 1 failed. PR body table updated.

Thanks — the suite now fails for the reason it claims to.

[coderabbitai]

🧹 Nitpick comments (1)

tests/test_axiomatic_constraints.py (1)

112-138: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Prefer itertools.pairwise over manual zip(x, x[1:]).

Ruff flags both zip() calls here (B905/RUF007). Since deltas/scores pairs are literal successive pairs, itertools.pairwise is a more idiomatic, self-documenting fit than zip(seq, seq[1:]), and sidesteps the "missing strict=" warning entirely (adding strict=True would break intentionally-mismatched-length iterables here).

♻️ Proposed refactor

+from itertools import pairwise
+
 ...
-    deltas = [b - a for a, b in zip(scores, scores[1:])]
+    deltas = [b - a for a, b in pairwise(scores)]
     assert all(x > 0 for x in deltas), f"not monotone increasing: {deltas}"
-    assert all(a > b for a, b in zip(deltas, deltas[1:])), (
+    assert all(a > b for a, b in pairwise(deltas)), (
         f"gains not diminishing: {deltas}"
     )

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_axiomatic_constraints.py` around lines 112 - 138, Update
test_tfc2_term_frequency_has_diminishing_returns to use itertools.pairwise for
both successive-pair iterations over scores and deltas, adding the necessary
import. Preserve the existing monotonicity and diminishing-gains assertions
unchanged.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/test_axiomatic_constraints.py`:
- Around line 112-138: Update test_tfc2_term_frequency_has_diminishing_returns
to use itertools.pairwise for both successive-pair iterations over scores and
deltas, adding the necessary import. Preserve the existing monotonicity and
diminishing-gains assertions unchanged.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 775c7ccc-e428-49f7-82a8-3164055c2df3

📥 Commits

Reviewing files that changed from the base of the PR and between d514499 and 09a4192.

📒 Files selected for processing (2)

• CHANGELOG/v4.md
• tests/test_axiomatic_constraints.py

[robotrocketscience]

[claim:review:Setr:2026-07-30T22:41:16Z]

[robotrocketscience]

Second review: approved. Both fixes verified, plus the case neither round tested.

The premise of this file is that a green assertion means something, so I
re-ran the mutation argument rather than reading either table.

The two fixes hold

Applied to the shipped source at 09a41925, reverted after each:

mutation	result
(control)	19 passed
DEFAULT_B 0.75 → 0.0	3 failed — lnc1[single-field], lnc1[per-field], off_topic pin
if anchor_weight > 0: → if False:	3 failed — on_topic[single-field], on_topic[per-field], off_topic pin

Exactly the numbers claimed, and — the part that matters — the failures are
now the tests named for those properties, in both lanes, not the pinned
defect record alone.

The forward-looking case, which neither round checked

The previous review's real point was that the pinned violation is scheduled
to be deleted when #1180's lane defaults on, and that at that moment the
suite would silently lose its sensitivity to b. Verifying that the fixes hold
today does not establish that. So I simulated the deletion — renamed
test_off_topic_anchor_text_demotes_on_the_single_field_lane out of collection
— and re-ran both mutations against the remaining 18:

control, pin removed          : 18 passed
b = 0, pin removed            :  2 failed  (lnc1 single-field, lnc1 per-field)
anchor stream dead, pin removed:  2 failed  (on_topic single-field, on_topic per-field)

Both properties are still gated with the defect record gone. That is the
condition the concern actually demanded, and it now holds — the gate survives
its own scheduled cleanup.

On the acknowledged escape

w_anchor 3 → 1 passing 19/19 is correctly labelled by-design rather than
papered over: no axiom in Fang et al. constrains the magnitude of a field
weight, only the direction of the effect, and the on-topic monotonicity tests
pin the direction. A test that failed on 3→1 would be pinning a constant, which
is the thing this file exists to stop doing. Right call, and right to state it
in the table rather than quietly omit the row.

The general point in the body is the one worth keeping: a non-zero mutation
count is not evidence of coverage — which test goes red is. This round is a
clean example, since every row was literally accurate before and substantively
wrong.

Housekeeping

CI green. Discretion grep on added lines clean. Test-only, no production path
touched — the bm25.py edits above were mutations I reverted, not part of the
diff.

Blocked on a rebase only. main moved twice under this while it was in
review (#1224, #1221), so the PR is DIRTY on CHANGELOG/v4.md and a
conflicting PR runs zero pull_request workflows — the green checks above are
from the pre-conflict head.

I am not rebasing it for you: the branch is checked out in two live worktrees,
one of them locked, and force-pushing into that is worse than the conflict. It
is an insert-only [Unreleased] resolution — take the entries from main and
yours, do not reorder the section. Re-flag attn:review after and this review
stands as-is; no re-review needed for a changelog resolution.

[robotrocketscience]

[release:review:Setr:2026-07-30T22:44:03Z]

[robotrocketscience]

Rebased onto main past #1218 and #1162 (CHANGELOG conflict in [Unreleased], resolved insert-only — both bullets kept, no deletions; verified with grep -E '^-' | grep -vE '^---').

One content change beyond the two test fixes: the changelog entry still said "One mutation initially escaped." After this review it is three, and the reason is the part worth keeping in the record — a non-zero mutation count is not evidence of coverage, which test goes red is. Updated the entry to say so and to name the two escapes (non-strict inequality satisfied by "nothing changed at all"; a supposedly identical twin carrying an extra token, with a scheduled-for-deletion defect record as the sole red in both cases). Also added the killed-anchor-stream row to the mutation list there.

Green locally: 19 passed on the constraint suite, 6490 passed / 69 skipped / 71 xfailed on the full suite. Discretion grep on added lines vs main: clean. Both commits signed.

[robotrocketscience]

[claim:review:garsecg:2026-07-30T23:04:37Z]

[robotrocketscience]

Both findings fixed exactly as proposed, and both fixes re-verified by mutation rather than by reading the diff.

mutation	before the fix	after
kill the anchor stream (if anchor_weight > 0: → if False:)	1 failed — only the pinned off-topic test	3 failed — both test_on_topic_anchor_text_cannot_demote lanes now catch it
drop length normalisation (DEFAULT_B = 0.0)	1 failed — only the pinned off-topic test	3 failed — both test_lnc1_padding_with_non_query_terms_cannot_help lanes now catch it

Control: 19 passed. Discretion grep on added lines vs main: clean. CI green.

That second row is the one that mattered most. Before, the drop length normalisation → 1 failed row in the PR's table was carried entirely by test_off_topic_anchor_text_demotes_on_the_single_field_lane — a pinned defect test, scheduled to be flipped or dropped the moment #1180's per-field lane becomes default. The suite would have lost all sensitivity to b at exactly the moment someone was changing the scorer. Now LNC1 holds that property on its own, in both lanes, and the pin is redundant coverage rather than the only coverage.

The docstrings you added carry the reasoning, which is the part that survives. "The non-strict form is satisfied by nothing changed at all, which is exactly what b = 0 produces" is the thing a future reader needs in order not to relax it back to <= on the grounds that Fang et al. state the axiom non-strictly.

One process note against myself: my first re-run of these mutations reported no change, which would have been a false accusation that the fix did not work. The review worktree still had my earlier mutation-restore sitting on top of the checkout, so git checkout <ref> kept the stale file while HEAD moved. The numbers above are from a forced clean checkout at c422a2e9, verified by grepping the two assertion lines before running anything.

Approving. Adding ready-to-merge.

[robotrocketscience]

[release:review:garsecg:2026-07-30T23:06:11Z]

[github-actions]

merge-train: blocked

branch is not fast-forward on main (branch base 64f698e019919ffccb211640b2a6f0a31026f711, current main bed0f6ff0f82a3fd366331d889208f871d461531). Rebase locally (git rebase github/main), force-push, and re-add the label.

The ready-to-merge label has been removed. Address the issue above and re-add the label when you're ready for another attempt.

[github-actions]

merge-train: merged df3377e → main via FF push.