fix(docs): widen the version-claim gate and correct AGENTS.md's coverage floor by jonlwheat2-gif · Pull Request #15966 · cheaper-inference/OmniRoute

jonlwheat2-gif · 2026-10-08T19:05:14Z

Summary

G-06 and G-08 of the #15159 audit — two doc gates reporting green while the docs were wrong.

G-06 — the version validator matched only two literal phrasings

/OmniRoute v(\d+\.\d+\.\d+)/
/Current version:\*{0,2}\s*\*{0,2}(\d+\.\d+\.\d+)/

At most one claim per file was ever seen, so four stale strings shipped through while the gate reported OK:

Location	Text (was)
`README.md:1012`	"reported by OmniRoute 3.8.51"
`README.md:64`	`v3.8.49 \| v3.8.50 \| v3.8.51+`
`README.md:571`	"Recent highlights from v3.8.20 → v3.8.50"
`llm.txt:277`	"## Key Features (v3.8.50)"

All four now say 3.8.52.

Fixing the strings alone is not enough — the next phrasing nobody anticipated slips through again. So the patterns are now anchored to contexts that actually assert the current version: product-named, an explicit Current version: label, a bold range (capturing its target), a single bold version, a version in a heading's parentheses, an inline range target, and a backticked floor.

G-08 — AGENTS.md advertised the wrong coverage floor

It said 60/60/60/60. Accurate in isolation (that is what test:coverage hardcodes), but the blocking ratchet enforces 67.33 / 67.33 / 72.02 / 65.08 — so an agent reading AGENTS.md first would run below the real floor and be surprised by a red quality-gate on correct work. AGENTS.md was not even in the validator's file list.

makeCoverageFloorClaimValidator now compares against floors read from quality-baseline.json, so a future rebaseline moves the docs and the check together instead of letting them drift.

The subtlety that makes widening safe

A version number alone is not a defect. v3.8.49 in a compatibility table, v3.9.0 LTS in the roadmap, and the v3.8.20 end of a "from X → Y" range are all correct and must stay green. My first widened attempt flagged all three — caught by running the gate, not by a unit test. Ranges are matched before their left endpoint is double-counted.

Scope note

llm.txt is an agent-instruction surface (as is AGENTS.md). docs-sync compares each i18n mirror body to root byte-for-byte, so all 66 mirrors are synced in this PR — otherwise check:docs-all fails.

Verification

15 new tests (TDD, watched red first); 69 check-docs-counts tests pass. check:docs-counts and check:docs-all both exit 0; lint clean.

⚠️ base-red inherited: #15306

…age floor G-06 and G-08 of the cheaper-inference#15159 audit. Two doc gates that reported green while the docs were wrong. G-06 - the version-claim validator matched only TWO literal phrasings: /OmniRoute v(\d+\.\d+\.\d+)/ /Current version:\*{0,2}\s*\*{0,2}(\d+\.\d+\.\d+)/ so it saw at most one claim per file, and four stale strings shipped through it while the gate reported OK: README.md "reported by OmniRoute 3.8.51", the compatibility table's "**v3.8.50**", "Recent highlights from v3.8.20 -> v3.8.50", and llm.txt's "## Key Features (v3.8.50)". All four now say 3.8.52. Fixing the strings alone is not enough - the next phrasing nobody anticipated slips through again - so the patterns are anchored to contexts that actually ASSERT the current version: product-named, an explicit "Current version:" label, a bold range (capturing its TARGET), a single bold version, a version in a heading's parentheses, an inline range target, and a backticked floor. The subtlety that makes widening safe: a version NUMBER alone is not a defect. `v3.8.49` in a compatibility table, `v3.9.0 LTS` in the roadmap and the `v3.8.20` end of a "from X -> Y" range are all CORRECT and must stay green. A naive `/\d+\.\d+\.\d+/` flagged all three - caught by running the gate, not by a unit test. Ranges are matched before their left endpoint is double-counted. G-08 - AGENTS.md advertised `60/60/60/60`. Accurate in isolation (that is what test:coverage hardcodes) but the BLOCKING ratchet enforces 67.33 / 67.33 / 72.02 / 65.08, so an agent reading AGENTS.md alone would run below the real floor and be surprised by a red quality-gate on correct work. AGENTS.md was not in the validator's file list at all. makeCoverageFloorClaimValidator now compares against the floors READ FROM quality-baseline.json, so a future rebaseline moves the docs and the check together rather than letting them drift. llm.txt's change propagates to all 66 i18n mirrors (docs-sync compares the mirror body to root byte-for-byte); all 66 synced in this commit. Verified: 15 new tests watched red before the fix; 69 docs-counts tests pass; check:docs-counts and check:docs-all both exit 0; lint clean.

diegosouzapw · 2026-10-10T20:30:32Z

Hi @jonlwheat2-gif — closing this pull request to clear the open queue. The branch fix/15159-w2-g06-g08-doc-version-and-coverage-floor is kept, so the commits stay. Comment here if it should be reopened.

diegosouzapw · 2026-10-10T22:49:42Z

Hi @jonlwheat2-gif — reopening this pull request. The close was a mistake, and it is back in the merge queue. The branch fix/15159-w2-g06-g08-doc-version-and-coverage-floor was kept.

# Conflicts: # README.md

On the current release tip the README compatibility table bolds the last SHIPPED release (`| v3.8.50 | **v3.8.51** | `v3.8.52+` |`) while package.json already carries the in-development 3.8.52, so the widened "single bold version" pattern reddened check:docs-counts on correct content. In a table row the bold cell is a column marker; the row's current-version claim is the backticked floor, which is still checked. Adds a regression test for the tip-shaped row (and keeps the stale-floor and non-table bold cases failing), and prints the coverage floors as numbers instead of [object Object].

…ocs/slim-agents-under-40k Carry cheaper-inference#15966's AGENTS.md coverage wording (67.33 / 67.33 / 72.02 / 65.08, enforced by the blocking ratchet in config/quality/quality-baseline.json; test:coverage itself hardcodes 60) into both the Quick Start comment and the Testing list of the slimmed AGENTS.md, so the new coverage-floor gate in check-docs-counts-sync passes. Trim a few duplicated phrases to stay under 40k (39,848 bytes).

@wahidsadik371-coder

…2743) Merged via /merge-batch (2026-10-19). Maintainer merge of the current base on top: the slimming was redone against the current AGENTS.md (it had grown from 62k to 66k since this PR), keeping everything added since — new Hard Rule text, Bun environment, sync-back landings, root paths, Dual Auth recipe, .husky/_ worktree step — and carrying the #15966 coverage wording. Evidence (owner-approved agent-instruction surface change, 2026-10-19): AGENTS.md 66,144 → 39,848 bytes; all 23 Hard Rules kept and numbered 1–23, every short-form rule carries its full binding text in docs/ops/HARD_RULES_DETAIL.md under the same number; a lost-fact script over the base AGENTS.md found 555/677 lines verbatim and every distinctive fact of the reworded 121 in AGENTS.md or the linked docs; check-docs-counts-sync (incl. the #15966 coverage-floor claims), check:docs-all, the 30 *15159* tests, docs-counts/fabricated-docs tests, typecheck core + open-sse all green. Remaining lint red is the base unused-suppression entry being fixed separately. Thanks @wahidsadik371-coder!

Clean merge: #17117 (nine resynced sources), #17111 (GUARDRAILS source), #15966 and others. The nine #17117 state entries are the tip's verbatim. GUARDRAILS keeps its honest record: mirrors and state still describe source 8c8f7fe5, so the drift gate keeps reporting the #17111 delta as stale.

jonlwheat2-gif requested a review from diegosouzapw as a code owner October 8, 2026 19:05

diegosouzapw mentioned this pull request Oct 10, 2026

fix(backend): whole-repository audit findings for v3.8.52 — 78 items (14 critical · 21 high · 29 medium · 14 keep) #15159

Closed

diegosouzapw closed this Oct 10, 2026

diegosouzapw reopened this Oct 10, 2026

diegosouzapw added 2 commits October 18, 2026 22:12

Merge remote-tracking branch 'origin/release/v3.8.52' into r6/15966

5a2e9f0

# Conflicts: # README.md

diegosouzapw merged commit 7d7ed3f into cheaper-inference:release/v3.8.52 Oct 19, 2026
8 checks passed

Navigation Menu

Repository navigation

fix(docs): widen the version-claim gate and correct AGENTS.md's coverage floor - #15966

fix(docs): widen the version-claim gate and correct AGENTS.md's coverage floor#15966
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.52from
jonlwheat2-gif:fix/15159-w2-g06-g08-doc-version-and-coverage-floor

jonlwheat2-gif commented Oct 8, 2026

Uh oh!

diegosouzapw commented Oct 10, 2026

Uh oh!

diegosouzapw commented Oct 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jonlwheat2-gif commented Oct 8, 2026

Summary

G-06 — the version validator matched only two literal phrasings

G-08 — AGENTS.md advertised the wrong coverage floor

The subtlety that makes widening safe

Scope note

Verification

Uh oh!

diegosouzapw commented Oct 10, 2026

Uh oh!

diegosouzapw commented Oct 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants