Skip to content

fix(docs): Auto-Combo factor count is read from DEFAULT_WEIGHTS - #11545

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.51from
maxmad64bis:docs/scoring-factor-count
Aug 25, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.51from
maxmad64bis:docs/scoring-factor-count

Conversation

@maxmad64bis

Copy link
Copy Markdown
Contributor

Summary

  • The Auto-Combo engine was described as 6-, 9-, 13- and 14-factor in seventeen places while DEFAULT_WEIGHTS declares fifteen factors. Two of the fifteen sit at weight 0 by default, which is where some lower counts came from, but they are computed for every candidate, and cacheAffinity gates prompt-cache deduplication outside the score — they are declared factors that do not vote by default, and AUTO-COMBO.md now says exactly that.
  • Instead of hand-fixing seventeen numbers, check:docs-counts grows a strict check that reads the count from scoring.ts and validates every N-factor claim across README, AGENTS.md, guides, diagrams and code comments. Adding a factor without updating the prose now fails CI. CHANGELOG.md is deliberately out of scope: old entries record what was true when they were written.
  • The strategy description string in routerStrategy.ts stops hard-coding an enumerable count ("15-factor weighted scoring (see DEFAULT_WEIGHTS)") so there is one fewer derived number to keep in sync.

Related Issues

  • None. Nothing open upstream on this topic at the time of writing (checked via issue search).

Validation

  • Change type: routing / docs
  • Focused tests and category gates from the golden path: check:docs-counts fails on the pre-change tree and passes after (verified in that order, otherwise the gate proves nothing); counting-helper unit tests pass
  • npm run lint

Local caveat: my machine's node_modules is broken (es-abstract incomplete) and a borrowed install carries a newer eslint-plugin-react-hooks than this branch's lockfile pins, which produces results that do not match CI. The only file my scoped run flagged is free-provider-rankings/page.tsx, untouched by this PR. CI is authoritative.

  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR (the new gate check and its counting helper are covered in tests/unit/check-docs-counts-sync.test.ts; code-comment strings are covered by the same gate)

Tests Added Or Updated

  • tests/unit/check-docs-counts-sync.test.ts — counting helper handles zero-weight factors, comment noise and unreadable sources; drift detection proven.
  • Existing lkgp-enabled-context-11181.test.ts and combo-matrix/auto.test.ts updated only where they quoted a stale factor count in a string.

Coverage Notes

  • Logic added lives in scripts/check/check-docs-counts-sync.mjs (build tooling) and is covered by the unit test above; runtime routing code changed only in doc-facing strings.

Reviewer Notes

  • The gate is strict on purpose: it reads the canonical count from source, so prose cannot drift silently again.
  • Unit suite on this exact base is currently red for an unrelated reason (159 vs 160 migrations). Fixed by the sibling PR docs/migrations-count-160 — merging that one first keeps this PR's CI green.

The engine was documented as 6-, 9-, 13- and 14-factor in seventeen places
while `DEFAULT_WEIGHTS` declares fifteen factors. Two of the fifteen sit at
weight 0 by default, which is where some of the lower counts came from, but
they are computed for every candidate and `cacheAffinity` gates prompt-cache
deduplication outside the score — they are declared factors that do not vote
by default, and `AUTO-COMBO.md` now says so.

`check:docs-counts` grows a strict check that reads the count from
`scoring.ts` and validates every `N-factor` claim in the README, `AGENTS.md`,
the guides, the diagrams and the code comments. Adding a factor without
updating the prose now fails. `CHANGELOG.md` is deliberately out of scope:
its old entries record what was true when they were written.

The gate fails on the pre-change tree and passes after, and the counting
helper is covered for zero-weight factors, comment noise and unreadable
sources.
@diegosouzapw
diegosouzapw merged commit a63b3e2 into cheaper-inference:release/v3.8.51 Aug 25, 2026
10 of 16 checks passed
@maxmad64bis
maxmad64bis deleted the docs/scoring-factor-count branch August 25, 2026 19:57
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…per-inference#11545)

Validated in a combined 3-PR batch worktree off release/v3.8.51 tip. The prerequisite sibling (159→160 migrations) was already covered by cheaper-inference#11502 — closed the redundant cheaper-inference#11543 separately.
- New check:docs-counts gate: proven fail-before/pass-after per the PR's own methodology, re-verified here (PASS, 2 unrelated soft-drift notices on cloud-agent/A2A doc counts, non-blocking)
- Focused tests: check-docs-counts-sync.test.ts, combo-matrix/auto.test.ts, lkgp-enabled-context-11181.test.ts — pass
- typecheck:core, file-size, changelog-integrity, complexity, cognitive-complexity — all OK

Thanks for replacing seventeen hand-maintained factor-count claims with one source-derived gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants