Skip to content

feat(routing): report why the zero-cost guard excluded a candidate - #12319

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.51from
maxmad64bis:feat/strict-zero-cost-exclusion-reason
Sep 1, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.51from
maxmad64bis:feat/strict-zero-cost-exclusion-reason

Conversation

@maxmad64bis

@maxmad64bis maxmad64bis commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Turn on freeAccessPolicy: "strict" and the read-only candidate listing starts hiding things. A model the zero-cost guard excludes doesn't show up as excluded — it's gone, and the operator can't tell "no free allowance left" from "the quota fetcher is broken".

That listing exists to be honest about the pool; its module header says a candidate the routing path would skip "is never dropped from this listing". #9133 settled the same question for the resilience filter, via a skip opt-out. The zero-cost guard never got it.

So it gets it. The guard is disabled for the inspector build, exactly as the resilience filter already is, and every candidate carries freeAccessExclusion — null when the guard is satisfied, otherwise one of:

Reason Means
not-in-catalog absent from FREE_MODEL_BUDGETS
regime-not-free catalogued, but the tier doesn't grant free access
no-hard-stop free tier, but exceeding it might silently bill
contradictory-noauth a no-auth candidate whose entry isn't keyless
exhausted a fresh reading says the allowance is used up
state-unknown no quota reading, or one too old to trust
no-connection no account to check at all

The last three are the point. An exhausted allowance fixes itself, a reading that never arrives means the fetcher is broken, and a candidate with no account was never going to be checked. All three used to look identical, because the row just disappeared. STRICT_ZERO_COST.md spells out what each one asks an operator to do.

Dispatch is untouched. The guard still applies to the routing pool, and a test pins that. What changed is which build skips it.

Under the hood the boolean deciding one connection's fate became a classification, and evaluateCandidateConnections is now a thin wrapper returning the same string[] it always did. The three existing guard suites weren't modified — they're the net, and their 31 cases still pass.

One order change: freshness is now checked before status, so a stale EXHAUSTED reading reports state-unknown. Same exclusion either way; the reason is just honest about what we know.

excludeTosAvoid still drops its candidates without a reason. Same hole, different guard — what a ToS exclusion should report is a separate question, so it's documented rather than closed.

Related Issues

Validation

  • Change type: routing
  • Focused tests and category gates from the golden path
  • npm run lint
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR

Red first: drop && !skip and the listing test fails — the excluded candidates vanish again. Restore it and 27 cases pass across the new files, the #9133 locked-model test and the #7819 route test. The three pre-existing guard suites (under vitest.mcp.config.ts) stay green at 31, unmodified.

npm run lint exits 2 here and on an untouched checkout of the base alike (suppressions left that do not occur anymore, zero reported errors).

Tests Added Or Updated

  • tests/unit/strict-zero-cost-exclusion-reason.test.ts (new: the classification, plus the contract the pool filter depends on)
  • tests/unit/auto-combo-candidates-free-access-reason.test.ts (new: dispatch untouched, the listing keeps excluded candidates with a reason, policy-off stays free)

Coverage Notes

strictZeroCostFilter.ts gains two exported functions and loses a private helper; both are covered directly, and the old public entry point keeps its contract under the tests it already had. virtualFactory.ts gains the skip term and an annotation pass that only runs for the inspector. autoComboCandidates.ts copies a field, the job it already does for the breaker, the cooldown and the model lock.

Reviewer Notes

  • The annotation lives in buildPreparedPool, not the handler: the handler has no access to settings, so doing it there would duplicate two thresholds with nothing keeping them in agreement.
  • For a multi-account candidate, exhausted beats state-unknown — an observed exhaustion is a fact, a missing reading is the absence of one. Without a rule the reason would depend on connection ordering.
  • With the policy off nothing is classified and the field is null.
  • resolveFreeAccessState is called on the same candidates the guard used to call it on, so the inspector's quota-lookup load is unchanged.

@maxmad64bis
maxmad64bis force-pushed the feat/strict-zero-cost-exclusion-reason branch from 1ed55eb to f4b6ec1 Compare September 1, 2026 14:18
@maxmad64bis
maxmad64bis force-pushed the feat/strict-zero-cost-exclusion-reason branch from f4b6ec1 to 25e16d2 Compare September 1, 2026 14:25
@diegosouzapw
diegosouzapw merged commit 2f33f2c into cheaper-inference:release/v3.8.51 Sep 1, 2026
15 of 16 checks passed
@maxmad64bis
maxmad64bis deleted the feat/strict-zero-cost-exclusion-reason branch September 24, 2026 21:11
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…heaper-inference#12319)

With freeAccessPolicy: "strict" the read-only candidate listing silently dropped rows, so an operator could not tell "no free allowance left" from "the quota fetcher is broken" — in a listing whose own module header promises a candidate the routing path would skip "is never dropped". cheaper-inference#9133 settled the same question for the resilience filter via a skip opt-out; the zero-cost guard never got one.

It gets it now: the guard is disabled for the inspector build exactly as the resilience filter already is, and every candidate carries freeAccessExclusion — null when satisfied, otherwise one of seven named reasons. The last three (exhausted, state-unknown, no-connection) are the point: they used to look identical because the row just disappeared. STRICT_ZERO_COST.md documents what each asks the operator to do.

Dispatch is untouched and a test pins that. The three existing guard suites were not modified — their 31 cases are the net and still pass. excludeTosAvoid still drops candidates without a reason; documented as a separate question rather than widened into here.

Verified in a combined batch worktree: 174/174 focused tests across all 11 PRs of this batch, typecheck:core clean, complexity 2706/3218, cognitive-complexity 1221/1437, check:cycles and check:docs-counts green.

Thanks @maxmad64bis — the reason table and the honest note about the order change (freshness before status) made this easy to review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants