feat(routing): subscription-first auto groupings (auto/subscription, auto/thrifty) [defer to 3.8.51] - #11146
Conversation
…auto/thrifty)
OmniRoute answers "is this model free?" (hidePaidModels) and "can this
connection ever bill me?" (STRICT_ZERO_COST), but both fail closed and every
paid-side mechanism (cost-optimized, budgetCap, the cost-saver mode pack) is
tier-agnostic. Nothing answers "use the plan quota I already pay for; when it
runs out either stop, or step up one rung at a time; and come back when it
resets."
The blocker was that billing is a property of the CONNECTION, not the model:
classifyTier() keys on (provider, model), while the same model is plan-included
through an OAuth connection and metered through an API-key one. auth_type is
not a safe proxy in either direction. So this adds a curated per-connection
billing catalog, hand-set from published terms, following the same pattern
FreeModelBudget.hardStopGuaranteed already established. Uncurated resolves to
unknown and is consumed as metered, so new providers start outside the
subscription rung.
Two ids, sharing one rung model (subscription > keyless > free > cheap >
premium):
- auto/subscription: rung 0 only, hard-stop overage only, live quota verified
per connection. Fails CLOSED — an empty pool is the intended answer.
- auto/thrifty: all rungs ordered, exhausted ones gated out, scoring still
runs within the survivors. Fails OPEN one rung at a time.
Both reuse STRICT_ZERO_COST's connection-safety invariant: each connection in
allowedConnectionIds is verified individually and the array is rewritten to the
surviving subset, so autoStrategy.ts can only dispatch to a verified account.
Reset re-entry: a cached quota reading whose own resetAt has passed is now
stale regardless of TTL, and rung eligibility is recomputed per pool build with
no persisted demotion that could outlive a reset. clampCooldownToReset() is
implemented and tested but not yet wired — the quota cache is invalidated in
auth.ts before any cooldown is written, so resetAt must be captured earlier
there; that hot-path change belongs in its own PR.
Both ids are opt-in by being requested; no existing pool, strategy or default
changes. Settings are tuning-only on purpose (no enabled flag that could leave
auto/subscription silently serving paid capacity).
Also corrects docs/guides/TIERS.md, which advertised a combo strategy named
"subscription" that has never existed in ROUTING_STRATEGY_VALUES.
…scription-first-routing
Refreshed onto the base tip — the previous red run was entirely stale-baseThe branch had drifted 98 commits behind The old run's failures were never this PR'sThe previous CI run ( The Merge-integrity failure was the same story — Re-verified on the merged tree
The 12-file diff vs. the new base is unchanged in substance — only the base moved. Still inherited, not mine
One question before this is mergeable, @diegosouzapwThe title carries |
Post-refresh CI: still red, still not this PRThe refreshed run ( The three shards that failed here:
Nothing in that list touches The So this stays blocked on the base, not on review. I have offered on #9985 to open the |
|
Hi @yourspraveen — excellent PR. The connection-billing-vs-model-tier distinction, the unknown→metered fail-safe curation direction, opposite admitUnknownQuota semantics for the two ids, and the reset-staleness fix are all exactly the kind of rigor this codebase wants; tests are in the right suite and the pool wiring reuses the existing bulk reads. Three small items before merge: (1) please make sure SUBSCRIPTION_LADDER.md and the settings description state plainly that rungBudgetUsd is accepted but not yet enforced (no spend resolver yet) so operators don't rely on paid-rung budgets; (2) clampCooldownToReset currently has no production caller — keep it but link the follow-up issue that wires it into the auth.ts cooldown path (or drop it from this PR); (3) just needs a green CI run against the refreshed base (the remaining reds look like the inherited #11449 tip issues, not yours). Once those are in, this is ready. |
…lFactory at merge size Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
b39e5ec
into
diegosouzapw:release/v3.8.51
…auto/thrifty) [defer to 3.8.51] (diegosouzapw#11146) Merged into release/v3.8.51 via batch validation: subscription-ladder + free-regime vitest suites green (32/32) on the combined tree, check:provider-consistency OK (353 canonical providers), static gates green (virtualFactory.ts frozen at merge size with dated rebaseline). Also pushed a docs commit marking rungBudgetUsd as not-yet-enforced per review, and synced the branch onto the updated release tip. Strong opt-in design failing closed where money is involved — thanks @yourspraveen!
What
Two new
auto/*groupings for cost-conscious operators, sharing one rung model:auto/subscriptionauto/thriftysubscription → keyless → free → cheap → premiumBoth are opt-in by being requested. No existing pool, strategy, or default changes; nothing routes through them unless a caller asks for the id by name.
Why
OmniRoute already answers "is this model catalogued free?" (
hidePaidModels) and "can this connection ever bill me?" (freeAccessPolicy: "strict"), but both fail closed — an exhausted free pool is an empty pool, never a step up to a paid option. Every paid-side mechanism (cost-optimized,budgetCap, thecost-savermode pack) is tier-agnostic. Nothing answers the question most operators actually ask:The blocker this had to solve first
Billing is a property of the connection, not the model.
classifyTier()keys on(provider, model), but the same model is plan-included through a Claude Code OAuth connection and billed per token through an API-key connection. Andauth_typeis not a safe proxy in either direction — metered OAuth connections exist, and plan-included API-key connections exist (a Copilot seat token is not a metered API key).So this adds a curated per-connection billing catalog (
open-sse/config/connectionBillingCatalog.ts), hand-set from each provider's published terms — deliberately the same patternFreeModelBudget.hardStopGuaranteedalready established. Uncurated resolves tounknownand is consumed asmetered, so a provider added tomorrow starts outside the subscription rung and has to be curated in deliberately.Providers whose overage terms allow opting into usage-based billing (
cursor,copilot-web) are recorded asunknownoverage rather thanhard-stop, which keeps them out ofauto/subscriptionwhile leaving them usable at rung 0 ofauto/thrifty.Rungs have different exhaustion signals
This is why it is not merely a sort:
Budget gating is implemented and unit-tested but inert until a spend resolver is wired — with no accounting available a paid rung is ordered but never gated. Rung ordering, quota-based exhaustion, and reset re-entry all work without it. The spend-ledger choice (reuse
usageAnalyticsvs. a dedicated ledger) felt like it deserved its own decision rather than being smuggled in here.Connection safety
Both groupings reuse STRICT_ZERO_COST's invariant verbatim: every id in
allowedConnectionIdsis verified individually and the array is rewritten to exactly the surviving subset — never the full original list. SinceautoStrategy.tsalready enforces that array as a hard allowlist before selecting a connection, "verified" and "actually used" are the same set by construction.Returning to the plan after a reset
Three things must expire; fixing only one leaves routing stuck on paid rungs after the plan refills.
resetAthas passed describes a window that no longer exists, so it is now stale regardless of TTL and forces a refresh (freeAccessQuota.ts).clampCooldownToReset()can only ever narrow a cooldown to the upstream's own reset instant. Wiring it needsresetAtcaptured beforesrc/sse/services/auth.ts:2462invalidates the quota cache — every cooldown write happens after that point, so a naive wiring would readundefinedand be dead code that looks correct. That is a change to the resilience hot path and belongs in its own reviewed PR rather than riding along here.Anti-flap: re-entry requires more headroom (
reentryMinRemainingPercent, default 5) than staying in did (exitCutoffPercent, default 2, matchingquotaPreflight.defaultThresholdPercent). The gap is the hysteresis band.Configuration — tuning only, deliberately no
enabledflagA toggle able to switch these off would leave
auto/subscriptionquietly serving the full pool, paid models included, under a name that promises the opposite.{ "subscriptionLadder": { "exitCutoffPercent": 2, "reentryMinRemainingPercent": 5, "rungBudgetUsd": { "cheap": 5.0, "premium": 0 } } }Also fixed
docs/guides/TIERS.mdadvertised a combo strategy namedsubscription. It has never existed —ROUTING_STRATEGY_VALUEShas 19 entries and that is not one of them. Corrected to point at the real mechanism.Testing
tests/unit/autoCombo/subscription-ladder.test.ts— 25 new tests, all passing. Covers classification resolution order, fail-closed vs. fail-open behavior, multi-account allowlist narrowing, rung ordering + stability, budget gating, the reset-staleness rule, the hysteresis band, and the cooldown clamp's never-extend property. Pure and dependency-light: every side effect is injected, and the billing catalog is overridden with a synthetic fixture so the tests survive edits to the curated entries.npm run test:vitest— green.tests/unit/auto-*.test.ts+hidePaidModelscatalog tests — 134/134 pass.npm run typecheck:core— clean.npm run lint— clean on every file touched here.npm run check:docs-all— no broken links, no fabricated references.Notes for review
a7e09eda5is still not release-green (latest verdict 2026-08-23 15:14: ESLint hard failure, offending range968fa9610..8f390efff/ feat(codex): self-contained codex app-server transport (executor + provider + sign-in) #11205). Untouched here, per the base-green rule.a7e09eda5(2026-08-23). The branch was 98 commits behind; every red check on the previous run came from that stale base and is re-verified below.docs/routing/SUBSCRIPTION_LADDER.md.