Skip to content

feat(routing): subscription-first auto groupings (auto/subscription, auto/thrifty) [defer to 3.8.51] - #11146

Merged
diegosouzapw merged 5 commits into
diegosouzapw:release/v3.8.51from
yourspraveen:feat/subscription-first-routing
Aug 25, 2026
Merged

diegosouzapw merged 5 commits into
diegosouzapw:release/v3.8.51from
yourspraveen:feat/subscription-first-routing

Conversation

@yourspraveen

@yourspraveen yourspraveen commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

What

Two new auto/* groupings for cost-conscious operators, sharing one rung model:

Id Policy On exhaustion
auto/subscription plan-included connections only, documented hard-stop overage only, live quota verified per connection fails closed — empty pool, never a billable fallback
auto/thrifty all rungs ordered subscription → keyless → free → cheap → premium fails open — steps up exactly one rung

Both are opt-in by being requested. No existing pool, strategy, or default changes; nothing routes through them unless a caller asks for the id by name.

Why

OmniRoute already answers "is this model catalogued free?" (hidePaidModels) and "can this connection ever bill me?" (freeAccessPolicy: "strict"), but both fail closed — an exhausted free pool is an empty pool, never a step up to a paid option. Every paid-side mechanism (cost-optimized, budgetCap, the cost-saver mode pack) is tier-agnostic. Nothing answers the question most operators actually ask:

"Use the quota I already pay for. When it runs out, either stop, or step up one rung at a time — and come back the moment it resets."

The blocker this had to solve first

Billing is a property of the connection, not the model. classifyTier() keys on (provider, model), but the same model is plan-included through a Claude Code OAuth connection and billed per token through an API-key connection. And auth_type is not a safe proxy in either direction — metered OAuth connections exist, and plan-included API-key connections exist (a Copilot seat token is not a metered API key).

So this adds a curated per-connection billing catalog (open-sse/config/connectionBillingCatalog.ts), hand-set from each provider's published terms — deliberately the same pattern FreeModelBudget.hardStopGuaranteed already established. Uncurated resolves to unknown and is consumed as metered, so a provider added tomorrow starts outside the subscription rung and has to be curated in deliberately.

Providers whose overage terms allow opting into usage-based billing (cursor, copilot-web) are recorded as unknown overage rather than hard-stop, which keeps them out of auto/subscription while leaving them usable at rung 0 of auto/thrifty.

Rungs have different exhaustion signals

This is why it is not merely a sort:

  • subscription / keyless / free exhaust on quota — observable, already tracked.
  • cheap / premium have no quota (a paid connection serves forever), so their only sane signal is a per-rung budget.

Budget gating is implemented and unit-tested but inert until a spend resolver is wired — with no accounting available a paid rung is ordered but never gated. Rung ordering, quota-based exhaustion, and reset re-entry all work without it. The spend-ledger choice (reuse usageAnalytics vs. a dedicated ledger) felt like it deserved its own decision rather than being smuggled in here.

Connection safety

Both groupings reuse STRICT_ZERO_COST's invariant verbatim: every id in allowedConnectionIds is verified individually and the array is rewritten to exactly the surviving subset — never the full original list. Since autoStrategy.ts already enforces that array as a hard allowlist before selecting a connection, "verified" and "actually used" are the same set by construction.

Returning to the plan after a reset

Three things must expire; fixing only one leaves routing stuck on paid rungs after the plan refills.

  1. Quota-state cache — ✅ wired. A cached entry whose own resetAt has passed describes a window that no longer exists, so it is now stale regardless of TTL and forces a refresh (freeAccessQuota.ts).
  2. Ladder state — ✅ none by design. Rung eligibility is recomputed per pool build; there is no persisted "currently on rung 3" record that could outlive a reset.
  3. Connection cooldown — ⚠️ helper implemented and tested, not yet wired. clampCooldownToReset() can only ever narrow a cooldown to the upstream's own reset instant. Wiring it needs resetAt captured before src/sse/services/auth.ts:2462 invalidates the quota cache — every cooldown write happens after that point, so a naive wiring would read undefined and be dead code that looks correct. That is a change to the resilience hot path and belongs in its own reviewed PR rather than riding along here.

Anti-flap: re-entry requires more headroom (reentryMinRemainingPercent, default 5) than staying in did (exitCutoffPercent, default 2, matching quotaPreflight.defaultThresholdPercent). The gap is the hysteresis band.

Configuration — tuning only, deliberately no enabled flag

A toggle able to switch these off would leave auto/subscription quietly serving the full pool, paid models included, under a name that promises the opposite.

{
  "subscriptionLadder": {
    "exitCutoffPercent": 2,
    "reentryMinRemainingPercent": 5,
    "rungBudgetUsd": { "cheap": 5.0, "premium": 0 }
  }
}

Also fixed

docs/guides/TIERS.md advertised a combo strategy named subscription. It has never existed — ROUTING_STRATEGY_VALUES has 19 entries and that is not one of them. Corrected to point at the real mechanism.

Testing

  • tests/unit/autoCombo/subscription-ladder.test.ts — 25 new tests, all passing. Covers classification resolution order, fail-closed vs. fail-open behavior, multi-account allowlist narrowing, rung ordering + stability, budget gating, the reset-staleness rule, the hysteresis band, and the cooldown clamp's never-extend property. Pure and dependency-light: every side effect is injected, and the billing catalog is overridden with a synthetic fixture so the tests survive edits to the curated entries.
  • npm run test:vitest — green.
  • tests/unit/auto-*.test.ts + hidePaidModels catalog tests — 134/134 pass.
  • npm run typecheck:core — clean.
  • npm run lint — clean on every file touched here.
  • npm run check:docs-all — no broken links, no fabricated references.

Notes for review

…auto/thrifty)

OmniRoute answers "is this model free?" (hidePaidModels) and "can this
connection ever bill me?" (STRICT_ZERO_COST), but both fail closed and every
paid-side mechanism (cost-optimized, budgetCap, the cost-saver mode pack) is
tier-agnostic. Nothing answers "use the plan quota I already pay for; when it
runs out either stop, or step up one rung at a time; and come back when it
resets."

The blocker was that billing is a property of the CONNECTION, not the model:
classifyTier() keys on (provider, model), while the same model is plan-included
through an OAuth connection and metered through an API-key one. auth_type is
not a safe proxy in either direction. So this adds a curated per-connection
billing catalog, hand-set from published terms, following the same pattern
FreeModelBudget.hardStopGuaranteed already established. Uncurated resolves to
unknown and is consumed as metered, so new providers start outside the
subscription rung.

Two ids, sharing one rung model (subscription > keyless > free > cheap >
premium):

  - auto/subscription: rung 0 only, hard-stop overage only, live quota verified
    per connection. Fails CLOSED — an empty pool is the intended answer.
  - auto/thrifty: all rungs ordered, exhausted ones gated out, scoring still
    runs within the survivors. Fails OPEN one rung at a time.

Both reuse STRICT_ZERO_COST's connection-safety invariant: each connection in
allowedConnectionIds is verified individually and the array is rewritten to the
surviving subset, so autoStrategy.ts can only dispatch to a verified account.

Reset re-entry: a cached quota reading whose own resetAt has passed is now
stale regardless of TTL, and rung eligibility is recomputed per pool build with
no persisted demotion that could outlive a reset. clampCooldownToReset() is
implemented and tested but not yet wired — the quota cache is invalidated in
auth.ts before any cooldown is written, so resetAt must be captured earlier
there; that hot-path change belongs in its own PR.

Both ids are opt-in by being requested; no existing pool, strategy or default
changes. Settings are tuning-only on purpose (no enabled flag that could leave
auto/subscription silently serving paid capacity).

Also corrects docs/guides/TIERS.md, which advertised a combo strategy named
"subscription" that has never existed in ROUTING_STRATEGY_VALUES.
@diegosouzapw diegosouzapw changed the title feat(routing): subscription-first auto groupings (auto/subscription, auto/thrifty) feat(routing): subscription-first auto groupings (auto/subscription, auto/thrifty) [defer to 3.8.51] Aug 22, 2026
@diegosouzapw diegosouzapw mentioned this pull request Aug 23, 2026
@yourspraveen

Copy link
Copy Markdown
Contributor Author

Refreshed onto the base tip — the previous red run was entirely stale-base

The branch had drifted 98 commits behind release/v3.8.50. Merged the tip (a7e09eda5) in at a92430feb; the merge is clean, zero conflicts.

The old run's failures were never this PR's

The previous CI run (32583221328, cut at merge-base 6cd4d38e2) went red on 4 unit shards, Fast Quality Gates, the ESLint-warnings gate and Merge integrity. None of the failing assertions touch anything this PR adds:

✖ Vietnamese locale has complete key parity with English      → #11208
✖ i18n pt-BR integrity / should contain every key present in en.json
✖ a private-host R2 image URL is blocked by the SSRF guard    → fixed in base by #10964
✖ guide-settings POST preserves existing OpenCode config fields → fixed in base by #11201
✖ config-generator / opencode (context-aware)
✖ buildSlackPayload / buildDiscordPayload / buildTelegramPayload — WEBHOOK_EVENTS
✖ sweep: uncloseai import fetches the live /models catalog
✖ getProviderCredentials still refuses a :free model on a credits_exhausted connection

The Merge-integrity failure was the same story — check:agent-skills-sync wanted to generate omni-webhooks, a skill unrelated to this diff. #11201 and #10964 have since landed the fixes for most of the above in the base.

Re-verified on the merged tree

Check Result
tests/unit/autoCombo/subscription-ladder.test.ts 25/25 pass
npm run test:vitest 45 files / 430 tests pass
npm run typecheck:core clean
npx eslint on all 9 changed source/test files clean
npm run check:agent-skills-sync now green — 46 unchanged, 0 generated, 0 orphans
npm run check:docs-all doc-links PASS (857 links), fabricated-docs PASS
npm run check:test-discovery OK — no new orphan (the suite is collected via vitest.mcp.config.ts → tests/unit/autoCombo/**)

The 12-file diff vs. the new base is unchanged in substance — only the base moved.

Still inherited, not mine

⚠️ base-red inherited: #9985 — the tip is still not release-green as of the 15:14 verdict today (ESLint hard failure, offending range 968fa9610..8f390efff / #11205). Updated the note in the body to point at the current cause rather than the stale one.

One question before this is mergeable, @diegosouzapw

The title carries [defer to 3.8.51] and the body opens with the same intent. release/v3.8.51 does not exist yet, so the PR still targets release/v3.8.50 and I have kept it there rather than guessing. Do you want it retargeted the moment the next cycle branch is cut, or is it fine to land on v3.8.50? Both auto/subscription and auto/thrifty are inert unless a caller asks for the id by name — no existing pool, strategy or default changes — so landing it early carries no behavioural risk to the release. Entirely your call on timing; I will retarget on a word from you.

@yourspraveen

Copy link
Copy Markdown
Contributor Author

Post-refresh CI: still red, still not this PR

The refreshed run (32654658163) came back red, so to be precise about what it is: every failing assertion reproduces on the pristine tip a7e09eda5 with nothing applied, and the maintainer's own #11262 fails the identical set. Full attribution posted at #9985 — 17 assertions across 12 files, traced to #11166, #11178, #11179, #11097, #11224 and #11227.

The three shards that failed here:

Shard Failures Status
1/4 i18n pt-BR ×2, Codex context override, search-route 400, vscode-token-routes ×3 base-red
2/4 chatCore combo skip, CLI_TOOLS ×3, settings-i18n-keys base-red
3/4 check-deps allowlist, cli-tools-schema, cli-catalog-display-contract, provider-models-route-codex, vscode gpt56 base-red
3/4 ttft() measures first-forwarded-chunk latency flake — 8/8 on the pristine tip, 3× 8/8 on this branch

Nothing in that list touches subscriptionLadder.ts, connectionBilling.ts, connectionBillingCatalog.ts, freeAccessQuota.ts or the two auto/* ids. This PR's own suite is 25/25 and npm run test:vitest is 430/430 on the merged tree.

The Fast Quality Gates and No new ESLint warnings failures are the same inherited story — #9985's own verdict has been reporting ESLint: could not parse eslint json on the base since the 15:14 run.

So this stays blocked on the base, not on review. I have offered on #9985 to open the fix/release-v3.8.50-basereds-* PR that drains all 17 with the originating authors co-credited — that would unblock this PR, #11262 and the other 28 open against the branch in one go. Happy to start on your word, @diegosouzapw.

@diegosouzapw

Copy link
Copy Markdown
Owner

Hi @yourspraveen — excellent PR. The connection-billing-vs-model-tier distinction, the unknown→metered fail-safe curation direction, opposite admitUnknownQuota semantics for the two ids, and the reset-staleness fix are all exactly the kind of rigor this codebase wants; tests are in the right suite and the pool wiring reuses the existing bulk reads. Three small items before merge: (1) please make sure SUBSCRIPTION_LADDER.md and the settings description state plainly that rungBudgetUsd is accepted but not yet enforced (no spend resolver yet) so operators don't rely on paid-rung budgets; (2) clampCooldownToReset currently has no production caller — keep it but link the follow-up issue that wires it into the auth.ts cooldown path (or drop it from this PR); (3) just needs a green CI run against the refreshed base (the remaining reds look like the inherited #11449 tip issues, not yours). Once those are in, this is ready.

Hermes Developer and others added 2 commits August 25, 2026 01:35
…lFactory at merge size

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit b39e5ec into diegosouzapw:release/v3.8.51 Aug 25, 2026
6 of 7 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…auto/thrifty) [defer to 3.8.51] (diegosouzapw#11146)

Merged into release/v3.8.51 via batch validation: subscription-ladder + free-regime vitest suites green (32/32) on the combined tree, check:provider-consistency OK (353 canonical providers), static gates green (virtualFactory.ts frozen at merge size with dated rebaseline). Also pushed a docs commit marking rungBudgetUsd as not-yet-enforced per review, and synced the branch onto the updated release tip. Strong opt-in design failing closed where money is involved — thanks @yourspraveen!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants