Skip to content

fix(resilience): honor dashboard quota snapshots in opencode-go preflight (#11234) - #11267

Merged
diegosouzapw merged 1 commit into
release/v3.8.50from
fix/11234-opencode-quota-preflight
Aug 23, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.50from
fix/11234-opencode-quota-preflight

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Closes #11234

Problem

The opencode-go quota preflight ignored the dashboard quota snapshots, so priority combos (e.g. opencode-go/deepseek-v4-pro) kept selecting sibling connections whose weekly window was already drained — the data was visible on the dashboard but never reached routing. Two independent gaps:

Gap A — fetcher never read the snapshots. fetchOpencodeQuota only consulted the live upstream endpoint, which has no public quota API (404 — the module JSDoc already admits this). Every preflight evaluated null → evaluateQuotaCutoff(null) → proceed (fail-open), even with a connection at 0% weekly remaining persisted by the dashboard scrape. Window keys also differed: the fetcher registry uses window_5h/window_weekly/window_monthly while the dashboard snapshots use session/weekly/mcp_monthly.

Gap B — the operator flag never armed sibling selection. QUOTA_PREFLIGHT_CUTOFF_ENABLED (resilience.quotaPreflight.enabled) was only read by the auto-strategy candidate builder and by the per-target cutoff that requires a pinned connectionId. The sibling-selection latency gate in getProviderCredentialsWithQuotaPreflight never consulted it, so priority combos (connectionId null at combo level) skipped preflight entirely.

Fix

A) Snapshot bridge in the fetcher (open-sse/services/opencodeQuotaFetcher.ts). When the live endpoint yields nothing and the connection has dashboard scrape config, the fetcher synthesizes its triple-window QuotaInfo from the cached snapshots — session→window_5h, weekly→window_weekly, mcp_monthly→window_monthly, percentUsed = 1 − remainingPercentage/100 — mirroring getQuotaWindowStatus semantics: an expired resetAt means the window rolled over and must not count as exhausted; fractionReported === false (#10095) is "unknown", never exhaustion. Read-only via src/domain/quotaCache.ts accessors (never SQL, never a re-scrape on the hot path), gated on scrape config so unconfigured connections never touch the snapshot store. Fail-open preserved: no snapshots → null, exactly as before. The quotaCache import is dynamic because a static edge would close an initialization cycle (fetcher → quotaCache → usage.ts → usage/opencode.ts → fetcher).

B) Flag scope (src/sse/services/auth.ts). resilience.quotaPreflight.enabled === true now also arms the sibling-selection latency gate, so priority combos filter exhausted sisters. Default unchanged (flag off = today's behavior).

Gap 3 from the issue analysis (isExhausted() requiring ALL windows at zero) was deliberately not touched — inverting it globally risks over-blocking providers with alternative windows.

Validation (TDD, Hard Rule #18)

New tests/unit/quota-exhaustion-cutoff-opencode.test.ts (5 tests, all RED→GREEN except the fail-open guard):

Test Before After
fetcher 404 + snapshots weekly=0%/session=80% → blocked RED (quota null) GREEN
weekly 0% with next_reset_at in the past → not blocked RED GREEN
per-window threshold override on mapped window_weekly key RED GREEN
configured dashboard, no snapshots → null (fail-open) GREEN (guard) GREEN
flag on, two sisters, priority-1 exhausted → picks healthy RED (picked exhausted) GREEN

Sibling suites green (~500 tests): opencode-quota-fetcher (18, 404-latch unchanged), quota-preflight, combo-priority-quota-exhaustion-cutoff-5923, combo-quota-exhaustion-only-fallback, issue-6686, 8431-multiwindow, quota-fetch-throttle-6911, sse-auth* (4 files), all quota fetchers (codex/bailian/deepseek/generic/agentrouter/grok/firecrawl/freemodel), combo strategies, snapshot/hydration tests. npx eslint (with suppressions) and npm run typecheck:core clean on all touched files.

…ight (#11234)

The opencode-go quota preflight ignored the dashboard quota snapshots, so
priority combos kept selecting connections whose weekly window was already
drained. Two gaps, two fixes:

A) fetchOpencodeQuota only consulted the live upstream endpoint, which has
   no public quota API (404 — the module JSDoc already admits this). Every
   preflight therefore evaluated null and proceeded (fail-open) even with a
   connection at 0% weekly remaining in plain sight on the dashboard. The
   fetcher now synthesizes its triple-window QuotaInfo from the cached
   dashboard snapshots when the live endpoint yields nothing, mapping
   session→window_5h, weekly→window_weekly, mcp_monthly→window_monthly and
   mirroring getQuotaWindowStatus semantics (expired resetAt = window rolled
   over = must not count as exhausted; fractionReported=false = unknown,
   never exhaustion). Read-only via src/domain/quotaCache.ts accessors —
   never SQL, never a re-scrape on the hot path — and gated on the
   connection actually having dashboard scrape config, so unconfigured
   connections never touch the snapshot store. Fail-open is preserved:
   no snapshots → null, exactly as before. The quotaCache import is dynamic
   because a static edge would close an initialization cycle
   (fetcher → quotaCache → usage.ts → usage/opencode.ts → fetcher).

B) The sibling-selection latency gate in getProviderCredentialsWithQuotaPreflight
   never consulted resilience.quotaPreflight.enabled
   (QUOTA_PREFLIGHT_CUTOFF_ENABLED) — that flag only armed the auto-strategy
   candidate builder and the per-target cutoff for pinned connections, so a
   priority combo over sibling opencode-go connections (connectionId null at
   combo level) skipped preflight entirely. The flag now arms the gate as
   well; the default (flag off) is unchanged.

TDD (Hard Rule #18), tests/unit/quota-exhaustion-cutoff-opencode.test.ts:
- fetcher 404 + seeded snapshots weekly=0%/session=80% → cutoff blocks
  (RED before, GREEN after); also asserts the bridge is read-only (single
  upstream fetch, no re-scrape).
- weekly 0% with next_reset_at in the past → not blocked (window dropped).
- per-window threshold override resolves against the mapped window_weekly
  key (50% override blocks at 40% remaining; factory 2% does not).
- fail-open guard: configured dashboard with no snapshots still returns null.
- selector level: flag on, two opencode-go sisters, priority-1 exhausted →
  selection skips to the healthy one (RED before, GREEN after).

Sibling suites green: opencode-quota-fetcher (18), quota-preflight,
combo-priority-quota-exhaustion-cutoff-5923, issue-6686, 8431, throttle-6911,
sse-auth*, quota fetchers, combo strategies, snapshot/hydration tests
(~500 tests). eslint (with suppressions) and typecheck:core clean.

Closes #11234
@diegosouzapw
diegosouzapw merged commit 92a083a into release/v3.8.50 Aug 23, 2026
16 of 22 checks passed
diegosouzapw pushed a commit that referenced this pull request Aug 23, 2026
…tap.testFiles

Base-red drain: #11267 added tests/unit/quota-exhaustion-cutoff-opencode.test.ts
(covers the mutated src/sse/services/auth.ts) without registering it, so the
mutation-test-coverage gate fails on the pristine release tip.
diegosouzapw added a commit that referenced this pull request Aug 23, 2026
…eview) (#11281)

TDD: 4 red-to-green tests reproducing the two review findings + 6 new cases; 30/30 in tests/unit/codex-app-server.test.ts; typecheck/eslint/env-doc-sync/file-size clean. Merged --admin over the inherited 2026-08-23 base-red cluster (#9985): the remaining reds (CLI catalog/registry tests, @testing-library allowlist) are proven tip failures unrelated to this diff — shard logs show only CLI-cluster failures, and this PR itself drains the mutation-test-coverage red (stryker registration for #11267's test). Behavior change for app-server deployments is documented in the PR body (sandbox default + binding refusal).
diegosouzapw pushed a commit that referenced this pull request Aug 23, 2026
, #11267)

Three reds the PR's CI surfaced after the base advanced past the branch
cut — each discriminated with its origin PR:

1. chatcore-translation-paths 'Combo skip behavior' (shard 3/4) — REAL
   BUG in #11178: the incompatible-reasoning action derivation switched
   from the explicit fallback config to isComboStep =
   Boolean(comboStepId || comboExecutionKey). Combos whose records carry
   no explicit stepId/executionKey (plain model-list combos) had their
   explicit reasoningTransportFallback: 'skip' config silently degraded
   to 'drop', contradicting the PR's own stated intent ('combos keep
   their explicit strategy'). Fix: isComboStep now honors the isCombo
   marker (isCombo || step ids present). RED->GREEN on the exact CI
   failing test; the #10959 single-target drop defaults stay green.

2. check-db-rules-classification 'recovery zero importers' (shard 1/4)
   — STALE GATE, not dead code: #11238 converted bin/cli/runtime.mjs
   dynamic imports to the Windows-safe projectFileUrl('...') idiom, and
   the gate's importer regexes only recognized static/from/template
   import forms. recovery's only importer became invisible. Fix: gate
   pattern set extended to recognize import(projectFileUrl('…/db/<mod>.ts')).

3. mutation-test-coverage gate — #11267 added
   tests/unit/quota-exhaustion-cutoff-opencode.test.ts covering
   src/sse/services/auth.ts without registering it in stryker.conf.json
   tap.testFiles. Fix: register it (gate green locally).

Refs #9985
@diegosouzapw
diegosouzapw deleted the fix/11234-opencode-quota-preflight branch August 23, 2026 21:46
@diegosouzapw diegosouzapw mentioned this pull request Aug 24, 2026
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ight (diegosouzapw#11234) (diegosouzapw#11267)

The opencode-go quota preflight ignored the dashboard quota snapshots, so
priority combos kept selecting connections whose weekly window was already
drained. Two gaps, two fixes:

A) fetchOpencodeQuota only consulted the live upstream endpoint, which has
   no public quota API (404 — the module JSDoc already admits this). Every
   preflight therefore evaluated null and proceeded (fail-open) even with a
   connection at 0% weekly remaining in plain sight on the dashboard. The
   fetcher now synthesizes its triple-window QuotaInfo from the cached
   dashboard snapshots when the live endpoint yields nothing, mapping
   session→window_5h, weekly→window_weekly, mcp_monthly→window_monthly and
   mirroring getQuotaWindowStatus semantics (expired resetAt = window rolled
   over = must not count as exhausted; fractionReported=false = unknown,
   never exhaustion). Read-only via src/domain/quotaCache.ts accessors —
   never SQL, never a re-scrape on the hot path — and gated on the
   connection actually having dashboard scrape config, so unconfigured
   connections never touch the snapshot store. Fail-open is preserved:
   no snapshots → null, exactly as before. The quotaCache import is dynamic
   because a static edge would close an initialization cycle
   (fetcher → quotaCache → usage.ts → usage/opencode.ts → fetcher).

B) The sibling-selection latency gate in getProviderCredentialsWithQuotaPreflight
   never consulted resilience.quotaPreflight.enabled
   (QUOTA_PREFLIGHT_CUTOFF_ENABLED) — that flag only armed the auto-strategy
   candidate builder and the per-target cutoff for pinned connections, so a
   priority combo over sibling opencode-go connections (connectionId null at
   combo level) skipped preflight entirely. The flag now arms the gate as
   well; the default (flag off) is unchanged.

TDD (Hard Rule diegosouzapw#18), tests/unit/quota-exhaustion-cutoff-opencode.test.ts:
- fetcher 404 + seeded snapshots weekly=0%/session=80% → cutoff blocks
  (RED before, GREEN after); also asserts the bridge is read-only (single
  upstream fetch, no re-scrape).
- weekly 0% with next_reset_at in the past → not blocked (window dropped).
- per-window threshold override resolves against the mapped window_weekly
  key (50% override blocks at 40% remaining; factory 2% does not).
- fail-open guard: configured dashboard with no snapshots still returns null.
- selector level: flag on, two opencode-go sisters, priority-1 exhausted →
  selection skips to the healthy one (RED before, GREEN after).

Sibling suites green: opencode-quota-fetcher (18), quota-preflight,
combo-priority-quota-exhaustion-cutoff-5923, issue-6686, 8431, throttle-6911,
sse-auth*, quota fetchers, combo strategies, snapshot/hydration tests
(~500 tests). eslint (with suppressions) and typecheck:core clean.

Closes diegosouzapw#11234

Co-authored-by: Xiangzhe <bakryun0718@proton.me>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…post-merge review) (diegosouzapw#11281)

TDD: 4 red-to-green tests reproducing the two review findings + 6 new cases; 30/30 in tests/unit/codex-app-server.test.ts; typecheck/eslint/env-doc-sync/file-size clean. Merged --admin over the inherited 2026-08-23 base-red cluster (diegosouzapw#9985): the remaining reds (CLI catalog/registry tests, @testing-library allowlist) are proven tip failures unrelated to this diff — shard logs show only CLI-cluster failures, and this PR itself drains the mutation-test-coverage red (stryker registration for diegosouzapw#11267's test). Behavior change for app-server deployments is documented in the PR body (sandbox default + binding refusal).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] OpenCode Go quota preflight ignores dashboard-derived quota snapshots and still dispatches to exhausted connections

2 participants