Skip to content

[defer] fix(routing): keep approximate context estimates advisory - #10162

Merged
diegosouzapw merged 46 commits into
cheaper-inference:release/v3.8.50from
xz-dev:fix/remove-approximate-combo-400
Aug 20, 2026
Merged

diegosouzapw merged 46 commits into
cheaper-inference:release/v3.8.50from
xz-dev:fix/remove-approximate-combo-400

Conversation

@xz-dev

@xz-dev xz-dev commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Remove Combo pre-dispatch 400 context_length_exceeded responses that treated approximate gateway token estimates as authoritative.
  • Keep context metadata advisory for target ordering, including restoring Auto's eligible pool when every known limit looks too small.
  • Preserve hard capability/output-token checks, concrete-target context handling, heterogeneous larger-context fallback, and homogeneous upstream context_length_exceeded short-circuiting.

Related Issues

Validation

Choose the change type and focused loop from the
Contribution Golden Path. The full unit suite,
Vitest, the 60% coverage gate, and the production build all run in CI on this PR (#8329):

  • Change type: routing
  • Focused tests and category gates from the golden path
  • npm run lint — changed files pass; the full command is base-red in three unchanged files, documented below
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR
  • SonarQube PR analysis is green or any remaining issues are explicitly documented below

Focused validation:

  • 54/54 tests pass across capability filtering, target resolution, priority/round-robin/Auto advisory context handling, homogeneous and heterogeneous upstream overflow behavior, and the fix: 400 Input exceeds context window for opencode/north-mini-code-free #8841 reproduction.
  • Prettier check passes for all changed non-deleted files.
  • ESLint passes for all changed non-deleted files.
  • git diff --check passes.
  • npm run typecheck:core reports 12 pre-existing diagnostics in unchanged files and none in changed files.

Tests Added Or Updated

  • tests/unit/8488-capability-filter-fail-closed.test.ts
  • tests/unit/combo-context-window-filter.test.ts
  • tests/unit/combo-target-resolution-split.test.ts
  • tests/unit/repro-8841-context-overflow-opencode.test.ts

Coverage Notes

The updated tests exercise the public handleComboChat() seam for priority, round-robin, and Auto dispatch when approximate estimates exceed every catalog hint. They also verify that a real upstream overflow still short-circuits a homogeneous same-model pool while a heterogeneous pool may try a larger-context target.

Reviewer Notes

  • This is intentionally a Draft because enhancement(routing): RFC — separate approximate token estimates from hard admission decisions #10141 is design-first and requests maintainer agreement on exact/estimated/upstream-managed semantics.
  • Hard tool, vision, structured-output, and requested output-token compatibility rules remain unchanged.
  • Direct/single-provider capability-filter semantics remain unchanged; the shared hasEstimableContent helper only moved to an existing imported module when knownContextOverflow.ts was deleted.
  • The active release base contains duplicate migration version 143 (143_api_key_cache_default_mode.sql and 143_job_registry.sql). Focused tests were run with the latter temporarily unavailable and restored immediately; this PR does not modify or repair either migration.
  • Full npm run lint is base-red only in unchanged src/shared/components/modelSelectModalHelpers.ts, tests/unit/combo/image-combo.test.ts, and tests/unit/db-adapters/driverFactory.test.ts.
  • The new changelog.d/fixes/10162-approximate-combo-context-advisory.md fragment passes the fragment validator in isolation. The full changelog-integrity command is base-red because two unchanged feature fragments do not start with Markdown bullets.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks for the thorough write-up here, and for being upfront that this is a Draft tied to #10141's design questions.

I went through the diff and validated it against the current release/v3.8.50 tip. The branch is currently a good ways behind (it was opened right around the time #10152 fixed the duplicate 143_* migration you flagged in the PR description — that's already resolved upstream), but it merges cleanly with zero conflicts, and all 31 tests across the four files you touched (repro-8841-context-overflow-opencode, combo-context-window-filter, combo-target-resolution-split, 8488-capability-filter-fail-closed) pass after the merge. The dast-smoke red on this PR is also base-inherited — the same failure shows up on several unrelated PRs against the same base today, so that's not something this change introduced.

On the change itself: removing knownContextOverflow.ts and its call sites is clean — I couldn't find any remaining consumer outside the files you touched, and it's a natural extension of the advisory-metadata precedent already shipped in #8944. One thing worth spelling out explicitly once there's a decision: the Auto-strategy branch collapse means compatFilterFailOpen: false no longer produces a hard 400 when every candidate looks too small by estimate — that's really the crux of what #10141 is asking maintainers to weigh in on, so it'd help future readers to link that specific behavior change directly in the PR description.

Since you've marked this Draft on purpose, I'm not pushing for a merge here — just flagging that the code side looks solid and ready to go as soon as maintainers answer #10141's open questions and the branch gets a rebase.

@diegosouzapw diegosouzapw changed the title fix(routing): keep approximate context estimates advisory [defer] fix(routing): keep approximate context estimates advisory Aug 15, 2026
@diegosouzapw diegosouzapw added deferred-v3.8.50 Adiada para o ciclo v3.8.50 (validacao VPS, refactor, ou escopo grande) merge-train-deferred PR ejetada do merge-train — triagem separada and removed merge-train-deferred PR ejetada do merge-train — triagem separada deferred-v3.8.50 Adiada para o ciclo v3.8.50 (validacao VPS, refactor, ou escopo grande) labels Aug 17, 2026
@xz-dev
xz-dev force-pushed the fix/remove-approximate-combo-400 branch from 5bba53d to e2e04b6 Compare August 19, 2026 15:28
diegosouzapw and others added 21 commits August 19, 2026 12:35
…mbleStandalone (cheaper-inference#10776)

Under heavy concurrent build I/O, the bulk .build/next/standalone -> outDir tree
copy can already have carried a prior pass's result into a
NATIVE_ASSET_ENTRIES/EXTRA_MODULE_ENTRIES dest before that entry's own copy runs
(an absolute pnpm-store symlink resolving to the exact same realpath as src, or a
stale node of a different type). fs.cpSync/fs.cp refuse to overwrite either case
even with force:true, throwing ERR_FS_CP_EINVAL ("src and dest cannot be the
same") or ERR_FS_CP_DIR_TO_NON_DIR/ERR_FS_CP_NON_DIR_TO_DIR — non-deterministically
crashing the build:release/build:cli deploy pipeline on whichever entry the race
happened to hit that run.

Adds resolvesToSamePath/clearStaleDest guards to all four copy call sites (the two
sync loops in copyNativeAssetsAndExtraModules, repairEmptyExternalPackageDirs, and
the async syncNativeAssetsToDir/syncExtraModulesToDir twins) so a dest already
pointing at src is skipped and any other stale occupant is cleared before the
fresh copy.

Co-authored-by: Markus Hartung <mail@hartmark.se>
…inference#10722, cheaper-inference#10797) + pre-existing chatBodyAdmission.ts drift

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10627) (cheaper-inference#10709)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…new cert (cheaper-inference#10467) (cheaper-inference#10715)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ted seeds, identity faces (cheaper-inference#10716)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
… Windows (cheaper-inference#10717)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…eaper-inference#10722)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10725)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…through (cheaper-inference#10729)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…cks (cheaper-inference#10681) (cheaper-inference#10730)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ken refresh (cheaper-inference#10732)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…e#10733)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
maxmad64bis and others added 15 commits August 20, 2026 06:30
…e#7992) (cheaper-inference#10779)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…cheaper-inference#10781)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…-inference#10782)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…eaper-inference#10783)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ocks (cheaper-inference#9692) (cheaper-inference#10791)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…e account (cheaper-inference#9708) (cheaper-inference#10792)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10734) (cheaper-inference#10793)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10017 control-line rules (cheaper-inference#10796)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…llision

cheaper-inference#10722 (Token Kiosk) and cheaper-inference#10729 (Cursor) each independently bumped the
gateways/specialty-media APIKEY_PROVIDERS count by one in the same merge-train
batch; neither PR could see the other's bump, so the merged tree landed at 230
while the surviving hardcoded assertion still said 229.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…tics (cheaper-inference#10797)

Merged — the 5 pre-existing tests that broke from this PR's intentional 404→401 remap (single-model no-credentials) are now realigned to the new contract. Thanks!
…fallback and combo hops

cheaper-inference#10792 (cheaper-inference#9708) added a same-account retry for retryable 502/503/504/507 transport
failures, applied uniformly inside handleSingleModelChat. Two other paths call
into the same function recursively/iteratively and each carries its own
documented single-call guarantee that the retry silently broke:

- Emergency fallback (cheaper-inference#1731): exactly one hop to the free fallback model, no
  extra calls against an already-exhausted provider. The retry was doubling
  that call whenever the fallback model itself returned a transient-looking
  status.
- Combo routing: target-level fallback is the combo's own policy (next target,
  not same-account retry). The retry delayed that policy and could surface the
  wrong terminal status when a later combo/global-fallback hop threw.

Both regressions were already covered by existing tests in
chat-route-coverage.test.ts (asserting exact call counts / preserved status) —
confirmed red on the release tip before this fix, green after.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…inference#10826)

Passo 10 of the Radar go-live: publish the OmniRoute catalog export the Radar
server consumes to a stable URL, so the 1 GB-RAM private server pulls it via
RADAR_EXPORT_URL instead of depending on the deploy-time snapshot.

- scripts/release/radar-export.mjs: emits {geradoEm, budgets, totais, registry,
  provenance} from the catalog config modules. Provenance (sourceCommit,
  sourceRef, runUrl, generatedBy) is never fabricated — unknown fields stay null.
- .github/workflows/radar-export.yml: on main catalog changes / manual dispatch /
  weekly, generates the export and clobbers the stable 'radar-export-latest'
  release asset (gh release, GH_TOKEN — checkout persist-credentials:false).
- tests/unit/radar-export.test.mjs: consumer contract (budgets[] non-empty) +
  provenance null-when-unknown + GitHub-env reflection.

Stable URL for RADAR_EXPORT_URL:
https://github.com/diegosouzapw/OmniRoute/releases/download/radar-export-latest/export-omniroute.json

Re-baselines zizmorFindings 190->192 (+2 unpinned-uses @vn, the repo-wide
deliberate convention; artipacked auto-fixed).

Co-authored-by: Xiangzhe <bakryun0718@proton.me>
Today's merge-train batch1 (cheaper-inference#10722 Token Kiosk, cheaper-inference#10729 Cursor) each added one
new APIKEY_PROVIDERS entry, bringing the live provider count to 343 — the
hardcoded '342' in README.md, AGENTS.md, llm.txt, package.json's description,
PROVIDER_REFERENCE.md, and 4 hero/comparison SVGs went stale as a result.
check:docs-counts (STRICT) now passes; regenerated PROVIDER_REFERENCE.md via
npm run gen:provider-reference. The v3.8.50 growth-log table row in README.md
(line 66) is left as-is — it's a historical point-in-time snapshot, not a
live claim.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ference#10723)

Merged — locally validated (72/72 focused tests, typecheck:core clean, all static gates green) after resolving base-drift conflicts (catalog.ts cooperative-yield insertion point, modelMetadataRegistry.ts snapshot-param signature). CI's red checks (Unit Tests fast-path shards, Fast Quality Gates, Docs Gates) are confirmed PRE-EXISTING base-red on the pure release tip — reproduced tests/unit/db-driver-bundling-externals.test.ts, tests/unit/model-catalog-runtime-invalidation.test.ts and others failing identically against origin/release/v3.8.50 with zero PR content, unrelated to this change. Thanks for the design and for absorbing cheaper-inference#10724's value here — great work on both review rounds!
…ence#8367)

Merged — locally validated (30/30 focused tests: chatcore-codex-account-pool, codex-account-cooldown-write, codex-account-pool, providers-route-codex-account-pool, resilience-explain-codex-account, sse-auth-codex-account-pool; typecheck:core clean; file-size/complexity/cognitive-complexity/changelog gates all green). Merges clean against the current release tip with zero conflicts. Great refactor — extracting persistCodexQuotaState out of chatCore.ts into a proper codexAccount/ module with virtual quota pool isolation is a solid improvement. Thanks!
@diegosouzapw
diegosouzapw marked this pull request as ready for review August 20, 2026 12:47
@diegosouzapw
diegosouzapw self-requested a review as a code owner August 20, 2026 12:47
@diegosouzapw
diegosouzapw merged commit 74c5482 into cheaper-inference:release/v3.8.50 Aug 20, 2026
1 of 5 checks passed
diegosouzapw added a commit that referenced this pull request Aug 21, 2026
…t-request input/context check (#10927)

Rescoped from #10606 — see PR body for the full rationale (combo-routing half made moot by #10162's advisory-only architecture, chatCore.ts hard-reject bypass retains real value).

Validated in an isolated worktree boarded onto origin/release/v3.8.50:
- 65/65 focused unit tests pass (chatcore-model-output-cap-wiring + feature-flags-settings).
- check-file-size, check-changelog-integrity: OK.
- typecheck:core: clean.
- check-complexity / check-cognitive-complexity: OK, both under baseline.

Co-authored-by: JxnLexn <10897478+JxnLexn@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…eaper-inference#10162)

Merged — per owner decision on RFC cheaper-inference#10141: gateway token estimates (chars/4) become advisory-only for combo routing/context checks, never a hard pre-dispatch 400. Locally validated (31/31 focused tests across capability filtering, target resolution, and the cheaper-inference#8841 repro), file-size/changelog gates clean, merges conflict-free against the current release tip. Thanks for the well-scoped fix and for flagging this as an RFC first!
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…t-request input/context check (cheaper-inference#10927)

Rescoped from cheaper-inference#10606 — see PR body for the full rationale (combo-routing half made moot by cheaper-inference#10162's advisory-only architecture, chatCore.ts hard-reject bypass retains real value).

Validated in an isolated worktree boarded onto origin/release/v3.8.50:
- 65/65 focused unit tests pass (chatcore-model-output-cap-wiring + feature-flags-settings).
- check-file-size, check-changelog-integrity: OK.
- typecheck:core: clean.
- check-complexity / check-cognitive-complexity: OK, both under baseline.

Co-authored-by: JxnLexn <10897478+JxnLexn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

enhancement(routing): RFC — separate approximate token estimates from hard admission decisions