Repository navigation
[defer] fix(routing): keep approximate context estimates advisory - #10162
diegosouzapw merged 46 commits into
Conversation
|
Thanks for the thorough write-up here, and for being upfront that this is a Draft tied to #10141's design questions. I went through the diff and validated it against the current On the change itself: removing Since you've marked this Draft on purpose, I'm not pushing for a merge here — just flagging that the code side looks solid and ready to go as soon as maintainers answer #10141's open questions and the branch gets a rebase. |
5bba53d to
e2e04b6
Compare
…sion Bridge (cheaper-inference#10702) (cheaper-inference#10760) Co-authored-by: Markus Hartung <mail@hartmark.se>
…cheaper-inference#10577) (cheaper-inference#10761) Co-authored-by: Markus Hartung <mail@hartmark.se>
…eaper-inference#10479) (cheaper-inference#10762) Co-authored-by: Markus Hartung <mail@hartmark.se>
…sVision (cheaper-inference#10703) (cheaper-inference#10763) Co-authored-by: Markus Hartung <mail@hartmark.se>
…lization (cheaper-inference#9144) (cheaper-inference#10764) Co-authored-by: Markus Hartung <mail@hartmark.se>
…mbleStandalone (cheaper-inference#10776) Under heavy concurrent build I/O, the bulk .build/next/standalone -> outDir tree copy can already have carried a prior pass's result into a NATIVE_ASSET_ENTRIES/EXTRA_MODULE_ENTRIES dest before that entry's own copy runs (an absolute pnpm-store symlink resolving to the exact same realpath as src, or a stale node of a different type). fs.cpSync/fs.cp refuse to overwrite either case even with force:true, throwing ERR_FS_CP_EINVAL ("src and dest cannot be the same") or ERR_FS_CP_DIR_TO_NON_DIR/ERR_FS_CP_NON_DIR_TO_DIR — non-deterministically crashing the build:release/build:cli deploy pipeline on whichever entry the race happened to hit that run. Adds resolvesToSamePath/clearStaleDest guards to all four copy call sites (the two sync loops in copyNativeAssetsAndExtraModules, repairEmptyExternalPackageDirs, and the async syncNativeAssetsToDir/syncExtraModulesToDir twins) so a dest already pointing at src is skipped and any other stale occupant is cleared before the fresh copy. Co-authored-by: Markus Hartung <mail@hartmark.se>
…inference#10722, cheaper-inference#10797) + pre-existing chatBodyAdmission.ts drift Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10627) (cheaper-inference#10709) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…new cert (cheaper-inference#10467) (cheaper-inference#10715) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ted seeds, identity faces (cheaper-inference#10716) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
… Windows (cheaper-inference#10717) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…eaper-inference#10722) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10725) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…through (cheaper-inference#10729) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…cks (cheaper-inference#10681) (cheaper-inference#10730) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ken refresh (cheaper-inference#10732) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…e#10733) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…e#7992) (cheaper-inference#10779) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…cheaper-inference#10781) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…-inference#10782) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…eaper-inference#10783) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ocks (cheaper-inference#9692) (cheaper-inference#10791) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…e account (cheaper-inference#9708) (cheaper-inference#10792) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10734) (cheaper-inference#10793) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…heaper-inference#10017 control-line rules (cheaper-inference#10796) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…llision cheaper-inference#10722 (Token Kiosk) and cheaper-inference#10729 (Cursor) each independently bumped the gateways/specialty-media APIKEY_PROVIDERS count by one in the same merge-train batch; neither PR could see the other's bump, so the merged tree landed at 230 while the surviving hardcoded assertion still said 229. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…tics (cheaper-inference#10797) Merged — the 5 pre-existing tests that broke from this PR's intentional 404→401 remap (single-model no-credentials) are now realigned to the new contract. Thanks!
…fallback and combo hops cheaper-inference#10792 (cheaper-inference#9708) added a same-account retry for retryable 502/503/504/507 transport failures, applied uniformly inside handleSingleModelChat. Two other paths call into the same function recursively/iteratively and each carries its own documented single-call guarantee that the retry silently broke: - Emergency fallback (cheaper-inference#1731): exactly one hop to the free fallback model, no extra calls against an already-exhausted provider. The retry was doubling that call whenever the fallback model itself returned a transient-looking status. - Combo routing: target-level fallback is the combo's own policy (next target, not same-account retry). The retry delayed that policy and could surface the wrong terminal status when a later combo/global-fallback hop threw. Both regressions were already covered by existing tests in chat-route-coverage.test.ts (asserting exact call counts / preserved status) — confirmed red on the release tip before this fix, green after. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…inference#10826) Passo 10 of the Radar go-live: publish the OmniRoute catalog export the Radar server consumes to a stable URL, so the 1 GB-RAM private server pulls it via RADAR_EXPORT_URL instead of depending on the deploy-time snapshot. - scripts/release/radar-export.mjs: emits {geradoEm, budgets, totais, registry, provenance} from the catalog config modules. Provenance (sourceCommit, sourceRef, runUrl, generatedBy) is never fabricated — unknown fields stay null. - .github/workflows/radar-export.yml: on main catalog changes / manual dispatch / weekly, generates the export and clobbers the stable 'radar-export-latest' release asset (gh release, GH_TOKEN — checkout persist-credentials:false). - tests/unit/radar-export.test.mjs: consumer contract (budgets[] non-empty) + provenance null-when-unknown + GitHub-env reflection. Stable URL for RADAR_EXPORT_URL: https://github.com/diegosouzapw/OmniRoute/releases/download/radar-export-latest/export-omniroute.json Re-baselines zizmorFindings 190->192 (+2 unpinned-uses @vn, the repo-wide deliberate convention; artipacked auto-fixed). Co-authored-by: Xiangzhe <bakryun0718@proton.me>
Today's merge-train batch1 (cheaper-inference#10722 Token Kiosk, cheaper-inference#10729 Cursor) each added one new APIKEY_PROVIDERS entry, bringing the live provider count to 343 — the hardcoded '342' in README.md, AGENTS.md, llm.txt, package.json's description, PROVIDER_REFERENCE.md, and 4 hero/comparison SVGs went stale as a result. check:docs-counts (STRICT) now passes; regenerated PROVIDER_REFERENCE.md via npm run gen:provider-reference. The v3.8.50 growth-log table row in README.md (line 66) is left as-is — it's a historical point-in-time snapshot, not a live claim. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ference#10723) Merged — locally validated (72/72 focused tests, typecheck:core clean, all static gates green) after resolving base-drift conflicts (catalog.ts cooperative-yield insertion point, modelMetadataRegistry.ts snapshot-param signature). CI's red checks (Unit Tests fast-path shards, Fast Quality Gates, Docs Gates) are confirmed PRE-EXISTING base-red on the pure release tip — reproduced tests/unit/db-driver-bundling-externals.test.ts, tests/unit/model-catalog-runtime-invalidation.test.ts and others failing identically against origin/release/v3.8.50 with zero PR content, unrelated to this change. Thanks for the design and for absorbing cheaper-inference#10724's value here — great work on both review rounds!
…ence#8367) Merged — locally validated (30/30 focused tests: chatcore-codex-account-pool, codex-account-cooldown-write, codex-account-pool, providers-route-codex-account-pool, resilience-explain-codex-account, sse-auth-codex-account-pool; typecheck:core clean; file-size/complexity/cognitive-complexity/changelog gates all green). Merges clean against the current release tip with zero conflicts. Great refactor — extracting persistCodexQuotaState out of chatCore.ts into a proper codexAccount/ module with virtual quota pool isolation is a solid improvement. Thanks!
74c5482
into
cheaper-inference:release/v3.8.50
…t-request input/context check (#10927) Rescoped from #10606 — see PR body for the full rationale (combo-routing half made moot by #10162's advisory-only architecture, chatCore.ts hard-reject bypass retains real value). Validated in an isolated worktree boarded onto origin/release/v3.8.50: - 65/65 focused unit tests pass (chatcore-model-output-cap-wiring + feature-flags-settings). - check-file-size, check-changelog-integrity: OK. - typecheck:core: clean. - check-complexity / check-cognitive-complexity: OK, both under baseline. Co-authored-by: JxnLexn <10897478+JxnLexn@users.noreply.github.com>
…eaper-inference#10162) Merged — per owner decision on RFC cheaper-inference#10141: gateway token estimates (chars/4) become advisory-only for combo routing/context checks, never a hard pre-dispatch 400. Locally validated (31/31 focused tests across capability filtering, target resolution, and the cheaper-inference#8841 repro), file-size/changelog gates clean, merges conflict-free against the current release tip. Thanks for the well-scoped fix and for flagging this as an RFC first!
…t-request input/context check (cheaper-inference#10927) Rescoped from cheaper-inference#10606 — see PR body for the full rationale (combo-routing half made moot by cheaper-inference#10162's advisory-only architecture, chatCore.ts hard-reject bypass retains real value). Validated in an isolated worktree boarded onto origin/release/v3.8.50: - 65/65 focused unit tests pass (chatcore-model-output-cap-wiring + feature-flags-settings). - check-file-size, check-changelog-integrity: OK. - typecheck:core: clean. - check-complexity / check-cognitive-complexity: OK, both under baseline. Co-authored-by: JxnLexn <10897478+JxnLexn@users.noreply.github.com>
Summary
400 context_length_exceededresponses that treated approximate gateway token estimates as authoritative.context_length_exceededshort-circuiting.Related Issues
Validation
Choose the change type and focused loop from the
Contribution Golden Path. The full unit suite,
Vitest, the 60% coverage gate, and the production build all run in CI on this PR (#8329):
npm run lint— changed files pass; the full command is base-red in three unchanged files, documented belowFocused validation:
git diff --checkpasses.npm run typecheck:corereports 12 pre-existing diagnostics in unchanged files and none in changed files.Tests Added Or Updated
tests/unit/8488-capability-filter-fail-closed.test.tstests/unit/combo-context-window-filter.test.tstests/unit/combo-target-resolution-split.test.tstests/unit/repro-8841-context-overflow-opencode.test.tsCoverage Notes
The updated tests exercise the public
handleComboChat()seam for priority, round-robin, and Auto dispatch when approximate estimates exceed every catalog hint. They also verify that a real upstream overflow still short-circuits a homogeneous same-model pool while a heterogeneous pool may try a larger-context target.Reviewer Notes
hasEstimableContenthelper only moved to an existing imported module whenknownContextOverflow.tswas deleted.143_api_key_cache_default_mode.sqland143_job_registry.sql). Focused tests were run with the latter temporarily unavailable and restored immediately; this PR does not modify or repair either migration.npm run lintis base-red only in unchangedsrc/shared/components/modelSelectModalHelpers.ts,tests/unit/combo/image-combo.test.ts, andtests/unit/db-adapters/driverFactory.test.ts.changelog.d/fixes/10162-approximate-combo-context-advisory.mdfragment passes the fragment validator in isolation. The full changelog-integrity command is base-red because two unchanged feature fragments do not start with Markdown bullets.