Repository navigation
fix(responses): replace synthetic reasoning keepalive - #10806
Merged
diegosouzapw merged 7 commits intoAug 20, 2026
Merged
diegosouzapw merged 7 commits into
diegosouzapw merged 7 commits into
Conversation
Use the existing response.in_progress data event for both startup and recurring early keepalives on /v1/responses. Remove the visible placeholder reasoning lifecycle and its now-unused output-index scaffolding.
xz-dev
force-pushed
the
fix/responses-neutral-keepalive
branch
from
August 20, 2026 10:15
a5f57e4 to
75ee9a9
Compare
…fallback and combo hops cheaper-inference#10792 (cheaper-inference#9708) added a same-account retry for retryable 502/503/504/507 transport failures, applied uniformly inside handleSingleModelChat. Two other paths call into the same function recursively/iteratively and each carries its own documented single-call guarantee that the retry silently broke: - Emergency fallback (cheaper-inference#1731): exactly one hop to the free fallback model, no extra calls against an already-exhausted provider. The retry was doubling that call whenever the fallback model itself returned a transient-looking status. - Combo routing: target-level fallback is the combo's own policy (next target, not same-account retry). The retry delayed that policy and could surface the wrong terminal status when a later combo/global-fallback hop threw. Both regressions were already covered by existing tests in chat-route-coverage.test.ts (asserting exact call counts / preserved status) — confirmed red on the release tip before this fix, green after. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…inference#10826) Passo 10 of the Radar go-live: publish the OmniRoute catalog export the Radar server consumes to a stable URL, so the 1 GB-RAM private server pulls it via RADAR_EXPORT_URL instead of depending on the deploy-time snapshot. - scripts/release/radar-export.mjs: emits {geradoEm, budgets, totais, registry, provenance} from the catalog config modules. Provenance (sourceCommit, sourceRef, runUrl, generatedBy) is never fabricated — unknown fields stay null. - .github/workflows/radar-export.yml: on main catalog changes / manual dispatch / weekly, generates the export and clobbers the stable 'radar-export-latest' release asset (gh release, GH_TOKEN — checkout persist-credentials:false). - tests/unit/radar-export.test.mjs: consumer contract (budgets[] non-empty) + provenance null-when-unknown + GitHub-env reflection. Stable URL for RADAR_EXPORT_URL: https://github.com/diegosouzapw/OmniRoute/releases/download/radar-export-latest/export-omniroute.json Re-baselines zizmorFindings 190->192 (+2 unpinned-uses @vn, the repo-wide deliberate convention; artipacked auto-fixed). Co-authored-by: Xiangzhe <bakryun0718@proton.me>
Today's merge-train batch1 (cheaper-inference#10722 Token Kiosk, cheaper-inference#10729 Cursor) each added one new APIKEY_PROVIDERS entry, bringing the live provider count to 343 — the hardcoded '342' in README.md, AGENTS.md, llm.txt, package.json's description, PROVIDER_REFERENCE.md, and 4 hero/comparison SVGs went stale as a result. check:docs-counts (STRICT) now passes; regenerated PROVIDER_REFERENCE.md via npm run gen:provider-reference. The v3.8.50 growth-log table row in README.md (line 66) is left as-is — it's a historical point-in-time snapshot, not a live claim. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ference#10723) Merged — locally validated (72/72 focused tests, typecheck:core clean, all static gates green) after resolving base-drift conflicts (catalog.ts cooperative-yield insertion point, modelMetadataRegistry.ts snapshot-param signature). CI's red checks (Unit Tests fast-path shards, Fast Quality Gates, Docs Gates) are confirmed PRE-EXISTING base-red on the pure release tip — reproduced tests/unit/db-driver-bundling-externals.test.ts, tests/unit/model-catalog-runtime-invalidation.test.ts and others failing identically against origin/release/v3.8.50 with zero PR content, unrelated to this change. Thanks for the design and for absorbing cheaper-inference#10724's value here — great work on both review rounds!
…ence#8367) Merged — locally validated (30/30 focused tests: chatcore-codex-account-pool, codex-account-cooldown-write, codex-account-pool, providers-route-codex-account-pool, resilience-explain-codex-account, sse-auth-codex-account-pool; typecheck:core clean; file-size/complexity/cognitive-complexity/changelog gates all green). Merges clean against the current release tip with zero conflicts. Great refactor — extracting persistCodexQuotaState out of chatCore.ts into a proper codexAccount/ module with virtual quota pool isolation is a solid improvement. Thanks!
diegosouzapw
marked this pull request as ready for review
August 20, 2026 12:48
diegosouzapw
merged commit Aug 20, 2026
62f6e87
into
cheaper-inference:release/v3.8.50
2 of 5 checks passed
This was referenced Aug 20, 2026
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…nce#10806) Merged — locally validated (61/61 focused tests: early-stream-keepalive, chat-body-admission, responses-parse-once-4041, responses-route-early-keepalive-wiring; file-size/changelog gates clean, merges conflict-free against the current release tip). Good catch replacing the synthetic reasoning placeholder with a real response.in_progress bookkeeping event — keeps event-level watchdogs (Codex etc.) happy without any replayable fake reasoning content. Thanks!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
/v1/responsesopens an early SSE stream when upstream routing, cooldown waits, or first-byte readiness exceed the route threshold. Its immediate startup frame currently creates a complete synthetic reasoning output item containing✨.Although the item is now structurally closed, downstream clients can still display or persist the placeholder as reasoning, and may replay it in a later request. An SSE comment is content-neutral but does not reset clients such as Codex that apply their idle timeout to the next parsed EventSource event rather than raw bytes.
Fix
Reuse OmniRoute's existing Responses heartbeat shape from
sseHeartbeat.ts:The Responses route now supplies that frame for both:
response.in_progressis a parsed Responses bookkeeping event, so it keeps event-level watchdogs active without creating an output item, reasoning summary, visible text, output index, or replayable reasoning signature.The change removes the obsolete synthetic reasoning frame and the output-index stack/test scaffolding that existed only to keep that placeholder lifecycle well formed. Fast-path responses, HTTP statuses, combo/readiness execution, thresholds, error framing, correlation logging, cancellation, and real body forwarding remain unchanged.
Related history
response.in_progressheartbeat.Verification
node --test tests/unit/responses-route-early-keepalive-wiring.test.ts— 1 passednode --import tsx/esm --test tests/unit/early-stream-keepalive.test.ts tests/unit/attempt-logging-early-keepalive-merge.test.ts— 22 passedOMNIROUTE_SSE_COMMENTS=on node --import tsx/esm --test tests/unit/sse-heartbeat.test.ts— 9 passednpm run check:cycles— passed, 424 files checkedgit diff --check— passednpm run typecheck:corecurrently reports only pre-existingomniglyphdependency API drift in untouched compression modules; there are zero diagnostics in files changed by this PR.Review notes
This is intentionally a Draft PR for maintainer review. It has not been deployed to production and has not been smoke-tested against external clients in this branch.