Skip to content

fix(responses): replace synthetic reasoning keepalive - #10806

Merged
diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.50from
xz-dev:fix/responses-neutral-keepalive
Aug 20, 2026
Merged

diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.50from
xz-dev:fix/responses-neutral-keepalive

Conversation

@xz-dev

@xz-dev xz-dev commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Problem

/v1/responses opens an early SSE stream when upstream routing, cooldown waits, or first-byte readiness exceed the route threshold. Its immediate startup frame currently creates a complete synthetic reasoning output item containing ✨.

Although the item is now structurally closed, downstream clients can still display or persist the placeholder as reasoning, and may replay it in a later request. An SSE comment is content-neutral but does not reset clients such as Codex that apply their idle timeout to the next parsed EventSource event rather than raw bytes.

Fix

Reuse OmniRoute's existing Responses heartbeat shape from sseHeartbeat.ts:

data: {"type":"response.in_progress"}

The Responses route now supplies that frame for both:

  • the immediate slow-path startup frame; and
  • recurring early keepalive ticks until the real handler response is available.

response.in_progress is a parsed Responses bookkeeping event, so it keeps event-level watchdogs active without creating an output item, reasoning summary, visible text, output index, or replayable reasoning signature.

The change removes the obsolete synthetic reasoning frame and the output-index stack/test scaffolding that existed only to keep that placeholder lifecycle well formed. Fast-path responses, HTTP statuses, combo/readiness execution, thresholds, error framing, correlation logging, cancellation, and real body forwarding remain unchanged.

Related history

Verification

  • node --test tests/unit/responses-route-early-keepalive-wiring.test.ts — 1 passed
  • node --import tsx/esm --test tests/unit/early-stream-keepalive.test.ts tests/unit/attempt-logging-early-keepalive-merge.test.ts — 22 passed
  • OMNIROUTE_SSE_COMMENTS=on node --import tsx/esm --test tests/unit/sse-heartbeat.test.ts — 9 passed
  • npm run check:cycles — passed, 424 files checked
  • Focused ESLint — passed
  • Focused Prettier check — passed
  • git diff --check — passed
  • Independent final review — approved with no remaining findings

npm run typecheck:core currently reports only pre-existing omniglyph dependency API drift in untouched compression modules; there are zero diagnostics in files changed by this PR.

Review notes

This is intentionally a Draft PR for maintainer review. It has not been deployed to production and has not been smoke-tested against external clients in this branch.

Use the existing response.in_progress data event for both startup and recurring early keepalives on /v1/responses. Remove the visible placeholder reasoning lifecycle and its now-unused output-index scaffolding.
@xz-dev
xz-dev force-pushed the fix/responses-neutral-keepalive branch from a5f57e4 to 75ee9a9 Compare August 20, 2026 10:15
hartmark and others added 5 commits August 20, 2026 08:08
…fallback and combo hops

cheaper-inference#10792 (cheaper-inference#9708) added a same-account retry for retryable 502/503/504/507 transport
failures, applied uniformly inside handleSingleModelChat. Two other paths call
into the same function recursively/iteratively and each carries its own
documented single-call guarantee that the retry silently broke:

- Emergency fallback (cheaper-inference#1731): exactly one hop to the free fallback model, no
  extra calls against an already-exhausted provider. The retry was doubling
  that call whenever the fallback model itself returned a transient-looking
  status.
- Combo routing: target-level fallback is the combo's own policy (next target,
  not same-account retry). The retry delayed that policy and could surface the
  wrong terminal status when a later combo/global-fallback hop threw.

Both regressions were already covered by existing tests in
chat-route-coverage.test.ts (asserting exact call counts / preserved status) —
confirmed red on the release tip before this fix, green after.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…inference#10826)

Passo 10 of the Radar go-live: publish the OmniRoute catalog export the Radar
server consumes to a stable URL, so the 1 GB-RAM private server pulls it via
RADAR_EXPORT_URL instead of depending on the deploy-time snapshot.

- scripts/release/radar-export.mjs: emits {geradoEm, budgets, totais, registry,
  provenance} from the catalog config modules. Provenance (sourceCommit,
  sourceRef, runUrl, generatedBy) is never fabricated — unknown fields stay null.
- .github/workflows/radar-export.yml: on main catalog changes / manual dispatch /
  weekly, generates the export and clobbers the stable 'radar-export-latest'
  release asset (gh release, GH_TOKEN — checkout persist-credentials:false).
- tests/unit/radar-export.test.mjs: consumer contract (budgets[] non-empty) +
  provenance null-when-unknown + GitHub-env reflection.

Stable URL for RADAR_EXPORT_URL:
https://github.com/diegosouzapw/OmniRoute/releases/download/radar-export-latest/export-omniroute.json

Re-baselines zizmorFindings 190->192 (+2 unpinned-uses @vn, the repo-wide
deliberate convention; artipacked auto-fixed).

Co-authored-by: Xiangzhe <bakryun0718@proton.me>
Today's merge-train batch1 (cheaper-inference#10722 Token Kiosk, cheaper-inference#10729 Cursor) each added one
new APIKEY_PROVIDERS entry, bringing the live provider count to 343 — the
hardcoded '342' in README.md, AGENTS.md, llm.txt, package.json's description,
PROVIDER_REFERENCE.md, and 4 hero/comparison SVGs went stale as a result.
check:docs-counts (STRICT) now passes; regenerated PROVIDER_REFERENCE.md via
npm run gen:provider-reference. The v3.8.50 growth-log table row in README.md
(line 66) is left as-is — it's a historical point-in-time snapshot, not a
live claim.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ference#10723)

Merged — locally validated (72/72 focused tests, typecheck:core clean, all static gates green) after resolving base-drift conflicts (catalog.ts cooperative-yield insertion point, modelMetadataRegistry.ts snapshot-param signature). CI's red checks (Unit Tests fast-path shards, Fast Quality Gates, Docs Gates) are confirmed PRE-EXISTING base-red on the pure release tip — reproduced tests/unit/db-driver-bundling-externals.test.ts, tests/unit/model-catalog-runtime-invalidation.test.ts and others failing identically against origin/release/v3.8.50 with zero PR content, unrelated to this change. Thanks for the design and for absorbing cheaper-inference#10724's value here — great work on both review rounds!
…ence#8367)

Merged — locally validated (30/30 focused tests: chatcore-codex-account-pool, codex-account-cooldown-write, codex-account-pool, providers-route-codex-account-pool, resilience-explain-codex-account, sse-auth-codex-account-pool; typecheck:core clean; file-size/complexity/cognitive-complexity/changelog gates all green). Merges clean against the current release tip with zero conflicts. Great refactor — extracting persistCodexQuotaState out of chatCore.ts into a proper codexAccount/ module with virtual quota pool isolation is a solid improvement. Thanks!
@diegosouzapw
diegosouzapw marked this pull request as ready for review August 20, 2026 12:48
@diegosouzapw
diegosouzapw self-requested a review as a code owner August 20, 2026 12:48
@diegosouzapw
diegosouzapw merged commit 62f6e87 into cheaper-inference:release/v3.8.50 Aug 20, 2026
2 of 5 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…nce#10806)

Merged — locally validated (61/61 focused tests: early-stream-keepalive, chat-body-admission, responses-parse-once-4041, responses-route-early-keepalive-wiring; file-size/changelog gates clean, merges conflict-free against the current release tip). Good catch replacing the synthetic reasoning placeholder with a real response.in_progress bookkeeping event — keeps event-level watchdogs (Codex etc.) happy without any replayable fake reasoning content. Thanks!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants