Skip to content

fix(responses-continuation): chain off the effective post-reconstruction input, not the pre-reconstruction client bytes - #12641

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
hartmark:fix/continuation-effective-input-chain
Sep 4, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
hartmark:fix/continuation-effective-input-chain

Conversation

@hartmark

@hartmark hartmark commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

previous_response_id continuation (OmniRoute-native virtualization,
src/lib/db/responsesContinuationStore.ts) silently corrupts conversation
history a few turns into any tool-heavy exchange: the reconstructed request
degrades to a handful of bare tool-call items with no leading system/user
message, which the upstream provider then rejects outright.

Why This Change Was Made

resolvePreviousResponseState() reconstructs a continued turn's full history
by reading the prior call-log's clientRawRequest.body.input. But
clientRawRequest is deliberately captured before chat.ts's own
previous_response_id reconstruction runs
(captureDeferredClientRawBody's whole point — it has to preserve the raw
client bytes for audit/guardrail purposes, not what OmniRoute rewrote the
request into).

For a prior turn that was itself a continuation, body.input is just the
client's own trimmed delta (relying on OmniRoute to have already
reconstructed history server-side) — not the full input that actually
dispatched. Chaining a later continuation off that instead of the real
effective input compounds: each hop's stored "input" is only the prior hop's
already-trimmed delta, so a few turns into a tool-heavy conversation the
reconstruction degrades to a handful of bare tool items with nothing before
them.

Live incident (2026-09-03): a real production session's 3rd turn failed
with a Gemini 400 — Please ensure that function call turn comes immediately after a user turn or after a function response turn — because OmniRoute
sent exactly that malformed 3-item request
(function_call_output, function_call, function_call_output, no leading
system/user message) upstream.

Fix

Capture the actual effective input a request dispatches with — after
previous_response_id reconstruction runs, whether or not it applied this
specific turn — as a new sibling field on clientRawRequest
(effectiveInput), threaded through the existing logClientRawRequest /
logClientRawRequestRedacted call chain. No new parameter threading needed
anywhere else in the request pipeline.

resolvePreviousResponseState now chains off effectiveInput, falling back
to body.input only for artifacts logged before this field existed (no
compat migration needed — self-healing: any conversation naturally recovers
correct reconstruction the moment one of its turns is logged post-fix).

User Impact

  • Multi-turn continuation for previous_response_id no longer degrades or
    corrupts history a few turns into a tool-heavy conversation.
  • No behavior change for a turn that was never itself a continuation (its
    effectiveInput always equals body.input in that case).
  • No config surface added.

Evidence

tests/unit/responses-continuation-store.test.ts:

  • New case "chains off effectiveInput, not the pre-reconstruction
    clientRawRequest.body"
    — proves a continued turn correctly reconstructs
    the full 4-item history from effectiveInput instead of the 1-item raw
    delta. Fails on pre-fix code (returns just the delta item); passes
    after.
  • New case "falls back to clientRawRequest.body.input when effectiveInput
    is absent (pre-fix artifacts)"
    — proves legacy artifacts without the new
    field still resolve exactly as before (no regression for existing data).

Full file: 13/13 passing. tsc --pretty false -p tsconfig.typecheck-core.json
clean on all 4 touched production files.

…ion input, not the pre-reconstruction client bytes

Root cause: resolvePreviousResponseState() reconstructs a continued turn's
full history by reading a prior call-log's clientRawRequest.body.input. But
clientRawRequest is deliberately captured BEFORE chat.ts's own
previous_response_id reconstruction runs (captureDeferredClientRawBody's
whole point -- it must preserve the raw client bytes for audit/guardrail
purposes, not the request OmniRoute rewrote it into). For a prior turn that
was itself a continuation, body.input is just the client's own trimmed
delta, not the full input that actually dispatched. Chaining a later
continuation off that instead of the real effective input compounds:
each hop's stored "input" is only the prior hop's already-trimmed delta,
so a few turns into a tool-heavy conversation the reconstruction degrades
to a handful of bare tool items with no leading system/user message.

Live incident (2026-09-03): Ping's 3rd turn in a session failed with a
Gemini 400 -- "Please ensure that function call turn comes immediately
after a user turn or after a function response turn" -- because OmniRoute
sent exactly that malformed 3-item request
(function_call_output, function_call, function_call_output, nothing before
it) upstream.

Fix: capture the actual effective `input` a request dispatches with --
after previous_response_id reconstruction runs, whether or not it applied
this specific turn -- as a new sibling field on clientRawRequest
(`effectiveInput`), threaded through the existing logClientRawRequest /
logClientRawRequestRedacted call chain (no new parameter threading needed
elsewhere). resolvePreviousResponseState now chains off effectiveInput,
falling back to body.input only for artifacts logged before this field
existed.

Tests: tests/unit/responses-continuation-store.test.ts -- new case proves a
continued turn correctly reconstructs full history from effectiveInput
(fails on pre-fix code: returns just the 1-item delta instead of the
4-item reconstructed history); new case proves the body.input fallback
still works for legacy artifacts lacking the field. Full file: 13/13
passing. tsc --pretty false -p tsconfig.typecheck-core.json clean.
hartmark added a commit to hartmark/OmniRoute that referenced this pull request Sep 3, 2026
…he effective post-reconstruction input, not the pre-reconstruction client bytes) into dev/omniroute-dev-combined
hartmark added a commit to hartmark/OmniRoute that referenced this pull request Sep 3, 2026
…he effective post-reconstruction input, not the pre-reconstruction client bytes) into dev/omniroute-dev-combined
hartmark added a commit to hartmark/OmniRoute that referenced this pull request Sep 3, 2026
…he effective post-reconstruction input, not the pre-reconstruction client bytes) into dev/omniroute-dev-combined
hartmark added a commit to hartmark/OmniRoute that referenced this pull request Sep 3, 2026
…he effective post-reconstruction input, not the pre-reconstruction client bytes) into dev/omniroute-dev-combined
@diegosouzapw
diegosouzapw merged commit 74c2d26 into diegosouzapw:release/v3.8.51 Sep 4, 2026
14 of 16 checks passed
diegosouzapw pushed a commit that referenced this pull request Sep 4, 2026
…mbo target retries (#12650)

Validado em lote numa worktree combinada com os 3 PRs desta leva sobre o tip de `release/v3.8.51`: os três boardaram sem conflito, `typecheck:core` limpo e **22/22** nos arquivos de teste que trazem.

O crescimento de `src/sse/handlers/chat.ts` (2450 → 2454) é do #12641 e vai num PR de rebaseline próprio.

Obrigado, @hartmark.
diegosouzapw added a commit that referenced this pull request Sep 4, 2026
…stence (#12680)

Rebaseline medido no tip com o #12641 mergeado.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ion input, not the pre-reconstruction client bytes (diegosouzapw#12641)

Validado em lote numa worktree combinada com os 3 PRs desta leva sobre o tip de `release/v3.8.51`: os três boardaram sem conflito, `typecheck:core` limpo e **22/22** nos arquivos de teste que trazem.

O crescimento de `src/sse/handlers/chat.ts` (2450 → 2454) é do diegosouzapw#12641 e vai num PR de rebaseline próprio.

Obrigado, @hartmark.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…mbo target retries (diegosouzapw#12650)

Validado em lote numa worktree combinada com os 3 PRs desta leva sobre o tip de `release/v3.8.51`: os três boardaram sem conflito, `typecheck:core` limpo e **22/22** nos arquivos de teste que trazem.

O crescimento de `src/sse/handlers/chat.ts` (2450 → 2454) é do diegosouzapw#12641 e vai num PR de rebaseline próprio.

Obrigado, @hartmark.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants