Repository navigation
fix(thinking): parse/scrub DSML tool-call markers and recognize adaptive thinking - #12905
Merged
diegosouzapw merged 9 commits intoSep 17, 2026
Conversation
added 3 commits
September 7, 2026 13:26
…ll markers
Two defects combined to break DeepSeek-V4-Flash turns and raise 502
empty_response on Claude Code autocompact.
Defect 1 — DSML tool-call markers leaked as visible content:
DeepSeek-V4-Flash occasionally emits tool calls in a non-standard DSML
text format using full-width pipes instead of the OpenAI tool_calls JSON.
Two shapes appear in production call logs:
- complete block: <｜DSML｜:Read><path>...</path></｜DSML｜:Read>
- stray closers (truncated call): </｜DSML｜parameter></｜DSML｜invoke>
</｜DSML｜tool_calls>, sometimes trailing a system-prompt echo
The openai-compatible path never parsed these, so the markers leaked to
the client as visible content and the turn ended incomplete.
Fix: add open-sse/utils/dsmlToolCalls.ts — parseDsmlToolCalls() converts
complete DSML blocks into OpenAI tool_calls and strips stray closing
markers from content (streaming-safe via a holdback for partial openers).
Wire it into the response translator before extractXmlInvokeBlocks so
DSML and XML invoke tool calls share the same pending queue.
Defect 2 — adaptive thinking silently suppressed:
A prior inline === 'enabled' check on body.thinking.type silently
suppressed adaptive (the intent Claude Code actually sends), so
reasoning was dropped. The model then emitted DSML tool-call markers
as plain text, producing an incomplete stop finish. Fix: use
hasActiveClaudeThinking() (which recognizes enabled AND adaptive) to
set requestedThinking, thread it through stream.ts and translator
state, and gate thinking block emission on state.requestedThinking
so upstream reasoning_content only relays when the client opted in.
Tests: 29/29 (6 dsml-tool-calls, 5 thinking-active-claude-adapter,
3 translator-resp-dsml-integration, 15 translator-resp-openai-to-claude
incl. requestedThinking suppression regression). typecheck:core clean.
…is/summary blocks DeepSeek-V4 and similar models echo the OMNIROUTE_SYSTEM_INSTRUCTION_APPEND directive (appended to the system tail by claude-to-openai.ts) and whole chunks of the system prompt (<analysis>/<system-reminder>/<summary> blocks, prose reproductions of the superpowers skill section) verbatim at the START of their reply — the 'system message leak' persisting after the request-side fix. Add two streaming-safe preamble strippers in directivePreambleStripper.ts: - createDirectivePreambleStripper(directive): drops a leading reproduction of the exact configured directive across arbitrary SSE chunk boundaries. - createSystemPreambleStripper(): removes <analysis>/<system-reminder>/ <summary> echo blocks and known prose heads (Phase B) from the very start of a stream, only while the stream is still a preamble. Wire both into openai-to-claude.ts content-delta path: chain the exact-directive stripper then the system-echo stripper before DSML/XML-invoke parsing, so a leading system echo is dropped before it reaches the client. Preserve large blocks (>= SYSTEM_ECHO_THRESHOLD=1000 chars) and blocks with no trailing content — these are the model's real response (e.g. a Claude Code autocompact summary), not a short system-echo. Stops the autocompact empty-response regression where a whole-summary <analysis> block was stripped to empty (3a8515). Regression: origin's markdown-boundary feature (bufferedPrefix / splitMarkdownBoundary, commit 1b39873) is preserved — preamble strip runs before the markdown buffer rehydration, and the scrubbed content flows into the existing DSML/XML-invoke/markdown pipeline unchanged. TDD: tests/unit/directive-preamble-strip.test.ts (7 cases), system-preamble-strip.test.ts (12 cases incl. 3a8515 regression), system-preamble-wiring.test.ts (3 integration cases); group F regression 24/24 green; typecheck:core 0 errors.
…block for reasoning-only responses
Reasoning-content (thinking) blocks were emitted unconditionally to
Claude-format clients, leaking reasoning to thinking-opt-out clients
(Claude Code sends thinking:{type:"disabled"}) — the operator reported
'reasoning is exposed'. On reasoning-only upstream responses (GLM-5.2
autocompact pattern), the unconditional thinking block also caused either
a 502 'no content block' at flush, or — after a text-block fallback — an
autocompact 'empty response' rejection that looped the session forever.
Streaming translator (openai-to-claude.ts):
- Compute hasReasoning outside the emission gate; accumulate into
state._reasoningAccum always (so fix B can fire).
- Gate only the thinking-block EMISSION on requestedThinking === true.
- FIX B at finish: when no text block was started and requestedThinking
!== true, synthesize a text content block from _reasoningAccum so
autocompact can extract the summary (no 502, compact applies).
- Skip fix B when requestedThinking === true to avoid double-exposure
(thinking block + text block both carrying reasoning).
Non-streaming translator (responseTranslator.ts):
- Thread requestedThinking through translateNonStreamingResponse into
convertOpenAINonStreamingToClaude.
- suppressThinking = requestedThinking === false: drop the thinking block
when content is present (no leak); relay reasoning as a text block when
the response is reasoning-only (no 502). requestedThinking === undefined
keeps the legacy 'always a thinking block' relay.
chatCore.ts: pass hasActiveClaudeThinking(body) to the non-stream
translate call (inline, since the shared const is in the stream branch's
temporal dead zone here).
Tests: 25/25 (5 gate-restore, 1 gate-502-repro, 4 nonstream-leak,
15 resp-openai-to-claude incl. requestedThinking suppression regression).
Group E (22) + F (14) regression-free. typecheck:core clean.
6 tasks done
Merged
5 of 6 tasks
Collaborator
|
Thanks for tackling three real problems here — the DSML marker parsing (Group F) in particular
|
This was referenced Sep 15, 2026
Merged
diegosouzapw
added a commit
to initguru/OmniRoute
that referenced
this pull request
Sep 15, 2026
…dd changelog fragment Points the #reasoning-bilingual comment at the real companion file (translator/response/openai-to-claude.ts's directivePreambleStripper.ts from cheaper-inference#12905) instead of the nonexistent "openai-responses.ts", and adds the changelog fragment referenced in the PR body but missing from the diff. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ranslation path The needsResponsesTranslation branch (openai-responses -> openai, used when the client also speaks Responses) silently dropped the requestToolIdentityMap argument to createSSETransformStreamWithLogger when the requestedThinking parameter was added, reverting the cheaper-inference#7936 tool-identity round-trip fix for that branch. The sibling needsTranslation branch was updated correctly; restore the same argument here. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
# Conflicts: # open-sse/handlers/chatCore.ts
The PR body already writes out the changelog.d/ entries for the DSML parser (Group F), the directive-preamble stripper (Group E), and the reasoning-gate thinking-leak fix (Group G), but none of the files existed in the diff. Add them so the release aggregator picks them up. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ng parameter createSSETransformStreamWithLogger gained a new requestedThinking parameter inserted before customToolNames. glm.ts's translateSseResponse still called it with the pre-existing positional argument list, so the new parameter silently absorbed the old customToolNames slot, and the GLM_STREAM_BUFFER_BYTES tuning value (cheaper-inference#12925) landed on requestToolIdentityMap instead of streamBufferBytes — a TS2345 (number is not assignable to Map<...> | null) caught by check:open-sse-typecheck, and a real loss of GLM's 64KB stream buffer budget. Insert an explicit `undefined` for requestedThinking to restore the original alignment. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
diegosouzapw
added a commit
that referenced
this pull request
Sep 17, 2026
…all target shapes (#12904) * fix(sse): inject global system prompt post-translation for codex/Responses path codex/Responses requests carry input[]+instructions, not messages[]. The existing injectSystemPrompt runs PRE-translation (chatCore.ts) and only handles messages[]/system fields, so the Global System Prompt (After Prompt = suffixPrompt) never reached the provider for codex — verified 0/84 call logs while the catalog base_instructions reached 84/84. Add injectSystemPromptPostTranslation() and call it after prepareUpstreamBody on the resolved messages[]. With multiple system/developer messages (codex normalises its per-item developer roles to system), prefix goes on the FIRST and suffix on the LAST so the After Prompt retains the highest recency position — the semantics injectSystemPrompt's single-findIndex buries. Also wire OMNIROUTE_SYSTEM_INSTRUCTION_APPEND on the /v1/messages (Claude Messages -> OpenAI Chat Completions) translation path. The directive was previously only wired on the Responses API path, so DeepSeek-V4 kept leaking English planning/chain-of-thought into the content field on Claude Code sessions that route through /v1/messages. Mirror the openai-responses.ts pattern: append to string system, append a text block for array content, or unshift a new system message when none exists. Tests: 23/23 (19 system-prompt incl. 6 postTranslation + codex regression; 4 claude-to-openai directive append). typecheck:core clean. * test(sse): reproduce global prompt double injection — single-injection contract tests * fix(sse): unify global prompt injection to single post-translation pass * fix(sse): carry single global-prompt injection across claude/gemini/responses target shapes * fix(sse): restore global-prompt coverage for carrier-less targets via gated pre-translation pass * fix(sse): cover codex/gemini source shapes in the carrier-less pre-translation gate * fix(types): preserve generic system prompt return * fix(sse): correct file reference in claude-to-openai.ts comment and add changelog fragment Points the #reasoning-bilingual comment at the real companion file (translator/response/openai-to-claude.ts's directivePreambleStripper.ts from #12905) instead of the nonexistent "openai-responses.ts", and adds the changelog fragment referenced in the PR body but missing from the diff. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Jihyun Son <jihyun.son@sk.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ream end
`createSystemPreambleStripper()` was wired DEFAULT-ON and unconditional in the
openai→claude streaming translator, unlike the exact-directive stripper right
above it, which only runs when the operator configured
OMNIROUTE_SYSTEM_INSTRUCTION_APPEND. Cause: the system-echo stripper recognises
its openers by English-prose heuristics ("# Skill usage", "# Verification
Process", <analysis>/<summary>/<system-reminder>), so leaving it always-on made
it mutate the response payload of EVERY openai→claude stream. A legitimate reply
opening with "# Skill usage: how to write one\n\nHere is the guide." lost that
whole section. It is now gated on OMNIROUTE_STRIP_SYSTEM_PREAMBLE=1, mirroring
the directive stripper's opt-in.
Second cause, same feature: neither stripper was ever flushed. Both buffer while
a construct is still undecided — a directive prefix that never completes, an
<analysis> block that never closes — and nothing released that buffer at the end
of the stream. A reply consisting of an unterminated echo block therefore reached
the client as an EMPTY message: the answer was held in the buffer and discarded
with the stripper. Both strippers now expose flush(), the finish handler calls it
for both, and the released text is emitted as a text block. A construct that WAS
finally classified as an echo is not resurrected (the drop is final).
Tests: tests/unit/system-preamble-gate-and-flush.test.ts pins the default-off
contract, the opted-in behaviour, the flush for both strippers (unit + wiring),
and the no-resurrection guard. system-preamble-wiring.test.ts now opts in
explicitly, since it exercises the stripping path.
…path
The streaming and non-streaming translators disagreed on the default meaning of
`requestedThinking`, so the SAME request produced different shapes depending on
`stream`. Cause: chatCore computes the client's intent
(hasActiveClaudeThinking) and threads it into the SSE translator, which relays
reasoning as a thinking block only when it is explicitly `true` — but NO caller
ever passed it to translateNonStreamingResponse(). The non-streaming
OpenAI→Claude conversion therefore only ever saw `undefined`, its legacy
"always relay a thinking block" default, and leaked reasoning to a client that
had opted out with `thinking: {"type":"disabled"}`. The streaming plumbing also
coerced an omitted value into an explicit `false`, hiding the divergence behind
two different spellings of "no intent".
Fix (least destructive of the options): do NOT flip either gate — both encode a
deliberate, regression-tested contract — but give the non-streaming path the
same input the streaming path already has. runNonStreamingProviderLeg owns the
client body (`sourceBody`), so it computes the intent with the very same helper
and passes it down through translateNonStreamingClientResponse. `undefined`
keeps its documented back-compat relay for callers that cannot express intent
(issue-7856 / issue-6623), and stream.ts no longer defaults the parameter to
`false`, so "absent" now means the same thing in both signatures.
No content is lost by the suppression: a reasoning-ONLY response is still
relayed as an ordinary text block (no empty response, no 502) — exactly what the
streaming finish handler does.
Tests: tests/unit/nonstream-requested-thinking-parity.test.ts drives the real
provider leg with thinking disabled / enabled / adaptive.
Collaborator
|
Fiz um rework nesta branch (2 commits) antes de mergear. Os grupos DSML e reasoning-gate são valor real e ficaram intocados; o que mexi foi no preamble stripper, onde a revisão encontrou três problemas que não estavam na análise anterior. Registro com as provas, porque dois deles apagavam resposta de usuário. 1. O stripper de sistema era default-on. 2. Bloco que nunca fecha engolia a resposta inteira. Não havia flush no fim do stream — um chunk 3. Provas, verificadas nos dois sentidos contra o head original desta branch: As 5 falhas cobrem os três itens: o gate relayando verbatim, o flush (unit ×2 + wiring) e a paridade non-streaming. O que ficou em aberto, explicitamente: a heurística do stripper ainda é sensível a como o upstream particiona os chunks SSE — o mesmo conteúdo produz saídas diferentes conforme o corte. O gate default-off tira isso do caminho de produção, mas quem ligar o recurso ainda pega esse comportamento. Redesenhar a heurística é trabalho separado. Crédito é seu; os fragmentos de changelog registram isso. |
diegosouzapw
merged commit Sep 17, 2026
b7192b7
into
cheaper-inference:release/v3.8.51
3 of 7 checks passed
diegosouzapw
added a commit
that referenced
this pull request
Sep 17, 2026
…e moved (#14002) Measured on the clean tip (83fa432), not on a branch. Each ceiling was attributed to the PR that moved it before being raised — no blanket rebaseline: - src/sse/handlers/chatHelpers.ts 1245 -> 1246 (#13551, combo scope on the fail-closed proxy guard) - open-sse/handlers/chatCore.ts 6203 -> 6219 (#12905 DSML/preamble, #13910 disguised-2xx classification, #12904 single post-translation system prompt) - open-sse/utils/stream.ts 3123 -> 3140 (#12905, and #12906 empty_response 502 retry + reasoning-aware timeout) - tests/integration/chat-pipeline.test.ts 1736 -> 1740 (#13419, the two exact header assertions updated for charset=utf-8) Other gates re-measured on the same tip and already green: typecheck:core, check:open-sse-typecheck, check:docs-counts, and the #2331-adjacent chatcore-translation-paths xhigh-effort case that was red before this wave.
2 of 3 tasks
This was referenced Sep 17, 2026
This was referenced Sep 18, 2026
diegosouzapw
added a commit
that referenced
this pull request
Sep 18, 2026
… keep acquireMany under the complexity ceiling The #12905 gate-restore suite encoded 'requestedThinking absent' as opt-out, the same undefined-means-false shape its non-streaming twin documents the other way and that the two-month-old #5786 suites contradict. The three opt-out cases now set the flag explicitly, which is what chatCore resolves for an opted-out client; the two opt-in cases already did. Both suites pass together (27/27). The failFast branch pushed acquireMany over the complexity ceiling it already sat on; the admission policy (fail-fast / bounded / unbounded queue) moves to findQueueRejection() and the new-code ratchet is back at its base. Refs #13866
diegosouzapw
added a commit
that referenced
this pull request
Sep 18, 2026
…the write, drain the 09-18 base-reds (#14101) * fix(quality): drain the 09-18 base-reds, part 1 — thinking gate parity, inventory, webpack externals Reproduced on the clean tip 7cc454d before touching anything. Five of the failures trace to one commit, #12905 (b7192b7): it gated thinking-block emission on `requestedThinking === true` in the streaming translator, while its own non-streaming path documents `undefined` as the legacy caller shape that keeps "always a thinking block". The two paths disagreed on the same input, and the streaming side also synthesized the reasoning into a TEXT block for that legacy shape. chatCore always resolves a boolean, so production never sends `undefined` — but every direct caller and the older #5786 suites do. Aligned the streaming gate to the documented tri-state: `false` suppresses, `true` and `undefined` relay, and the fix-B text synthesis fires only on an explicit opt-out. The #12905 test that asserted suppression used a bare createState() (`undefined`) to mean "client did not request thinking"; it now passes `requestedThinking: false`, which is what that sentence resolves to in production. The whole thinking family — dsml, adapter, translator, non-stream parity, #13620, #5786, markdown boundary — is 77/77. #12864 added requestRejectedFailure.ts with a getProviderConnectionById read that seeds the refusal streak across restarts; inventoried as a connection state read next to the family-cooldown site it resembles. #13909 made machineToken.ts import ./dataPaths; the isolated webpack compile has no repo tree, so it joins the sibling externals. The free-tier budget card SVG was one wave behind again (482 -> 491 models). Refs #13866 * fix(sse): restore maxQueueDepth=0 as unbounded, sanitize refusals at the write, drain the rest Part 2 of the 09-18 base-red drain. Two of the remaining failures were not stale tests but production defects the tests had caught. #12911 taught accountSemaphore to read `maxQueueSize: 0` as "reject when the slot is busy", which is what its Codex WS lease wants. But chatCore forwards `resilienceSettings.requestQueue.maxQueueDepth` into that option, and that setting's documented default since #6593 is `0 = disabled`. Under default settings every request that found its account slot occupied was answered 429 "Semaphore queue full (0)" instead of waiting — the managed-lease routing test saw exactly that. `0` (and any non-positive value) is unbounded again; the lease gets an explicit `failFast` option and its four tests stay green, so the #12911 behaviour is preserved where it was meant to apply. A contract test pins the #6593 semantics on the semaphore itself. #12864 moved two providerFailure persistence branches out of chatCore into requestRejectedFailure.ts and the sanitization did not travel with them: three `lastError` writes stored the message as received. The only caller already hands in the projected persistentMessage, so nothing leaks today, but a persistence branch must be safe at its own write (docs/security/ ERROR_SANITIZATION.md) rather than trust whoever calls it. The module now sanitizes on entry, and the public-boundary guard — which caught this by counting sanitized writes in chatCore and coming up two short — covers the extracted module too, verified by mutating one write back to raw. The rest are tests that had fallen behind legitimate changes: - #12905 inserted `requestedThinking` as the 14th positional argument of createSSETransformStreamWithLogger; two tests passed customToolNames or the buffer budget at their old positions. Both production callers were already correct. - #12754 added a per-connection reset-card fetch after the quota fetch; the spacing test now marks a chunk at the quota request only. - #13910 renamed `error` to `errorMetadata` in the timeout classification; the probe matches the identifier with a backreference and still fails when BodyTimeoutError is removed from both sites. Refs #13866 * fix(test): pin the opt-out thinking cases to requestedThinking=false; keep acquireMany under the complexity ceiling The #12905 gate-restore suite encoded 'requestedThinking absent' as opt-out, the same undefined-means-false shape its non-streaming twin documents the other way and that the two-month-old #5786 suites contradict. The three opt-out cases now set the flag explicitly, which is what chatCore resolves for an opted-out client; the two opt-in cases already did. Both suites pass together (27/27). The failFast branch pushed acquireMany over the complexity ceiling it already sat on; the admission policy (fail-fast / bounded / unbounded queue) moves to findQueueRejection() and the new-code ratchet is back at its base. Refs #13866 * fix(test): suspend the #14110 redaction assertion inline; refresh the budget card The 57 commits merged since the previous validation moved two things. #13295 changed how an unknown-root path with an ambiguous tail is answered: where `Provider failed at /custom/internal secret directory` used to become `Provider failed at <path>` it now ships verbatim. The #12506 boundary guard caught it. Two candidate fixes were tried and each breaks one of the two live contracts — #12506's fail-closed swallow, or #13144's rule that a route in prose must survive — so the choice is the owner's (#14110). The one contested assertion is suspended inline with the exact line and the issue; the other nine stay active. The isolated-child harness requires tests == pass, which is why it is a comment and not a todo. The free-tier budget card was one wave behind again (491 -> 489 models). Refs #13866, #14110 * fix(providers): type the TinyCMS DOM stub global as a loose record #13957 typed the mock global as `typeof globalThis & Record<string, unknown>`. The api-route typecheck loads lib.dom, so that intersection carries the real Window / HTMLCanvasElement / document signatures — every stub assignment fails against a DOM constructor, and `delete g.window` narrows the object to `never` (13 diagnostics, the API Route Typecheck base-red on the tip). The function exists to overwrite those globals with stubs; it is now typed as the plain record it manipulates. 29/29 tinycms tests unchanged. Refs #13866 * fix(test): pin the last opt-out thinking sibling to requestedThinking=false translator-reasoning-gate-502-repro is the third #12905 test that encoded a bare state as opt-out; the previous sweep matched files by glob and missed it. The family is now enumerated by grep on requestedThinking (7 files) plus the two pre-#12905 suites: 83/83 together. Refs #13866
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…all target shapes (cheaper-inference#12904) * fix(sse): inject global system prompt post-translation for codex/Responses path codex/Responses requests carry input[]+instructions, not messages[]. The existing injectSystemPrompt runs PRE-translation (chatCore.ts) and only handles messages[]/system fields, so the Global System Prompt (After Prompt = suffixPrompt) never reached the provider for codex — verified 0/84 call logs while the catalog base_instructions reached 84/84. Add injectSystemPromptPostTranslation() and call it after prepareUpstreamBody on the resolved messages[]. With multiple system/developer messages (codex normalises its per-item developer roles to system), prefix goes on the FIRST and suffix on the LAST so the After Prompt retains the highest recency position — the semantics injectSystemPrompt's single-findIndex buries. Also wire OMNIROUTE_SYSTEM_INSTRUCTION_APPEND on the /v1/messages (Claude Messages -> OpenAI Chat Completions) translation path. The directive was previously only wired on the Responses API path, so DeepSeek-V4 kept leaking English planning/chain-of-thought into the content field on Claude Code sessions that route through /v1/messages. Mirror the openai-responses.ts pattern: append to string system, append a text block for array content, or unshift a new system message when none exists. Tests: 23/23 (19 system-prompt incl. 6 postTranslation + codex regression; 4 claude-to-openai directive append). typecheck:core clean. * test(sse): reproduce global prompt double injection — single-injection contract tests * fix(sse): unify global prompt injection to single post-translation pass * fix(sse): carry single global-prompt injection across claude/gemini/responses target shapes * fix(sse): restore global-prompt coverage for carrier-less targets via gated pre-translation pass * fix(sse): cover codex/gemini source shapes in the carrier-less pre-translation gate * fix(types): preserve generic system prompt return * fix(sse): correct file reference in claude-to-openai.ts comment and add changelog fragment Points the #reasoning-bilingual comment at the real companion file (translator/response/openai-to-claude.ts's directivePreambleStripper.ts from cheaper-inference#12905) instead of the nonexistent "openai-responses.ts", and adds the changelog fragment referenced in the PR body but missing from the diff. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Jihyun Son <jihyun.son@sk.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…ive thinking (cheaper-inference#12905) * fix(thinking): recognize adaptive thinking + parse/scrub DSML tool-call markers Two defects combined to break DeepSeek-V4-Flash turns and raise 502 empty_response on Claude Code autocompact. Defect 1 — DSML tool-call markers leaked as visible content: DeepSeek-V4-Flash occasionally emits tool calls in a non-standard DSML text format using full-width pipes instead of the OpenAI tool_calls JSON. Two shapes appear in production call logs: - complete block: <｜DSML｜:Read><path>...</path></｜DSML｜:Read> - stray closers (truncated call): </｜DSML｜parameter></｜DSML｜invoke> </｜DSML｜tool_calls>, sometimes trailing a system-prompt echo The openai-compatible path never parsed these, so the markers leaked to the client as visible content and the turn ended incomplete. Fix: add open-sse/utils/dsmlToolCalls.ts — parseDsmlToolCalls() converts complete DSML blocks into OpenAI tool_calls and strips stray closing markers from content (streaming-safe via a holdback for partial openers). Wire it into the response translator before extractXmlInvokeBlocks so DSML and XML invoke tool calls share the same pending queue. Defect 2 — adaptive thinking silently suppressed: A prior inline === 'enabled' check on body.thinking.type silently suppressed adaptive (the intent Claude Code actually sends), so reasoning was dropped. The model then emitted DSML tool-call markers as plain text, producing an incomplete stop finish. Fix: use hasActiveClaudeThinking() (which recognizes enabled AND adaptive) to set requestedThinking, thread it through stream.ts and translator state, and gate thinking block emission on state.requestedThinking so upstream reasoning_content only relays when the client opted in. Tests: 29/29 (6 dsml-tool-calls, 5 thinking-active-claude-adapter, 3 translator-resp-dsml-integration, 15 translator-resp-openai-to-claude incl. requestedThinking suppression regression). typecheck:core clean. * fix(sse): strip echoed system-prompt preamble + preserve large analysis/summary blocks DeepSeek-V4 and similar models echo the OMNIROUTE_SYSTEM_INSTRUCTION_APPEND directive (appended to the system tail by claude-to-openai.ts) and whole chunks of the system prompt (<analysis>/<system-reminder>/<summary> blocks, prose reproductions of the superpowers skill section) verbatim at the START of their reply — the 'system message leak' persisting after the request-side fix. Add two streaming-safe preamble strippers in directivePreambleStripper.ts: - createDirectivePreambleStripper(directive): drops a leading reproduction of the exact configured directive across arbitrary SSE chunk boundaries. - createSystemPreambleStripper(): removes <analysis>/<system-reminder>/ <summary> echo blocks and known prose heads (Phase B) from the very start of a stream, only while the stream is still a preamble. Wire both into openai-to-claude.ts content-delta path: chain the exact-directive stripper then the system-echo stripper before DSML/XML-invoke parsing, so a leading system echo is dropped before it reaches the client. Preserve large blocks (>= SYSTEM_ECHO_THRESHOLD=1000 chars) and blocks with no trailing content — these are the model's real response (e.g. a Claude Code autocompact summary), not a short system-echo. Stops the autocompact empty-response regression where a whole-summary <analysis> block was stripped to empty (3a8515). Regression: origin's markdown-boundary feature (bufferedPrefix / splitMarkdownBoundary, commit f03d99a) is preserved — preamble strip runs before the markdown buffer rehydration, and the scrubbed content flows into the existing DSML/XML-invoke/markdown pipeline unchanged. TDD: tests/unit/directive-preamble-strip.test.ts (7 cases), system-preamble-strip.test.ts (12 cases incl. 3a8515 regression), system-preamble-wiring.test.ts (3 integration cases); group F regression 24/24 green; typecheck:core 0 errors. * fix(sse): gate thinking block on requestedThinking + synthesize text block for reasoning-only responses Reasoning-content (thinking) blocks were emitted unconditionally to Claude-format clients, leaking reasoning to thinking-opt-out clients (Claude Code sends thinking:{type:"disabled"}) — the operator reported 'reasoning is exposed'. On reasoning-only upstream responses (GLM-5.2 autocompact pattern), the unconditional thinking block also caused either a 502 'no content block' at flush, or — after a text-block fallback — an autocompact 'empty response' rejection that looped the session forever. Streaming translator (openai-to-claude.ts): - Compute hasReasoning outside the emission gate; accumulate into state._reasoningAccum always (so fix B can fire). - Gate only the thinking-block EMISSION on requestedThinking === true. - FIX B at finish: when no text block was started and requestedThinking !== true, synthesize a text content block from _reasoningAccum so autocompact can extract the summary (no 502, compact applies). - Skip fix B when requestedThinking === true to avoid double-exposure (thinking block + text block both carrying reasoning). Non-streaming translator (responseTranslator.ts): - Thread requestedThinking through translateNonStreamingResponse into convertOpenAINonStreamingToClaude. - suppressThinking = requestedThinking === false: drop the thinking block when content is present (no leak); relay reasoning as a text block when the response is reasoning-only (no 502). requestedThinking === undefined keeps the legacy 'always a thinking block' relay. chatCore.ts: pass hasActiveClaudeThinking(body) to the non-stream translate call (inline, since the shared const is in the stream branch's temporal dead zone here). Tests: 25/25 (5 gate-restore, 1 gate-502-repro, 4 nonstream-leak, 15 resp-openai-to-claude incl. requestedThinking suppression regression). Group E (22) + F (14) regression-free. typecheck:core clean. * fix(sse): restore requestToolIdentityMap in the Codex CLI responses-translation path The needsResponsesTranslation branch (openai-responses -> openai, used when the client also speaks Responses) silently dropped the requestToolIdentityMap argument to createSSETransformStreamWithLogger when the requestedThinking parameter was added, reverting the cheaper-inference#7936 tool-identity round-trip fix for that branch. The sibling needsTranslation branch was updated correctly; restore the same argument here. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * docs(changelog): add the 3 fragments documented in the PR body The PR body already writes out the changelog.d/ entries for the DSML parser (Group F), the directive-preamble stripper (Group E), and the reasoning-gate thinking-leak fix (Group G), but none of the files existed in the diff. Add them so the release aggregator picks them up. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(sse): realign GLM's positional call after the new requestedThinking parameter createSSETransformStreamWithLogger gained a new requestedThinking parameter inserted before customToolNames. glm.ts's translateSseResponse still called it with the pre-existing positional argument list, so the new parameter silently absorbed the old customToolNames slot, and the GLM_STREAM_BUFFER_BYTES tuning value (cheaper-inference#12925) landed on requestToolIdentityMap instead of streamBufferBytes — a TS2345 (number is not assignable to Map<...> | null) caught by check:open-sse-typecheck, and a real loss of GLM's 64KB stream buffer budget. Insert an explicit `undefined` for requestedThinking to restore the original alignment. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(sse): make the system-preamble stripper opt-in and flush it at stream end `createSystemPreambleStripper()` was wired DEFAULT-ON and unconditional in the openai→claude streaming translator, unlike the exact-directive stripper right above it, which only runs when the operator configured OMNIROUTE_SYSTEM_INSTRUCTION_APPEND. Cause: the system-echo stripper recognises its openers by English-prose heuristics ("# Skill usage", "# Verification Process", <analysis>/<summary>/<system-reminder>), so leaving it always-on made it mutate the response payload of EVERY openai→claude stream. A legitimate reply opening with "# Skill usage: how to write one\n\nHere is the guide." lost that whole section. It is now gated on OMNIROUTE_STRIP_SYSTEM_PREAMBLE=1, mirroring the directive stripper's opt-in. Second cause, same feature: neither stripper was ever flushed. Both buffer while a construct is still undecided — a directive prefix that never completes, an <analysis> block that never closes — and nothing released that buffer at the end of the stream. A reply consisting of an unterminated echo block therefore reached the client as an EMPTY message: the answer was held in the buffer and discarded with the stripper. Both strippers now expose flush(), the finish handler calls it for both, and the released text is emitted as a text block. A construct that WAS finally classified as an echo is not resurrected (the drop is final). Tests: tests/unit/system-preamble-gate-and-flush.test.ts pins the default-off contract, the opted-in behaviour, the flush for both strippers (unit + wiring), and the no-resurrection guard. system-preamble-wiring.test.ts now opts in explicitly, since it exercises the stripping path. * fix(sse): thread the client's thinking intent into the non-streaming path The streaming and non-streaming translators disagreed on the default meaning of `requestedThinking`, so the SAME request produced different shapes depending on `stream`. Cause: chatCore computes the client's intent (hasActiveClaudeThinking) and threads it into the SSE translator, which relays reasoning as a thinking block only when it is explicitly `true` — but NO caller ever passed it to translateNonStreamingResponse(). The non-streaming OpenAI→Claude conversion therefore only ever saw `undefined`, its legacy "always relay a thinking block" default, and leaked reasoning to a client that had opted out with `thinking: {"type":"disabled"}`. The streaming plumbing also coerced an omitted value into an explicit `false`, hiding the divergence behind two different spellings of "no intent". Fix (least destructive of the options): do NOT flip either gate — both encode a deliberate, regression-tested contract — but give the non-streaming path the same input the streaming path already has. runNonStreamingProviderLeg owns the client body (`sourceBody`), so it computes the intent with the very same helper and passes it down through translateNonStreamingClientResponse. `undefined` keeps its documented back-compat relay for callers that cannot express intent (issue-7856 / issue-6623), and stream.ts no longer defaults the parameter to `false`, so "absent" now means the same thing in both signatures. No content is lost by the suppression: a reasoning-ONLY response is still relayed as an ordinary text block (no empty response, no 502) — exactly what the streaming finish handler does. Tests: tests/unit/nonstream-requested-thinking-parity.test.ts drives the real provider leg with thinking disabled / enabled / adaptive. --------- Co-authored-by: Jihyun Son <jihyun.son@sk.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…e moved (cheaper-inference#14002) Measured on the clean tip (bebb827), not on a branch. Each ceiling was attributed to the PR that moved it before being raised — no blanket rebaseline: - src/sse/handlers/chatHelpers.ts 1245 -> 1246 (cheaper-inference#13551, combo scope on the fail-closed proxy guard) - open-sse/handlers/chatCore.ts 6203 -> 6219 (cheaper-inference#12905 DSML/preamble, cheaper-inference#13910 disguised-2xx classification, cheaper-inference#12904 single post-translation system prompt) - open-sse/utils/stream.ts 3123 -> 3140 (cheaper-inference#12905, and cheaper-inference#12906 empty_response 502 retry + reasoning-aware timeout) - tests/integration/chat-pipeline.test.ts 1736 -> 1740 (cheaper-inference#13419, the two exact header assertions updated for charset=utf-8) Other gates re-measured on the same tip and already green: typecheck:core, check:open-sse-typecheck, check:docs-counts, and the cheaper-inference#2331-adjacent chatcore-translation-paths xhigh-effort case that was red before this wave.
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…the write, drain the 09-18 base-reds (cheaper-inference#14101) * fix(quality): drain the 09-18 base-reds, part 1 — thinking gate parity, inventory, webpack externals Reproduced on the clean tip 7784784 before touching anything. Five of the failures trace to one commit, cheaper-inference#12905 (8b5f4dd): it gated thinking-block emission on `requestedThinking === true` in the streaming translator, while its own non-streaming path documents `undefined` as the legacy caller shape that keeps "always a thinking block". The two paths disagreed on the same input, and the streaming side also synthesized the reasoning into a TEXT block for that legacy shape. chatCore always resolves a boolean, so production never sends `undefined` — but every direct caller and the older cheaper-inference#5786 suites do. Aligned the streaming gate to the documented tri-state: `false` suppresses, `true` and `undefined` relay, and the fix-B text synthesis fires only on an explicit opt-out. The cheaper-inference#12905 test that asserted suppression used a bare createState() (`undefined`) to mean "client did not request thinking"; it now passes `requestedThinking: false`, which is what that sentence resolves to in production. The whole thinking family — dsml, adapter, translator, non-stream parity, cheaper-inference#13620, cheaper-inference#5786, markdown boundary — is 77/77. cheaper-inference#12864 added requestRejectedFailure.ts with a getProviderConnectionById read that seeds the refusal streak across restarts; inventoried as a connection state read next to the family-cooldown site it resembles. cheaper-inference#13909 made machineToken.ts import ./dataPaths; the isolated webpack compile has no repo tree, so it joins the sibling externals. The free-tier budget card SVG was one wave behind again (482 -> 491 models). Refs cheaper-inference#13866 * fix(sse): restore maxQueueDepth=0 as unbounded, sanitize refusals at the write, drain the rest Part 2 of the 09-18 base-red drain. Two of the remaining failures were not stale tests but production defects the tests had caught. cheaper-inference#12911 taught accountSemaphore to read `maxQueueSize: 0` as "reject when the slot is busy", which is what its Codex WS lease wants. But chatCore forwards `resilienceSettings.requestQueue.maxQueueDepth` into that option, and that setting's documented default since cheaper-inference#6593 is `0 = disabled`. Under default settings every request that found its account slot occupied was answered 429 "Semaphore queue full (0)" instead of waiting — the managed-lease routing test saw exactly that. `0` (and any non-positive value) is unbounded again; the lease gets an explicit `failFast` option and its four tests stay green, so the cheaper-inference#12911 behaviour is preserved where it was meant to apply. A contract test pins the cheaper-inference#6593 semantics on the semaphore itself. cheaper-inference#12864 moved two providerFailure persistence branches out of chatCore into requestRejectedFailure.ts and the sanitization did not travel with them: three `lastError` writes stored the message as received. The only caller already hands in the projected persistentMessage, so nothing leaks today, but a persistence branch must be safe at its own write (docs/security/ ERROR_SANITIZATION.md) rather than trust whoever calls it. The module now sanitizes on entry, and the public-boundary guard — which caught this by counting sanitized writes in chatCore and coming up two short — covers the extracted module too, verified by mutating one write back to raw. The rest are tests that had fallen behind legitimate changes: - cheaper-inference#12905 inserted `requestedThinking` as the 14th positional argument of createSSETransformStreamWithLogger; two tests passed customToolNames or the buffer budget at their old positions. Both production callers were already correct. - cheaper-inference#12754 added a per-connection reset-card fetch after the quota fetch; the spacing test now marks a chunk at the quota request only. - cheaper-inference#13910 renamed `error` to `errorMetadata` in the timeout classification; the probe matches the identifier with a backreference and still fails when BodyTimeoutError is removed from both sites. Refs cheaper-inference#13866 * fix(test): pin the opt-out thinking cases to requestedThinking=false; keep acquireMany under the complexity ceiling The cheaper-inference#12905 gate-restore suite encoded 'requestedThinking absent' as opt-out, the same undefined-means-false shape its non-streaming twin documents the other way and that the two-month-old cheaper-inference#5786 suites contradict. The three opt-out cases now set the flag explicitly, which is what chatCore resolves for an opted-out client; the two opt-in cases already did. Both suites pass together (27/27). The failFast branch pushed acquireMany over the complexity ceiling it already sat on; the admission policy (fail-fast / bounded / unbounded queue) moves to findQueueRejection() and the new-code ratchet is back at its base. Refs cheaper-inference#13866 * fix(test): suspend the cheaper-inference#14110 redaction assertion inline; refresh the budget card The 57 commits merged since the previous validation moved two things. cheaper-inference#13295 changed how an unknown-root path with an ambiguous tail is answered: where `Provider failed at /custom/internal secret directory` used to become `Provider failed at <path>` it now ships verbatim. The cheaper-inference#12506 boundary guard caught it. Two candidate fixes were tried and each breaks one of the two live contracts — cheaper-inference#12506's fail-closed swallow, or cheaper-inference#13144's rule that a route in prose must survive — so the choice is the owner's (cheaper-inference#14110). The one contested assertion is suspended inline with the exact line and the issue; the other nine stay active. The isolated-child harness requires tests == pass, which is why it is a comment and not a todo. The free-tier budget card was one wave behind again (491 -> 489 models). Refs cheaper-inference#13866, cheaper-inference#14110 * fix(providers): type the TinyCMS DOM stub global as a loose record cheaper-inference#13957 typed the mock global as `typeof globalThis & Record<string, unknown>`. The api-route typecheck loads lib.dom, so that intersection carries the real Window / HTMLCanvasElement / document signatures — every stub assignment fails against a DOM constructor, and `delete g.window` narrows the object to `never` (13 diagnostics, the API Route Typecheck base-red on the tip). The function exists to overwrite those globals with stubs; it is now typed as the plain record it manipulates. 29/29 tinycms tests unchanged. Refs cheaper-inference#13866 * fix(test): pin the last opt-out thinking sibling to requestedThinking=false translator-reasoning-gate-502-repro is the third cheaper-inference#12905 test that encoded a bare state as opt-out; the previous sweep matched files by glob and missed it. The family is now enumerated by grep on requestedThinking (7 files) plus the two pre-cheaper-inference#12905 suites: 83/83 together. Refs cheaper-inference#13866
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Group F: DSML tool-call markers + adaptive thinking
fix(thinking): parse/scrub DSML tool-call markers and recognize adaptive thinking
Summary
<tool_call>...</tool_call>, etc.) into reasoning, parse them into normal tool calls and scrub them from response text. Add the newdsmlToolCalls.tsmodule and wire it intoopenai-to-claude.ts/stream.ts/chatCore.ts/translator/index.ts.Related Issues
Validation
node --import tsx/esm --test tests/unit/dsml-tool-calls.test.ts tests/unit/translator-resp-dsml-integration.test.ts tests/unit/thinking-active-claude-adapter.test.ts tests/unit/translator-resp-openai-to-claude.test.tsnpm run lintrelease/v3.8.51base; focused checks rerun afterwardTests Added Or Updated
tests/unit/dsml-tool-calls.test.tstests/unit/translator-resp-dsml-integration.test.tstests/unit/thinking-active-claude-adapter.test.tstests/unit/translator-resp-openai-to-claude.test.ts(updated)Coverage Notes
Reviewer Notes
dsmlToolCalls.tsdoes not exist; 0 occurrences ofdsmlin the canonical tree. CanonicalnormalizeClaudeAdaptiveThinking(chatCore.ts:2677) is the canonical adaptive-compression service and is unrelated to response-side DSML recognition.f6eb328ba. New fileopen-sse/utils/dsmlToolCalls.ts.changelog.d/features/12905-dsml-tool-call-markers-adaptive-thinking.md
Group E: system/directive preamble strip + preserve autocompact empty responses
fix(sse): strip echoed system preamble and preserve large analysis blocks
Summary
/v1/messagesresponses. Strip it with the newdirectivePreambleStripper.tsmodule.systemPreambleStripperstrips the entire<analysis>summary block to an empty string, causing autocompact to treat it as an empty response. Add logic to preserve large analysis/summary blocks.Related Issues
Validation
node --import tsx/esm --test tests/unit/directive-preamble-strip.test.ts tests/unit/system-preamble-strip.test.ts tests/unit/system-preamble-wiring.test.tsnpm run lintrelease/v3.8.51base; focused checks rerun afterwardTests Added Or Updated
tests/unit/directive-preamble-strip.test.tstests/unit/system-preamble-strip.test.tstests/unit/system-preamble-wiring.test.tsCoverage Notes
Reviewer Notes
directivePreambleStripper.tsdoes not exist; 0 occurrences ofautocompact.696587b69+6141f0524+ad893c106+3488c9214. Includesad893c106(analysis-block preservation) from memory [[reasoning-gate-restore-7acddd91a-fix]].openai-to-claude.ts→ overlaps with Group G; apply in order (E first, G second).changelog.d/fixes/12905-directive-preamble-strip-analysis-preserve.md
Group G: reasoning gate / thinking leak
fix(sse): gate thinking block emission on requestedThinking to stop reasoning leak
Summary
requestedThinkingfor both streaming and non-streaming. The canonical implementation always opens a thinking block when the upstream emits reasoning, causing a reasoning leak.requestedThinkingis true, preventing double exposure of reasoning.95ce30612).Related Issues
Validation
node --import tsx/esm --test tests/unit/translator-reasoning-gate-502-repro.test.ts tests/unit/translator-reasoning-gate-restore-7acddd91a.test.ts tests/unit/translator-resp-openai-to-claude.test.ts tests/unit/translator-nonstream-reasoning-leak-502.test.tsnpm run lintrelease/v3.8.51base; focused checks rerun afterwardTests Added Or Updated
tests/unit/translator-reasoning-gate-502-repro.test.tstests/unit/translator-reasoning-gate-restore-7acddd91a.test.tstests/unit/translator-resp-openai-to-claude.test.ts(updated)tests/unit/translator-nonstream-reasoning-leak-502.test.tsCoverage Notes
Reviewer Notes
openai-to-claude.ts:300-333always emits thinking blocks (norequestedThinkinggate); 0 occurrences of requestedThinking inchatCore.ts/responseTranslator.ts:557,723.cf518362d(non-stream gate) +ba0bd3af8(streaming flush) +0af44ec68(streaming gate) +85deb09fb(skip synthesis when requestedThinking) +8da8e3ee3(synthesize) +14534ee95(part of remove-gate cycle).openai-to-claude.ts/chatCore.ts/responseTranslator.tsstate at local95ce30612. Exclude 5 debug-logging commits.openai-to-claude.tsis shared with Group E → implement E first, then G, and run typecheck after each commit.changelog.d/fixes/12905-reasoning-gate-thinking-leak.md