Restore Kimi-compatible streams when providers send null deltas - #3815
Conversation
…ible streams
The Veryfront Cloud Moonshot gateway encodes absent optional delta fields as
`null` rather than omitting them. Every kimi-k2.6 / kimi-k2.5 stream opens with
`{"reasoning_content":null,"role":"assistant","content":""}` and closes with
`{"reasoning_content":null}` alongside the finish reason.
The Chat Completions SSE parser rejected any non-string `reasoning_content`,
so it threw on the first chunk of every request:
veryfront-cloud request failed: invalid successful stream
(reasoning delta was malformed)
Kimi has been unusable through veryfront-cloud since #3235 hardened this
validation on 2026-08-02. That change turned a type-narrowing check into a hard
one without carrying over null tolerance, which `content`, `refusal`,
`tool_calls`, and `finish_reason` all already have in the same function.
Treat null as "not present on this chunk" for `reasoning_content` and `role`.
Genuinely wrong types (number, object, a non-assistant role) still reject the
stream.
Tests replay the verbatim chunk shape captured from the live gateway.
Fixes veryfront/veryfront-issue-inbox#542
|
Warning Review limit reached
Next review available in: 14 minutes Limit details: You’ve used all 1 included review currently available under your plan. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📦 Client bundle boundary
A server module in a client graph aborts hydration in the browser. New leaks fail CI; known leaks are tracked in |
The same gateway that nulls `reasoning_content` also repeats tool-call
identity fields as `null` on continuation fragments. Captured from
`kimi-k2.6` with a forced tool call:
{"id":"functions.list_events:0","index":0,"type":"function",
"function":{"name":"list_events","arguments":"{\"date\":\""}}
{"id":null,"index":0,"type":"function",
"function":{"name":null,"arguments":"2026-08"}}
{"id":null,"index":0,"type":null,
"function":{"name":null,"arguments":"-18\"}"}}
Only the opening fragment carries real values. With the reasoning fix alone
the stream still died on the second fragment with "tool call id was
malformed", so any tools-enabled Kimi agent stayed broken -- which is every
agent that does real work.
Treat null as absent for `id`, `type`, `function.name`, and
`function.arguments`, so the fragment merges into the call being assembled.
The downstream merge logic already gates on positive `typeof === "string"`
checks, so null flows through without contributing. Wrong types still reject.
Replaying the captured tool stream now yields the full sequence:
reasoning-start, 13 reasoning-delta, reasoning-end, tool-input-start,
3 tool-input-delta, tool-call({"date":"2026-08-18"}), finish(tool-calls).
Second commit: the tool-call path was still brokenReview flagged that tool-call deltas kept the same null intolerance this PR was fixing elsewhere. That turned out to be real, and it meant the original fix would not have unblocked any agent that actually uses tools — including the calendar agent from the bug report ( Captured live from the gateway with {"id":"functions.list_events:0","index":0,"type":"function",
"function":{"name":"list_events","arguments":"{\"date\":\""}}
{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"2026-08"}}
{"id":null,"index":0,"type":null, "function":{"name":null,"arguments":"-18\"}"}}Only the opening fragment carries With the first commit alone, replaying that captured stream still died: So Both captured streams, replayed through the parser
Thanks to both reviewers for catching this; it was the difference between the model parsing and the model actually working. |
Null must behave exactly like an omitted field. These assert it cannot be used to change a call's identity mid-stream, collide at another index, skip the id/name ordering rule, or emit a tool call that never received an id, name, or arguments -- all verified to still reject.
Review noted the previous commit message claimed this case was locked while no test covered it. Add the missing one rather than soften the claim: a null fragment at a second index must not let a reused id slip past the index-collision guard.
|
Exact-head review requested for @codex review |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
requested='f0379f51fc96fa04083973ef7676211707609d75'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nREQUESTED=%s\n' "$actual" "$requested"
git status --short
git show --no-ext-diff --format=fuller --stat "$requested"
printf '\n--- DIFF ---\n'
git diff --no-ext-diff "${requested}^" "$requested" --check
git diff --no-ext-diff "${requested}^" "$requested" -- \
'extensions/ext-llm-openai/**' \
'deno.json' \
'src/utils/version-constant.ts'
printf '\n--- CHANGED FILE OUTLINES ---\n'
git diff --name-only "${requested}^" "$requested" | while IFS= read -r file; do
case "$file" in
*.ts|*.tsx|*.js|*.jsx)
printf '\n### %s\n' "$file"
ast-grep outline "$file" --items all --view compact || true
;;
esac
doneLength of output: 4570 🏁 Script executed: #!/bin/bash
set -euo pipefail
commit='f0379f51fc96fa04083973ef7676211707609d75'
printf '%s\n' '--- COMMIT ANCESTRY AND RANGE ---'
git log --no-ext-diff --oneline --decorate -12 "$commit"
base="$(git merge-base "$commit" HEAD)"
printf 'MERGE_BASE_WITH_CHECKED_OUT_HEAD=%s\n' "$base"
printf '%s\n' '--- FILES CHANGED SINCE THE EARLIEST RELATED COMMIT ---'
git log --no-ext-diff --format='%H %s' --reverse "${commit}~6..${commit}"
git diff --no-ext-diff --name-status "${commit}~6" "$commit"
printf '%s\n' '--- EXACT-COMMIT PARSER OUTLINE ---'
git show "${commit}:extensions/ext-llm-openai/src/openai-chat-stream.ts" > /tmp/openai-chat-stream.ts
ast-grep outline /tmp/openai-chat-stream.ts --items all --view signatures
printf '%s\n' '--- NULLABLE DELTA VALIDATION AND TOOL-CALL MERGE LOGIC ---'
git show "${commit}:extensions/ext-llm-openai/src/openai-chat-stream.ts" |
nl -ba |
sed -n '130,410p'
printf '%s\n' '--- EXACT-COMMIT TEST CONTEXT AND HELPERS ---'
git show "${commit}:extensions/ext-llm-openai/src/openai-chat-stream.test.ts" |
nl -ba |
sed -n '1,110p;500,730p'
printf '%s\n' '--- NULL-RELATED TEST CASES AT EXACT COMMIT ---'
git grep -n -i -C 3 'null' "$commit" -- \
'extensions/ext-llm-openai/src/openai-chat-stream.test.ts' \
'extensions/ext-llm-openai/src/openai-chat-stream.ts' || trueLength of output: 5085 🏁 Script executed: #!/bin/bash
set -euo pipefail
base='d84a7d4af427e01fddb7e25097c4aecb3173fecc'
commit='f0379f51fc96fa04083973ef7676211707609d75'
parser='extensions/ext-llm-openai/src/openai-chat-stream.ts'
testfile='extensions/ext-llm-openai/src/openai-chat-stream.test.ts'
printf '%s\n' '--- EXACT PR DIFF: PROVIDER PARSER AND TESTS ---'
git diff --no-ext-diff --check "$base" "$commit"
git diff --no-ext-diff --unified=80 "$base" "$commit" -- "$parser" "$testfile"
printf '%s\n' '--- PARSER: VALIDATION, TOOL-CALL STATE, AND FINALIZATION ---'
git show "${commit}:${parser}" |
sed -n '1,460p'
printf '%s\n' '--- TEST: HELPERS, NULL FIXTURES, AND NEARBY TOOL-CALL CASES ---'
git show "${commit}:${testfile}" |
sed -n '1,160p;430,760p'
printf '%s\n' '--- EXACT-COMMIT SYMBOL REFERENCES ---'
git grep -n -E \
'OpenAIStreamToolCallState|toolCallsByIndex|reasoning_content|delta\.role|tool call (id|type|name|arguments)|assertCompleteToolInput' \
"$commit" -- "$parser" "$testfile" || trueLength of output: 60260 🏁 Script executed: #!/bin/bash
set -euo pipefail
commit='f0379f51fc96fa04083973ef7676211707609d75'
parser='extensions/ext-llm-openai/src/openai-chat-stream.ts'
testfile='extensions/ext-llm-openai/src/openai-chat-stream.test.ts'
printf '%s\n' '--- PARSER TAIL: STREAM COMPLETION AND TOOL-CALL EMISSION ---'
git show "${commit}:${parser}" | sed -n '440,590p'
printf '%s\n' '--- EXISTING TOOL-CALL IDENTITY, INDEX, AND COMPLETENESS TESTS ---'
git show "${commit}:${testfile}" | sed -n '190,430p'
printf '%s\n' '--- TEST NAMES AT EXACT COMMIT ---'
git show "${commit}:${testfile}" |
grep -nE '^[[:space:]]*it\('
printf '%s\n' '--- DIFF WHITESPACE CHECK ---'
git diff --no-ext-diff --check 'd84a7d4af427e01fddb7e25097c4aecb3173fecc' "$commit"Length of output: 12541
The nullable-field handling preserves the type, identity, ordering, and completeness guards. The regression tests cover the captured text and tool-call continuation shapes. The existing tests retain cross-index ID-reuse and resource-limit checks. Runtime test execution was not available in this sandbox.
|
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
The failed Node job is the recurring scheduler-sensitive link observer test, not this PRs streaming change. I reproduced the test defect deterministically and opened the isolated test-only fix in #3816. I will keep this PR unmerged until #3816 lands and this exact head has a fresh fully green CI run. |
kojiwakayama
changed the title
2 participants
Problem
Every
veryfront-cloud/moonshotai/kimi-k2.6andkimi-k2.5run failed on the first successful stream chunk because the gateway encodes absent optional delta fields asnull. The OpenAI-compatible parser accepted only strings or omitted fields, so Kimi reasoning failed immediately; captured tool continuations then exposed the same defect forid,type,function.name, andfunction.arguments.Fix
Treat
nullas absence for optional reasoning, role, and tool-call continuation fields. Preserve the existing fail-closed contract for wrong types, non-assistant roles, tool-call identity changes, cross-index ID reuse, argument ordering, incomplete calls, and stream resource bounds.The regression tests replay the captured Kimi text and tool streams. The tool fixture assembles
functions.list_events:0across null-bearing continuation fragments into{"date":"2026-08-18"}and finishes astool-calls; adversarial fixtures prove null cannot launder identities or bypass completeness checks.Release
Bumps Veryfront to
0.1.1241and synchronizes the generated hydration runtime version.Fixes veryfront/veryfront-issue-inbox#542
Constraint: Moonshot encodes absent optional fields as null
Rejected: Drop stream validation | would allow wrong types and malformed tool-call continuations through the OpenAI-compatible parser
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Treat null as absence only; wrong types and identity, ordering, and completeness violations must still reject
Tested: Captured text and tool fixtures; ext-llm-openai 10 tests and 156 steps; targeted check, lint, format, and diff verification; full pre-push 3,921 tests and 29,965 steps
Not-tested: Post-release live staging Kimi run
Related: veryfront/veryfront-issue-inbox#542