Skip to content

fix(kiro): emit valid concatenable tool_calls.arguments deltas - #1573

Open
wahyuzero wants to merge 1 commit into
decolua:masterfrom
wahyuzero:fix/kiro-tool-args-streaming-bug
Open

wahyuzero wants to merge 1 commit into
decolua:masterfrom
wahyuzero:fix/kiro-tool-args-streaming-bug

Conversation

@wahyuzero

Copy link
Copy Markdown

Summary

Fixes a streaming protocol bug in the Kiro executor that caused random Unterminated string JSON parse errors in downstream consumers (opencode, claude-code, etc.) for tool calls with substantial argument content.

Root cause

open-sse/executors/kiro.js reads toolUseEvent.input and emits its JSON.stringify as an OpenAI tool_calls[].function.arguments delta on every event. Kiro/AWS CodeWhisperer streams input as partial objects that grow across events:

event 1: { command: "cat /home" }
event 2: { command: "cat /home/wxsys" }
event 3: { command: "cat /home/wxsys/Project/naskin/.impeccable.md" }

Stringifying each one produces overlapping prefixes:

{"command":"cat /home"}
{"command":"cat /home/wxsys"}
{"command":"cat /home/wxsys/Project/naskin/.impeccable.md"}

The OpenAI streaming protocol requires arguments deltas to be concatenable string fragments that join into one valid JSON. Concatenating the three above produces:

{"command":"cat /home"}{"command":"cat /home/wxsys"}{"command":"cat /home/wxsys/Project/naskin/.impeccable.md"}

→ JSON.parse fails with Unexpected non-whitespace character after JSON at position N or Unterminated string depending on where the parser bails out.

The truncation position is non-deterministic because it tracks the prefix length of whichever partial object Kiro happened to send. This makes the bug present as random "the network truncated my tool call" failures — {"comm, {"comman, {"command": — when in fact the protocol contract was violated upstream.

Reproduction

Any Kiro session that streams a tool whose arguments grow over multiple toolUseEvents will hit this. Common triggers:

  • File paths longer than a few words
  • Multi-line content parameters in write tools
  • Shell commands that get progressively expanded

I encountered it in opencode while writing a 1500-byte markdown file via the write tool routed through 9router → Kiro → Claude Opus 4.7. The write call failed with Unterminated string at variable byte offsets across retries.

Fix

  1. String-form input payloads are already concatenable deltas — emit them incrementally as before, tracking accumulated text per toolCallId in state.toolArgsEmitted.

  2. Object-form input payloads are buffered into state.toolArgsBuffered keyed by toolCallId. The latest canonical (full JSON of the most recent partial object) overwrites earlier buffered values — earlier partials are noise once a more complete one arrives.

  3. New helper flushBufferedToolArgs() is invoked at all three finish boundaries to emit each buffered canonical exactly once as a complete arguments string:

    • messageStopEvent
    • The dual-event finish path (hasMeteringEvent && hasContextUsage)
    • The TransformStream flush() callback

The result: downstream consumers receive arguments deltas that concatenate into valid JSON regardless of whether Kiro chose to send string fragments or growing partial objects.

Tests

New regression coverage (tests/unit/kiro-tool-args-streaming.test.js, 7 tests):

  • Single buffered tool call → valid flush
  • 4-event partial-object stream (the exact reported repro) → 1 valid flush
  • Multiple concurrent tool calls → indices preserved
  • Idempotent flush (canonical equals already-emitted) → no re-emit
  • Empty buffer → no-op
  • Contract test: re-stringifying partials is unparseable
  • Contract test: emitting once is parseable

No regression: all 14 existing kiro tests (openai-to-kiro.test.js, rtkKiro.test.js) continue to pass.

End-to-end verified: built the patched CLI tarball locally, reinstalled 9router@0.4.66, restarted the proxy, and reproduced the original failure case (substantial write tool call ~1500 bytes) — now succeeds. Multiple test writes confirmed.

Notes

  • The fix is executor-only — translator/response/kiro-to-openai.js has the same shape of bug but is a separate (non-streaming-executor) code path. If this PR lands, that file deserves a follow-up; happy to include it here if preferred.
  • state initialization adds two new Maps (toolArgsEmitted, toolArgsBuffered) — no change to existing field types.
  • flushBufferedToolArgs is exported so the unit tests can target it directly without standing up a full TransformStream pipeline.

Kiro/AWS CodeWhisperer streams `toolUseEvent.input` as PARTIAL OBJECTS
that grow over successive events (e.g. `{command:"cat /home"}` then
`{command:"cat /home/wxsys"}` then `{command:"cat /home/wxsys/Project"}`).

The executor was calling `JSON.stringify(toolInput)` per event and
emitting the full string as an OpenAI `tool_calls[].function.arguments`
delta. The OpenAI streaming protocol requires arguments deltas to be
**concatenable string fragments** that join into one valid JSON. Emitting
full re-stringified prefixes per chunk produced overlapping garbage:

    {"command":"cat /home"}{"command":"cat /home/wxsys"}{"command":"cat /home/wxsys/Project"}

Downstream consumers (opencode, claude-code, etc.) then hit
`Unterminated string` JSON parse errors with a non-deterministic
truncation point matching the prefix length of each successive partial
serialization — variable cutoffs like `{"comm`, `{"comman`, `{"command":`
that look like network truncation but are deterministic protocol
violations.

This pattern is observable in real use: any time Kiro sends a tool with
arguments that grow through multiple events, the final concatenated
arguments string is unparseable. The bug surfaced as random "Unterminated
string" failures on tools with longer string parameters (file paths,
shell commands, multi-line content), with the apparent failure size
varying because Kiro's chunk boundaries vary.

## Fix

- String-form `input` payloads are already concatenable deltas; emit
  them incrementally as before, tracking accumulated text per
  `toolCallId` in `state.toolArgsEmitted`.
- Object-form `input` payloads are buffered into `state.toolArgsBuffered`
  keyed by `toolCallId`. The latest canonical (full JSON of the most
  recent partial object) overwrites earlier buffered values — earlier
  partials are noise once a more complete one arrives.
- A new helper `flushBufferedToolArgs()` is invoked at all three finish
  boundaries (`messageStopEvent`, the dual-event finish path, and the
  TransformStream `flush()`) to emit each buffered canonical exactly
  once as a complete arguments string.

The result: downstream consumers receive arguments deltas that
concatenate into valid JSON regardless of whether Kiro chose to send
string fragments or growing partial objects.

## Tests

- New regression test `tests/unit/kiro-tool-args-streaming.test.js`
  with 7 cases including the exact reported reproduction (4-event
  partial-object stream produces 1 valid flush) and the contract test
  (re-stringifying partials is unparseable; emitting once is parseable).
- All 14 existing kiro tests (`openai-to-kiro.test.js`, `rtkKiro.test.js`)
  continue to pass — no regression in the request-translation or RTK
  paths.
- Verified end-to-end against a live patched build by reproducing the
  original failure case (substantial `write` tool call) which now
  succeeds.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved — PR #1573

Reviewed via Hermes Agent batch. Changes are well-scoped for a PR of this size. No blocking issues identified in this pass.


Reviewed by Hermes Agent

bavuongco10 added a commit to bavuongco10/9router that referenced this pull request Jun 11, 2026
diegosouzapw pushed a commit to cheaper-inference/OmniRoute that referenced this pull request Jun 26, 2026
Kiro/CodeWhisperer streams toolUseEvent.input as PARTIAL OBJECTS that grow over
time (e.g. {command:"cat /home"} then {command:"cat /home/wxsys"}). The executor
re-stringified each partial object and emitted it as an OpenAI argument delta, so
the overlapping JSON prefixes concatenated downstream into unparseable garbage
("Unterminated string").

Buffer object-form payloads in state.toolArgsBuffered keyed by toolCallId, keep
only the latest canonical, and flush once at finish boundaries (messageStopEvent
and the terminal flush). String-form payloads remain concatenable incremental
deltas. The flush is idempotent against state.toolArgsEmitted so the dual
boundaries never double-emit.

Co-authored-by: Wahyu Febri <wxsys@local>
Inspired-by: decolua/9router#1573
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants