relay: emit streamed tool metadata only once

[kylefu8]

Summary

• emit id, type, and function name only on the first Responses-to-Chat tool-call chunk
• keep continuation chunks keyed by index with argument deltas only
• omit an empty type field instead of serializing it as null
• add regressions for long argument continuations and parallel tool calls

Why

The Responses-to-Chat streaming converter currently repeats the same call ID and type on every function-argument delta. Although the chunks keep the same tool index, some OpenAI-compatible clients interpret each repeated ID as a new tool call. For a large tool payload, one logical call can therefore become many calls; only the first fragment has a function name and later fragments fail as unnamed tools.

This is distinct from the duplicate-index issue fixed in #6225: the tool index is stable, but call metadata is still repeated on every continuation chunk.

OpenAI-style streaming only needs the call metadata on the initial chunk. Subsequent chunks can safely contain the stable index and argument delta.

Validation

• cd relaykit && GOMAXPROCS=2 go test -p 2 ./...
• production-shaped long tool stream: 857 chunks, one tool index, exactly one ID/type/name, and arguments reconstructed as valid JSON
• parallel-tool regression confirms each new tool still receives its own metadata and index
• end-to-end client validation completed a 200-line streamed tool write without producing unnamed or duplicate tool calls

Summary by CodeRabbit

• Bug Fixes

  • Improved streamed tool-call responses so identifiers and types appear only on the initial chunk.
  • Prevented repeated tool metadata in subsequent argument updates.
  • Preserved correct metadata handling for multiple parallel tool calls.

• Tests

  • Added coverage for streamed tool calls, including incremental arguments and parallel calls.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e3fa8c73-5633-4d38-b522-6cea73e50191

📥 Commits

Reviewing files that changed from the base of the PR and between dede291 and 411efad.

📒 Files selected for processing (1)

• relaykit/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go

🚧 Files skipped from review as they are similar to previous changes (1)

• relaykit/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go

---

Walkthrough

The change emits tool-call ID and type only on the first streamed chunk. Empty Type values are omitted during JSON serialization. Tests cover argument deltas and parallel tool calls.

Changes

Tool-call streaming

Layer / File(s)	Summary
Initial tool metadata and validation relaykit/dto/openai_response.go, relaykit/relayconvert/internal/oai_responses/to_oai_chat_stream_resp.go, relaykit/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go	The stream converter emits ID and type only on the first tool-call chunk. Empty type values are omitted from JSON. Tests validate later argument chunks and parallel tool indexes.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• QuantumNous/new-api#6593: Both changes modify OpenAI tool-call conversion and streamed metadata serialization.

Poem

I’m a rabbit with a streaming tune,
Metadata hops in once, then leaves the room.
Arguments follow, light and neat,
Parallel tools keep indexes complete.
JSON stays quiet when fields are bare—
A tidy burrow everywhere.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: emitting streamed tool metadata only on the first chunk.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@relaykit/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go`:
- Around line 284-295: Expand the continuation-chunk regression tests around the
single-tool and parallel-tool paths: assert each continuation’s Index and
Function.Arguments values in addition to omitted metadata. Add interleaved
argument-delta events for both parallel output indexes, then verify each emitted
chunk retains the correct per-tool index and arguments while still omitting ID,
Type, and Function.Name.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a8cc2d84-57a1-4226-b7d1-90b7dc8096dd

📥 Commits

Reviewing files that changed from the base of the PR and between 5c3abff and dede291.

📒 Files selected for processing (3)

• relaykit/dto/openai_response.go
• relaykit/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go
• relaykit/relayconvert/internal/oai_responses/to_oai_chat_stream_resp.go

[kylefu8]

Additional real-world context: I encountered this while using GPT-5.6 Luna through NewAPI's ChatCompletions-to-Responses compatibility mode in WorkBuddy.

A show_widget call with a large SVG payload was streamed as many argument chunks. Because every continuation chunk repeated the same call ID, WorkBuddy interpreted the fragments as separate tool calls. The first fragment was recognized as show_widget, while the remaining fragments had empty names and failed with Tool not found in agent cli. After its retries were exhausted, the conversation remained stuck in the thinking state.

After applying this patch, I repeated the same WorkBuddy test with a large show_widget payload. WorkBuddy reconstructed and executed one tool call successfully, returned the final response, and exited the thinking state normally. I also validated a separate 200-line streamed Write tool call end to end.