test(agent): pin the Gemini 3.x tool-result replay to the live wire shape - #3827
Conversation
📦 Client bundle boundary
A server module in a client graph aborts hydration in the browser. New leaks fail CI; known leaks are tracked in |
|
Important Review available on request
Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c7f304c1f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…hape Gemini 3.x rejects the tool-result leg with HTTP 400 "Function call is missing a thought_signature in functionCall parts" unless the signed model turn is replayed verbatim. 2.5 accepts an unsigned replay, so the regression is invisible on that generation. The existing continuation tests use a synthetic single-part stream. Add a case modelled on a gemini-3.1-pro-preview streamGenerateContent response, where the signature rides the functionCall part in the first chunk and a trailing empty-text part arrives in a separate chunk with the finish reason. That two-chunk shape is what the live API accepts, and it is the shape the retained raw parts must reproduce. The signature value is opaque to the replay path, so the fixture carries a fabricated placeholder instead of a provider-generated value. Verified red against 5887da1 (the commit before #3820) and green on main, plus a live round trip through the staging AI gateway on gemini-3.1-pro-preview, gemini-3.5-flash, gemini-2.5-pro and gemini-2.5-flash. Refs veryfront-issue-inbox#549 Claude-Session: https://claude.ai/code/session_01J2e7P4tmqYjYZQ1paDomg5
c7f304c to
1511185
Compare
2 participants
What this is
A regression test, not a fix. The fix for the reported bug already landed on
mainin #3820. This pins the behaviour to the shape the live API actually returns, and records the verification.The bug (veryfront-issue-inbox#549)
google-ai-studio/gemini-3.1-pro-previewandgoogle-ai-studio/gemini-3.5-flashpassed text runs but failed every tool call. The tool call itself succeeded; the run died on the next provider request — the one carrying the tool result back.Captured verbatim from the provider:
{"error": {"code": 400, "message": "Function call is missing a thought_signature in functionCall parts. This is required for tools to work correctly, and missing thought_signature may lead to degraded model performance. Additional data, function call `default_api:get_weather` , position 2. Please refer to https://ai.google.dev/gemini-api/docs/thought-signatures for more details.", "status": "INVALID_ARGUMENT"}}Replaying the model turn four ways against the live gateway isolates the cause to exactly one field:
Not the
functionResponseshape, not part ordering, not the trailing{"text":""}part 3.x emits. OnlythoughtSignature. 2.5 tolerates its absence, which is why the same round trip passed there and made this look 3.x-specific rather than replay-specific.Why it is already fixed
ext-llm-googlewas correct on both ends the whole time:google-stream.tsretains the signed raw part and emits it asproviderMetadata.google.rawAssistantParts, andgoogle-request-builder.tsreplays those parts verbatim. The metadata never reached the replay path, because the agent loop rebuilds the prompt from framework messages on every step and had no field to carry it. #3820 plumbedproviderMetadatathrough that boundary.What this test adds
The existing continuation tests drive a synthetic single-part stream. A real
gemini-3.1-pro-previewresponse splits differently: the signature rides thefunctionCallpart in the first chunk, and a trailing empty-text part arrives in a separate chunk alongsidefinishReason: STOP. Both parts are retained, and the verbatim two-part replay is what the live API accepts. This test uses that captured shape, so a future change that drops the trailing part or the signature fails here instead of in production against one model generation.Verification
5887da120(the commit before fix(agent): replay provider metadata across tool steps #3820), green onmain.mainvs pre-fix:deno test --allow-all extensions/ext-llm-google/— 4 passed (106 steps).deno test --allow-all src/agent/runtime/provider-metadata-continuation.test.ts— 6 steps, all pass.text-generation-runtime-message-converter.test.tsandruntime-bridge.test.ts— 2 passed (78 steps).Still open: the fix is unreleased
#3820 merged after
v0.1.1240and is in no published release. Staging runsveryfront@0.1.1231. Until a release ships and is deployed, #549 still reproduces in staging and production regardless of this PR. That release is the remaining action, not more code.Also checked and clear: reloaded multi-turn history (unsigned tool calls from earlier turns) returns 200 on 3.x. Gemini only enforces the signature on the tool call immediately preceding the tool result, so conversations resumed from persistence are unaffected by the metadata being in-memory only.
Refs veryfront-issue-inbox#549
https://claude.ai/code/session_01J2e7P4tmqYjYZQ1paDomg5