fix(agent): replay provider metadata across tool steps - #3820
Conversation
📦 Client bundle boundary
A server module in a client graph aborts hydration in the browser. New leaks fail CI; known leaks are tracked in |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (15)
📝 WalkthroughWalkthroughThe change preserves assistant provider metadata through direct generation, streaming, lifecycle snapshots, agent messages, and tool-result continuations. It also updates two API reference source links and adds focused Google continuation tests. ChangesProvider metadata propagation
API reference links
Estimated code review effort: 3 (Moderate) | ~25 minutes ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
609cb88 to
0a86157
Compare
|
Please run an exact-head Codex review on |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
0a86157 to
57a9313
Compare
|
@codex review exact head |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@coderabbitai review exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
target='57a9313307131b35c5d43420b2a0ec4d1131b7f0'
printf '%s\n' '== Exact commit =='
git show -s --format='commit=%H%nparents=%P%nsubject=%s' "$target"
printf '%s\n' '== Checked-out HEAD =='
git rev-parse HEAD
printf '%s\n' '== Changed files against first parent =='
git diff-tree --no-commit-id --name-status -r "$target"
printf '%s\n' '== Diff statistics =='
git diff-tree --no-commit-id --stat -r "$target"
printf '%s\n' '== TypeScript file outlines =='
git diff-tree --no-commit-id --name-only -r "$target" \
| while IFS= read -r file; do
case "$file" in
*.ts|*.tsx)
printf '\n-- %s --\n' "$file"
ast-grep outline "$file" --items all --view expanded || true
;;
esac
doneLength of output: 35296 🏁 Script executed: #!/bin/bash
set -euo pipefail
target='57a9313307131b35c5d43420b2a0ec4d1131b7f0'
parent="$(git rev-parse "${target}^")"
printf '%s\n' '== Exact diff =='
git diff --find-renames --find-copies "$parent" "$target" -- \
src/agent/runtime/chat-stream-handler.ts \
src/agent/runtime/index.ts \
src/agent/runtime/provider-metadata.ts \
src/agent/runtime/runtime-tool-types.ts \
src/agent/runtime/text-generation-runtime-message-converter.ts \
src/agent/runtime/text-generation-runtime-message-types.ts \
src/agent/streaming/lifecycle/live-adapter.ts \
src/agent/streaming/lifecycle/reducer.ts \
src/agent/streaming/lifecycle/runtime-provider-adapter.ts \
src/agent/streaming/lifecycle/types.ts \
src/runtime/runtime-bridge.ts \
src/runtime/runtime-bridge.test.ts \
src/agent/runtime/provider-metadata-continuation.test.ts
printf '%s\n' '== Metadata attachment and replay uses =='
rg -n -C 5 '\b(attachProviderMetadata|readAttachedProviderMetadata|providerMetadata)\b' \
src/agent/runtime \
src/agent/streaming/lifecycle \
src/runtime/runtime-bridge.ts
printf '%s\n' '== Targeted function slices =='
sed -n '850,1045p' src/runtime/runtime-bridge.ts
sed -n '1180,1325p' src/agent/runtime/index.ts
sed -n '1430,1625p' src/agent/runtime/index.ts
sed -n '2080,2305p' src/agent/runtime/index.ts
sed -n '250,565p' src/agent/runtime/text-generation-runtime-message-converter.ts
sed -n '360,430p' src/agent/runtime/chat-stream-handler.ts
sed -n '720,775p' src/agent/streaming/lifecycle/reducer.ts
sed -n '215,285p' src/agent/streaming/lifecycle/live-adapter.ts
sed -n '430,510p' src/agent/streaming/lifecycle/runtime-provider-adapter.tsLength of output: 105374 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== Agent loop response and history updates =='
sed -n '1560,1880p' src/agent/runtime/index.ts
sed -n '2440,2740p' src/agent/runtime/index.ts
printf '%s\n' '== Streaming tool-result history updates =='
sed -n '2440,2740p' src/agent/runtime/index.ts
sed -n '2740,3030p' src/agent/runtime/index.ts
printf '%s\n' '== All request conversion call sites =='
rg -n -C 5 'convertToTextGenerationRuntimeRequestMessages\s*\(' src
printf '%s\n' '== All provider metadata consumers and model prompt adapters =='
rg -n -C 6 '\.providerMetadata\b|providerMetadata\b' \
extensions \
src \
-g '*.ts' \
-g '!src/agent/runtime/provider-metadata-continuation.test.ts'
printf '%s\n' '== Message mutation and memory append helpers =='
rg -n -C 4 '\b(currentMessages\.push|messages\.push|addMessage|appendMessage|saveMessage|assistantMessage)\b' \
src/agent/runtime/index.ts \
src/agent/memory \
-g '*.ts'Length of output: 201480
Found one issue.
|
|
Addressed after merge in follow-up #3821. The suggested consume-once behavior would violate Gemini 3 sequential function calling: Google explicitly defines request 3 as retaining the first signed function call and says to include signatures for all function calls in the loop. See https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures#sequential_function_calling_example. The existing WeakMap replay behavior is therefore correct for every request that replays that assistant step within the current turn. #3821 adds a three-request regression requiring signature A on requests 2 and 3, plus signature B on request 3, while confirming neither leaks into public messages. PR #3820 was merged externally before this response and test-only follow-up could be pushed. |
|
Addressed in follow-up PR #3824. The RED reproduced request 3 replaying both the prior and current thought signatures; the GREEN consumes replay metadata only when its assistant runtime message is retained in the actual outbound provider request, preserving metadata for trimmed messages until they are later sent. Full pre-push suite is green. |
|
Correction to my earlier follow-up note: the consume-once suggestion is rejected after validating the provider contract. Google requires all accumulated Primary source: https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures#sequential_function_calling_example |
PR #3820 attaches the provider's raw replay metadata to every streamed assistant message, but when the runtime suppresses an unavailable tool call the persisted parts no longer mirror the raw model turn. The Google request builder then fails its exact-history validation (validateGoogleToolReplay throws TypeError) before any request is sent, killing the continuation client-side. Skip the metadata attach when state.suppressedToolCalls is non-empty so the continuation falls back to synthesized parts — the pre-#3820 behavior for that turn — while the extension's fail-fast validation for genuinely corrupted history stays intact. Refs VERYFRONT-AGENT-9
PR #3820 attaches the provider's raw replay metadata to every streamed assistant message, but when the runtime suppresses an unavailable tool call the persisted parts no longer mirror the raw model turn. The Google request builder then fails its exact-history validation (validateGoogleToolReplay throws TypeError) before any request is sent, killing the continuation client-side. Skip the metadata attach when state.suppressedToolCalls is non-empty so the continuation falls back to synthesized parts — the pre-#3820 behavior for that turn — while the extension's fail-fast validation for genuinely corrupted history stays intact. Refs VERYFRONT-AGENT-9
…hape Gemini 3.x rejects the tool-result leg with HTTP 400 "Function call is missing a thought_signature in functionCall parts" unless the signed model turn is replayed verbatim. 2.5 accepts an unsigned replay, so the regression is invisible on that generation. The existing continuation tests use a synthetic single-part stream. Add a case modelled on a gemini-3.1-pro-preview streamGenerateContent response, where the signature rides the functionCall part in the first chunk and a trailing empty-text part arrives in a separate chunk with the finish reason. That two-chunk shape is what the live API accepts, and it is the shape the retained raw parts must reproduce. The signature value is opaque to the replay path, so the fixture carries a fabricated placeholder instead of a provider-generated value. Verified red against 5887da1 (the commit before #3820) and green on main, plus a live round trip through the staging AI gateway on gemini-3.1-pro-preview, gemini-3.5-flash, gemini-2.5-pro and gemini-2.5-flash. Refs veryfront-issue-inbox#549 Claude-Session: https://claude.ai/code/session_01J2e7P4tmqYjYZQ1paDomg5
1 participant
Summary
Root cause
The Google adapter already captured exact Gemini assistant parts, including
thoughtSignature, inproviderMetadata. The central runtime bridge discarded that metadata while normalizing generate and stream results, and the agent history builders therefore could not return it on the tool-result leg. The active stream lifecycle also discarded finish metadata while projecting its terminal snapshot. Gemini 2.5 tolerated the omission, while Gemini 3 rejects the continuation with HTTP 400 because function-call thought signatures are mandatory.Google protocol reference: https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures
RED-GREEN TDD
RED:
Generate and legacy stream initially made the second provider request with
providerMetadata: undefined. After fixing those paths, an explicit active-lifecycle regression still failed withproviderMetadata: undefined.GREEN:
Verification
deno task --quiet verify:quickdeno task --quiet typecheckdeno task --quiet lintdeno task docs:api-reference:checkgit diff --checkFixes veryfront/veryfront-issue-inbox#549
Summary by CodeRabbit
New Features
Bug Fixes
Tests