Repository navigation
fix(vertex): reserve final_result step + graceful cap recovery in native Anthropic loop - #1125
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
✅ Single Commit Policy - COMPLIANTStatus: Policy requirements met • 1 commit • Valid format • Ready for merge 📊 View validation details📝 Commit Details
✅ Validation Results
🤖 Automated validation by NeuroLink Single Commit Enforcement |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthroughThis PR updates GoogleVertexProvider’s native Anthropic streaming and generate execution paths to reserve a finalization step, handle aborts gracefully, and guarantee non-empty terminal output. It also adds ChangesAnthropic Vertex Cap and Abort Handling
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant GoogleVertexProvider
participant AnthropicClient
participant ToolExecutor
Caller->>GoogleVertexProvider: executeNativeAnthropicStream/Generate(abortSignal)
GoogleVertexProvider->>GoogleVertexProvider: compute agenticStepBudget (reserve final step)
loop agentic steps
GoogleVertexProvider->>AnthropicClient: request (with abortSignal)
alt abort signaled
AnthropicClient-->>GoogleVertexProvider: aborted
GoogleVertexProvider->>GoogleVertexProvider: mark wasAborted, break loop
else tool_use returned
GoogleVertexProvider->>ToolExecutor: execute tool
ToolExecutor-->>GoogleVertexProvider: result or abort
GoogleVertexProvider->>GoogleVertexProvider: append soft-budget nudge if near limit
else final_result/no tools
GoogleVertexProvider->>GoogleVertexProvider: mark modelFinished
end
end
alt terminal without model finish
GoogleVertexProvider->>AnthropicClient: forced finalization call (tool_choice: final_result or none)
AnthropicClient-->>GoogleVertexProvider: finalization result or cap message
end
GoogleVertexProvider->>GoogleVertexProvider: map finishReason (length/tool-calls/stop)
GoogleVertexProvider-->>Caller: result with finishReason and non-empty content
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🤖 AI Review & Build Compliance ✅Status: AI analysis complete • Build rules validated • Ready for review 📊 View detailed analysis results🛡️ Analysis Complete
📋 Ready for Merge When
🤖 AI analysis complete - check individual code comments for specific feedback |
Tara-ag
left a comment
There was a problem hiding this comment.
Review Complete - Approved
I've thoroughly reviewed this PR which fixes the tool_choice:"any" structured-output deadlock in the native Anthropic (Claude-on-Vertex) agentic loop.
What was reviewed:
- package.json: New test script entry
- src/lib/providers/googleVertex.ts: Core implementation changes (~724 lines added)
- src/lib/types/stream.ts: Metadata type enhancement
- test/continuous-test-suite-anthropic-cap.ts: Comprehensive test suite (28 tests)
Key verification points:
- ✅ Imports validated:
buildToolLoopCapMessageandisAbortErrorcorrectly imported from existing exports ingoogleNativeGemini3.js - ✅ Architecture alignment: Implementation mirrors the approved Gemini fix (PR #1123) with appropriate Anthropic-specific adaptations
- ✅ No breaking changes: Public API unchanged; finishReason values ("tool-calls", "length", "stop") align with existing Gemini paths
- ✅ Abort handling: Proper signal propagation through SDK requests with graceful terminal handling
- ✅ Test coverage: 28 tests covering forced finalization, abort scenarios, budget nudges, and edge cases
Notable implementation details confirmed:
- Reserved finalization step correctly calculates
agenticStepBudget = maxSteps - 1when schema is active - Terminal re-check for aborts before finalization calls prevents unnecessary API requests
tool_choice: {type:"none"}correctly used instead of dropping tools (Anthropic API requirement)finishReasonexposed via both getter and metadata for wrapper compatibility
The fix addresses the root cause described in the PR: with tool_choice:"any", the text exit was unreachable, causing empty content when step budgets were exhausted. The reserved finalization step guarantees structured output delivery.
Recommendation: Approve for merge.
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
test/continuous-test-suite-anthropic-cap.ts (1)
173-268: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winMocks never verify the abortSignal is actually forwarded to the SDK.
makeGenerateMock'screate(lines 194-203) ignores theopts?.signalparameter its own type declares (lines 177-180), andmakeStreamMock'scontroller.abort(lines 246-250) is a documented no-op. All abort tests (e.g. "generate: abort mid-turn", "stream: abort mid-turn") simulate cancellation purely by havingonCallfirecontroller.abort()on the test's ownAbortControllerand then unconditionally rejecting viabehavior.reject. This means the tests would still pass even if the provider never actually passesoptions.abortSignalintoclient.messages.create()/client.messages.stream(), since the mock forces the abort-shaped rejection regardless of real wiring. Given the PR's stated goal is to "wire" abortSignal into both paths, this leaves that specific contract unverified.♻️ Suggested tightening
function makeGenerateMock( plan: (callIndex: number, params: AnthropicRequestParams) => CallBehavior, ): GenerateRecorder { const calls: AnthropicRequestParams[] = []; + const signals: (AbortSignal | undefined)[] = []; return { calls, + signals, client: { messages: { - create: async (p: AnthropicRequestParams) => { + create: async ( + p: AnthropicRequestParams, + opts?: { signal?: AbortSignal }, + ) => { const index = calls.length; calls.push(p); + signals.push(opts?.signal); const behavior = plan(index, p); behavior.onCall?.(); if (behavior.reject) { throw behavior.reject; } return behavior.response as AnthropicResponse; }, }, }, }; }Then assert
mock.signals[i] === controller.signalin the abort tests to confirm real forwarding, instead of relying solely on the injected rejection.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/continuous-test-suite-anthropic-cap.ts` around lines 173 - 268, The abort tests for the Anthropic paths are not actually verifying that the SDK receives the AbortSignal, because `makeGenerateMock().client.messages.create` and `makeStreamMock().client.messages.stream` ignore the `opts?.signal`/abort wiring and only simulate cancellation via `behavior.reject`. Update the tests around `generate` and `stream` to record the passed signal and assert it matches the test controller’s signal, using the existing `makeGenerateMock`, `makeStreamMock`, and the abort-related test cases to confirm real forwarding instead of relying on injected rejection behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/providers/googleVertex.ts`:
- Around line 5117-5124: The capped-turn prose handling in googleVertex should
accumulate all mid-loop text instead of overwriting it with only the latest
non-empty chunk. Update the step-text tracking around the agentic loop to use
appendStepText() for lastStepText, and apply the same accumulation change in the
other Gemini-mirroring blocks mentioned so capped generate turns preserve
earlier prose alongside tool calls.
- Around line 4254-4261: The abort handling in googleVertex’s tool-call flow
leaves `stepStorageCalls` and `stepStorageResults` out of sync because the
aborted branch in the tool execution path (`options.abortSignal`,
`isAbortError`, `endToolSpan`) breaks before a result is recorded. Update the
abort path so it either appends a neutral cancellation result to
`stepStorageResults` or removes the corresponding pending call before
persisting, and apply the same pairing fix in the other abort sites mentioned in
the tool-loop helpers to keep replayed tool history consistent.
- Around line 4758-4764: Preserve the stream accessors when wrapping the result
in executeStream and wrapStreamResultWithLifecycle, because the current object
spread snapshots finishReason before the background loop updates
finishReasonRef. Update the wrapping logic to copy property descriptors instead
of spreading the result so finishReason remains a live getter for public
callers, and verify the same behavior still holds alongside usage and metadata
accessors.
---
Nitpick comments:
In `@test/continuous-test-suite-anthropic-cap.ts`:
- Around line 173-268: The abort tests for the Anthropic paths are not actually
verifying that the SDK receives the AbortSignal, because
`makeGenerateMock().client.messages.create` and
`makeStreamMock().client.messages.stream` ignore the `opts?.signal`/abort wiring
and only simulate cancellation via `behavior.reject`. Update the tests around
`generate` and `stream` to record the passed signal and assert it matches the
test controller’s signal, using the existing `makeGenerateMock`,
`makeStreamMock`, and the abort-related test cases to confirm real forwarding
instead of relying on injected rejection behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: d8d42d55-a809-49aa-8178-e52a390e04b0
📒 Files selected for processing (4)
package.jsonsrc/lib/providers/googleVertex.tssrc/lib/types/stream.tstest/continuous-test-suite-anthropic-cap.ts
…ive Anthropic loop Extends ed289b7 (9.79.3) and PR #1123 (both Gemini-only) to the native Anthropic (Claude-on-Vertex) paths. Fixes the tool_choice:"any" structured-output deadlock: with options.schema active, the model is structurally forbidden from ever answering in plain text, so a turn whose step budget was consumed by other tool calls (runaway loop, or every tool failing) ended with content:"" and finishReason "stop" — surfaced to users as the consumer's generic empty-content fallback (two production incidents on 2026-07-01). Changes, applied to BOTH executeNativeAnthropicGenerate and executeNativeAnthropicStream: - Reserve the last step of the budget as a forced finalization call with tool_choice {type:"tool", name:"final_result"} — same cache-breakpoint treatment as loop steps, usage counted into totals. If maxSteps <= 1 the finalization call is the only call. - Soft budget nudge: when <= 3 agentic steps remain, a trailing text block on the tool_result user turn tells the model to consolidate and finish. - Backstop — never return "": failed/malformed finalization falls back to buildToolLoopCapMessage; non-schema pure tool-loops get one tools-disabled synthesis call (tool_choice:"none" with tools retained — Anthropic rejects tool_use/tool_result history without a tools param — plus a system-suffix countermand), then the cap message. Both twins also carry an absolute empty-content guard for tool-running turns. - Honest finishReason: "length" (max_tokens) takes precedence; capped turns without a clean/forced answer report "tool-calls" (aligned with the Gemini paths); clean and forced finishes report "stop". The stream twin exposes finishReason both as a result getter and on metadata (the mutable reference that survives wrapper result spreads, and the field neurolink.ts's stream span reads). - Abort wiring in executeNativeAnthropicGenerate: loop-entry check, options.abortSignal passed as SDK request signal, per-step and tool-exec catches break gracefully via isAbortError instead of throwing into abortSignal-less fallback retries; aborted tool calls are not recorded as failures. Both twins re-check the signal at terminal-handling entry (covers aborts landing during the final budgeted step's tool exec), persist completed tools of an aborted step to conversation memory before breaking, and prefer prose already produced/delivered over the cap message (generate returns lastStepText; stream suppresses the cap chunk when live deltas already reached the consumer). The stream twin's abort now closes the stream with one graceful cap chunk instead of erroring the channel. Adds test/continuous-test-suite-anthropic-cap.ts (pnpm run test:anthropic-cap, 28 cases) covering the finalization, backstop, nudge, abort, persistence, and finishReason matrix with a mocked Anthropic client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
d317ba9 to
6754f47
Compare
|
Addressed all CodeRabbit feedback in the amended commit 6754f47 (single-commit policy — amended + force-pushed):
Validation: 🤖 Generated with Claude Code |
🤖 AI Review & Build Compliance ✅Status: AI analysis complete • Build rules validated • Ready for review 📊 View detailed analysis results🛡️ Analysis Complete
📋 Ready for Merge When
🤖 AI analysis complete - check individual code comments for specific feedback |
Tara-ag
left a comment
There was a problem hiding this comment.
Review Summary
I've completed a thorough review of PR #1125. This is a well-crafted fix that extends the Gemini-3 step-cap recovery pattern (PR #1123) to the native Anthropic (Claude-on-Vertex) paths.
What This PR Does Well
-
Addresses Root Cause: Fixes the
tool_choice:"any"structured-output deadlock where the text exit is structurally unreachable, preventing empty content returns when step budgets are exhausted. -
Comprehensive Implementation:
- Reserved finalization step with forced
final_resultcall - Proper abort signal wiring through the Anthropic SDK
- Graceful cap recovery with
buildToolLoopCapMessage - Tool history pairing for aborted calls
- Accessor preservation through wrapper spreads
- Reserved finalization step with forced
-
Test Coverage: 28 tests covering schema/non-schema paths, abort scenarios, budget nudges, finish reason mapping, and lifecycle wrapping.
-
Code Quality:
- Follows existing patterns from the Gemini paths
- Proper TypeScript types throughout
- Comprehensive comments explaining the rationale
- No hardcoded secrets or security issues
Verification
- ✅ All existing review comments resolved
- ✅ Imports correctly wired (
isAbortError,appendStepText,buildToolLoopCapMessage,createTextChannel) - ✅ Abort signal properly passed to SDK requests
- ✅ Tool call/result pairing maintained for Anthropic API compatibility
- ✅ Getter preservation fixes the snapshot issue
- ✅ No breaking changes to public SDK API
Recommendation
Approve - This PR is ready for merge. The fix addresses confirmed production incidents and maintains parity with the already-merged Gemini implementation.
|
🎉 This PR is included in version 9.80.3 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Description
What does this PR do?
Fixes the
tool_choice:"any"structured-output deadlock in the native Anthropic (Claude-on-Vertex) agentic loop. Withoptions.schemaset, thefinal_resultpattern forces a tool call on every assistant turn, so the model is structurally forbidden from ever answering in plain text. A turn whosemaxStepsbudget was consumed by other tool calls (runaway tool loop, or every tool failing with 403s) ended withcontent: ""and a maskingfinishReason: "stop"— surfaced to end users as the consumer's generic "I was unable to generate a response" fallback. Two production incidents confirmed on 2026-07-01 (641s runaway loop; 239s all-tools-403 outage, both endingcontentlength=0, finishReason=stop).This extends ed289b7 (9.79.3) and PR #1123 (both Gemini-only) to the Anthropic paths, and additionally wires
options.abortSignalintoexecuteNativeAnthropicGenerate(previously a complete no-op there — a caller's 600s wall-clock abort was observed doing nothing while the turn ran 641s).Related Issues
Relates to #1123 (Gemini-3 twin of this fix, merged as 60ff175).
Type of Change
Motivation and Context
Production incidents on Claude-on-Vertex structured-output turns (Curator/Tara). Root cause chain:
options.schemaactivates thefinal_resulttool pattern withtool_choice: {type:"any"}.tool_choice:"any"forces a tool call every turn → the loop's text exit (toolUseBlocks.length === 0) is unreachable.final_resultbeing called; when the step budget ran out first, the while-loop just ended withfinalText=""andtool_usestop_reason mapped to"stop".abortSignal.Changes Made
Applied to both
executeNativeAnthropicGenerateandexecuteNativeAnthropicStreaminsrc/lib/providers/googleVertex.ts:maxSteps − 1steps; iffinal_resultwas never called, one forced call withtool_choice: {type:"tool", name:"final_result"}guarantees structured output (Anthropic guarantees the pinned tool is called).maxSteps ≤ 1→ the finalization call is the only call. Total API calls never exceedmaxStepson schema turns.tool_resultuser turn tells the model to consolidate and finish — the forced call is a fallback, not the norm.buildToolLoopCapMessage(reused from fix(vertex): honor abortSignal + graceful step-cap message in native Gemini-3 loop #1123, no divergent copy). Non-schema pure tool-loops get one tools-disabled synthesis call, then the cap message. Both twins carry an absolute empty-content guard for tool-running turns.tool_use/tool_resultblocks without atoolsparam, so "tools disabled" is expressed astool_choice: {type:"none"}with tools retained — same intent, API-valid, and keeps the cached tools prefix byte-identical.max_tokens → "length"(takes precedence) > capped-without-answer →"tool-calls"(matches the Gemini paths, so consumers detect capped turns uniformly) >"stop"(incl. successful forced finalization/backstop). The stream twin now exposesfinishReasonas a result getter and onmetadata(the mutable reference that survives wrapper result spreads;neurolink.ts's stream span readsmetadata.finishReasonfirst).options.abortSignalrides as the SDK request signal (client.messages.create(params, {signal}), verified against @anthropic-ai/sdk 0.102.0), per-step + tool-exec catches break viaisAbortError(reused from fix(vertex): honor abortSignal + graceful step-cap message in native Gemini-3 loop #1123), aborted tool calls are not recorded as failures.lastStepText(Gemini parity); the stream suppresses the cap chunk when live deltas already reached the consumer (no contradictory "step limit" text after a visible partial answer).channel.error.Known/intentional behaviors reviewers may flag (all matching the merged Gemini design in 60ff175): caller aborts resolve gracefully with a cap message rather than rejecting (spec of this fix — a rejection re-enters signal-less fallback retries); abort-shaped tool throws break the turn even without a caller signal (
isAbortErroris the sole discriminator, exactly as the Gemini suite asserts); the defensive stall timer also resolves to the graceful cap path.Breaking Changes
Public API unchanged. Normal within-budget turns are byte-identical (verified: no finalization call is issued, same call counts, same cache-breakpoint prefix construction).
finishReason: "tool-calls"on capped turns is the same value the Gemini paths already emit.server_tool_usefiltering,final_resultfiltering fromtoolsUsed/toolExecutions, and themax_tokens → "length"mapping are all preserved.Testing
test/continuous-test-suite-anthropic-cap.ts(pnpm run test:anthropic-cap), 28/28 pass, driven through a mocked Anthropic client (privatecreateAnthropicVertexClientoverride, mirroringcontinuous-test-suite-gemini-abort.ts)test:gemini-abort27/27 (Gemini paths untouched),test:cache20/20 (prompt-caching non-regression),test:providers-mocked30/30,test:bugfixes86/86tsc --noEmit --strict, ESLint, Prettier all cleanManual Testing Steps
pnpm run test:anthropic-capneurolink generate "<tool-heavy prompt>" --provider vertex --model claude-sonnet-4-5@20250929 --max-steps 3with a schema + failing tools → expect valid structured JSON (forced finalization) or a graceful cap message, never empty content;finishReason"stop"/"tool-calls"accordingly.Code Quality
Commit Message Format
fix(vertex): reserve final_result step + graceful cap recovery in native Anthropic loopDependencies
Performance Impact
Security Considerations
Deployment Notes
🤖 Generated with Claude Code
Summary by CodeRabbit