Skip to content

fix(vertex): reserve final_result step + graceful cap recovery in native Anthropic loop - #1125

Merged
murdore merged 1 commit into
releasefrom
fix/vertex-anthropic-loop-cap
Jul 2, 2026
Merged

murdore merged 1 commit into
releasefrom
fix/vertex-anthropic-loop-cap

Conversation

@pdogra1299

@pdogra1299 pdogra1299 commented Jul 2, 2026 •

Copy link
Copy Markdown
Collaborator

Description

What does this PR do?

Fixes the tool_choice:"any" structured-output deadlock in the native Anthropic (Claude-on-Vertex) agentic loop. With options.schema set, the final_result pattern forces a tool call on every assistant turn, so the model is structurally forbidden from ever answering in plain text. A turn whose maxSteps budget was consumed by other tool calls (runaway tool loop, or every tool failing with 403s) ended with content: "" and a masking finishReason: "stop" — surfaced to end users as the consumer's generic "I was unable to generate a response" fallback. Two production incidents confirmed on 2026-07-01 (641s runaway loop; 239s all-tools-403 outage, both ending contentlength=0, finishReason=stop).

This extends ed289b7 (9.79.3) and PR #1123 (both Gemini-only) to the Anthropic paths, and additionally wires options.abortSignal into executeNativeAnthropicGenerate (previously a complete no-op there — a caller's 600s wall-clock abort was observed doing nothing while the turn ran 641s).

Related Issues

Relates to #1123 (Gemini-3 twin of this fix, merged as 60ff175).

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Motivation and Context

Production incidents on Claude-on-Vertex structured-output turns (Curator/Tara). Root cause chain:

  1. options.schema activates the final_result tool pattern with tool_choice: {type:"any"}.
  2. Anthropic's tool_choice:"any" forces a tool call every turn → the loop's text exit (toolUseBlocks.length === 0) is unreachable.
  3. The only other exit was final_result being called; when the step budget ran out first, the while-loop just ended with finalText="" and tool_use stop_reason mapped to "stop".
  4. No recovery existed on the Anthropic paths (the 9.79.3 fix and fix(vertex): honor abortSignal + graceful step-cap message in native Gemini-3 loop #1123 were Gemini-only), and generate never honored abortSignal.

Changes Made

Applied to both executeNativeAnthropicGenerate and executeNativeAnthropicStream in src/lib/providers/googleVertex.ts:

  • Reserved finalization step: with a schema, the agentic loop runs maxSteps − 1 steps; if final_result was never called, one forced call with tool_choice: {type:"tool", name:"final_result"} guarantees structured output (Anthropic guarantees the pinned tool is called). maxSteps ≤ 1 → the finalization call is the only call. Total API calls never exceed maxSteps on schema turns.
  • Soft budget nudge: at ≤ 3 remaining agentic steps, a trailing text block on the tool_result user turn tells the model to consolidate and finish — the forced call is a fallback, not the norm.
  • Backstop — never empty: failed/malformed finalization → graceful buildToolLoopCapMessage (reused from fix(vertex): honor abortSignal + graceful step-cap message in native Gemini-3 loop #1123, no divergent copy). Non-schema pure tool-loops get one tools-disabled synthesis call, then the cap message. Both twins carry an absolute empty-content guard for tool-running turns.
    • ⚠️ Deliberate deviation from "drop tools": Anthropic rejects requests whose history contains tool_use/tool_result blocks without a tools param, so "tools disabled" is expressed as tool_choice: {type:"none"} with tools retained — same intent, API-valid, and keeps the cached tools prefix byte-identical.
  • Honest finishReason: max_tokens → "length" (takes precedence) > capped-without-answer → "tool-calls" (matches the Gemini paths, so consumers detect capped turns uniformly) > "stop" (incl. successful forced finalization/backstop). The stream twin now exposes finishReason as a result getter and on metadata (the mutable reference that survives wrapper result spreads; neurolink.ts's stream span reads metadata.finishReason first).
  • Abort wiring (generate): loop-entry check breaks into terminal handling instead of throwing (a throw routes consumers into abortSignal-less fallback retries — the observed 641s no-op), options.abortSignal rides as the SDK request signal (client.messages.create(params, {signal}), verified against @anthropic-ai/sdk 0.102.0), per-step + tool-exec catches break via isAbortError (reused from fix(vertex): honor abortSignal + graceful step-cap message in native Gemini-3 loop #1123), aborted tool calls are not recorded as failures.
  • Abort hardening from self-review (verified multi-agent review, findings fixed in this commit):
    • Both twins re-check the signal at terminal-handling entry, covering aborts that land during the final budgeted step's tool execution (no post-abort finalization/backstop calls, no extra billing).
    • Completed tools of an aborted step are persisted to conversation memory before breaking (real side effects stay auditable in chat history).
    • Aborts prefer prose already produced/delivered: generate returns lastStepText (Gemini parity); the stream suppresses the cap chunk when live deltas already reached the consumer (no contradictory "step limit" text after a visible partial answer).
    • Stream abort closes the channel with one graceful cap chunk instead of channel.error.

Known/intentional behaviors reviewers may flag (all matching the merged Gemini design in 60ff175): caller aborts resolve gracefully with a cap message rather than rejecting (spec of this fix — a rejection re-enters signal-less fallback retries); abort-shaped tool throws break the turn even without a caller signal (isAbortError is the sole discriminator, exactly as the Gemini suite asserts); the defensive stall timer also resolves to the graceful cap path.

Breaking Changes

  • No breaking changes

Public API unchanged. Normal within-budget turns are byte-identical (verified: no finalization call is issued, same call counts, same cache-breakpoint prefix construction). finishReason: "tool-calls" on capped turns is the same value the Gemini paths already emit. server_tool_use filtering, final_result filtering from toolsUsed/toolExecutions, and the max_tokens → "length" mapping are all preserved.

Testing

  • Unit tests added/updated — new suite test/continuous-test-suite-anthropic-cap.ts (pnpm run test:anthropic-cap), 28/28 pass, driven through a mocked Anthropic client (private createAnthropicVertexClient override, mirroring continuous-test-suite-gemini-abort.ts)
  • Existing tests pass — test:gemini-abort 27/27 (Gemini paths untouched), test:cache 20/20 (prompt-caching non-regression), test:providers-mocked 30/30, test:bugfixes 86/86
  • tsc --noEmit --strict, ESLint, Prettier all clean

Manual Testing Steps

  1. pnpm run test:anthropic-cap
  2. For live verification: neurolink generate "<tool-heavy prompt>" --provider vertex --model claude-sonnet-4-5@20250929 --max-steps 3 with a schema + failing tools → expect valid structured JSON (forced finalization) or a graceful cap message, never empty content; finishReason "stop"/"tool-calls" accordingly.

Code Quality

  • Code follows the project's style guidelines (ESLint passes)
  • Code is properly formatted (Prettier applied)
  • Self-review of code completed — plus an adversarial multi-agent review pass; all 6 fixable findings addressed in this commit, remaining ones are deliberate Gemini-parity behaviors documented above
  • No console.log statements (using logger instead)
  • No hardcoded API keys or secrets
  • TypeScript strict mode compliance
  • Proper error handling implemented

Commit Message Format

  • Single commit, semantic format: fix(vertex): reserve final_result step + graceful cap recovery in native Anthropic loop

Dependencies

  • No dependency changes

Performance Impact

  • No performance impact — happy-path turns are unchanged; capped/failed turns gain at most one extra bounded model call (finalization or backstop), which replaces an unbounded consumer retry loop in practice. The finalization call reuses the identical cached system/tools prefix.

Security Considerations

  • No security implications

Deployment Notes

  • No special deployment steps

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added a new npm script to run the Anthropic cap/abort continuous test suite.
  • Bug Fixes
    • Improved native Claude-on-Vertex streaming/generation to avoid empty output when tool-heavy runs hit the step budget.
    • Refined abort handling to stop cleanly and report the correct finish status for tool-driven, truncated, and terminated cases.
    • Enhanced structured-output completion reliability with correct finish-reason behavior.
  • Tests
    • Added an extensive suite covering budget caps, abort scenarios, tool-loop behavior, and stream non-emptiness/finishReason persistence.

@vercel

vercel Bot commented Jul 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
neurolink Ready Ready Preview, Comment Jul 2, 2026 6:39am

@github-actions

github-actions Bot commented Jul 2, 2026 •

Copy link
Copy Markdown
Contributor

✅ Single Commit Policy - COMPLIANT

Status: Policy requirements met • 1 commit • Valid format • Ready for merge

📊 View validation details

📝 Commit Details

  • Hash: 6754f4742f43e5e389ab6f294bd1f21c490b4200
  • Message: fix(vertex): reserve final_result step + graceful cap recovery in native Anthropic loop
  • Author: Parth Dogra

✅ Validation Results

  • Single commit requirement met
  • No merge commits in branch
  • Semantic commit message format verified
  • Ready for squash merge to release branch

🤖 Automated validation by NeuroLink Single Commit Enforcement

@coderabbitai

coderabbitai Bot commented Jul 2, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8c68b1aa-943d-4bd0-8e45-bc42ae469bb9

📥 Commits

Reviewing files that changed from the base of the PR and between d317ba9 and 6754f47.

📒 Files selected for processing (4)
  • package.json
  • src/lib/providers/googleVertex.ts
  • src/lib/types/stream.ts
  • test/continuous-test-suite-anthropic-cap.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • package.json
  • src/lib/types/stream.ts
  • test/continuous-test-suite-anthropic-cap.ts
  • src/lib/providers/googleVertex.ts

📝 Walkthrough

Walkthrough

This PR updates GoogleVertexProvider’s native Anthropic streaming and generate execution paths to reserve a finalization step, handle aborts gracefully, and guarantee non-empty terminal output. It also adds finishReason to StreamResult.metadata, a new continuous test suite, and a script to run it.

Changes

Anthropic Vertex Cap and Abort Handling

Layer / File(s) Summary
Streaming state and step-budget setup
src/lib/providers/googleVertex.ts
Introduces reserved agentic step budget and refactors stream state to use metadata, structuredOutputRef, finishReasonRef, and liveTextPushedLength tracking.
Streaming loop: abort, finish-reason, tool execution
src/lib/providers/googleVertex.ts
Updates delta handling, mid-flight abort resolution, stop-reason capture, structured-output completion detection, and abort-safe tool execution in the streaming loop.
Streaming terminal recovery and finish-reason exposure
src/lib/providers/googleVertex.ts
Rewrites terminal recovery to force finalization or synthesize backstop text, exposes finishReason via getter, and preserves getter-based stream result fields through wrappers.
Generate: step-budget setup and abortable request
src/lib/providers/googleVertex.ts
Reserves the finalization step in the generate budget, adds accumulated step prose recovery, and passes abortSignal into the SDK request.
Generate loop: completion detection and tool abort handling
src/lib/providers/googleVertex.ts
Updates completion detection, no-tools exit behavior, tool storage ordering, soft-budget nudge injection, and abort handling in tool execution and request flow.
Generate terminal recovery and finish-reason mapping
src/lib/providers/googleVertex.ts, src/lib/types/stream.ts
Guarantees non-empty terminal content after tool activity and updates finish-reason mapping to distinguish length, tool-calls, and stop; adds the StreamResult metadata finishReason field.
Test suite and script for cap/abort behavior
test/continuous-test-suite-anthropic-cap.ts, package.json
Adds a mocked-client test suite covering cap, abort, finalization, and nudge scenarios across generate and stream modes, plus a script to run it.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant GoogleVertexProvider
  participant AnthropicClient
  participant ToolExecutor

  Caller->>GoogleVertexProvider: executeNativeAnthropicStream/Generate(abortSignal)
  GoogleVertexProvider->>GoogleVertexProvider: compute agenticStepBudget (reserve final step)
  loop agentic steps
    GoogleVertexProvider->>AnthropicClient: request (with abortSignal)
    alt abort signaled
      AnthropicClient-->>GoogleVertexProvider: aborted
      GoogleVertexProvider->>GoogleVertexProvider: mark wasAborted, break loop
    else tool_use returned
      GoogleVertexProvider->>ToolExecutor: execute tool
      ToolExecutor-->>GoogleVertexProvider: result or abort
      GoogleVertexProvider->>GoogleVertexProvider: append soft-budget nudge if near limit
    else final_result/no tools
      GoogleVertexProvider->>GoogleVertexProvider: mark modelFinished
    end
  end
  alt terminal without model finish
    GoogleVertexProvider->>AnthropicClient: forced finalization call (tool_choice: final_result or none)
    AnthropicClient-->>GoogleVertexProvider: finalization result or cap message
  end
  GoogleVertexProvider->>GoogleVertexProvider: map finishReason (length/tool-calls/stop)
  GoogleVertexProvider-->>Caller: result with finishReason and non-empty content
Loading

Possibly related PRs

  • juspay/neurolink#1027: Both PRs extend the same native Anthropic streaming/generate termination and abort/tool handling logic in executeNativeAnthropicStream and executeNativeAnthropicGenerate.

Suggested reviewers: Tara-ag

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main Anthropic Vertex loop fix: reserving the final_result step and improving cap recovery.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/vertex-anthropic-loop-cap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

🤖 AI Review & Build Compliance ✅

Status: AI analysis complete • Build rules validated • Ready for review

📊 View detailed analysis results

🛡️ Analysis Complete

  • ✅ Security scan (vulnerabilities, API keys)
  • ✅ TypeScript safety & code quality
  • ✅ Error handling & best practices
  • ✅ Build rule enforcement validated
  • ✅ Commit format & compliance checks

📋 Ready for Merge When

  • All CI checks passing
  • Manual review approved
  • Any AI-flagged issues resolved

🤖 AI analysis complete - check individual code comments for specific feedback

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Complete - Approved

I've thoroughly reviewed this PR which fixes the tool_choice:"any" structured-output deadlock in the native Anthropic (Claude-on-Vertex) agentic loop.

What was reviewed:

  • package.json: New test script entry
  • src/lib/providers/googleVertex.ts: Core implementation changes (~724 lines added)
  • src/lib/types/stream.ts: Metadata type enhancement
  • test/continuous-test-suite-anthropic-cap.ts: Comprehensive test suite (28 tests)

Key verification points:

  1. ✅ Imports validated: buildToolLoopCapMessage and isAbortError correctly imported from existing exports in googleNativeGemini3.js
  2. ✅ Architecture alignment: Implementation mirrors the approved Gemini fix (PR #1123) with appropriate Anthropic-specific adaptations
  3. ✅ No breaking changes: Public API unchanged; finishReason values ("tool-calls", "length", "stop") align with existing Gemini paths
  4. ✅ Abort handling: Proper signal propagation through SDK requests with graceful terminal handling
  5. ✅ Test coverage: 28 tests covering forced finalization, abort scenarios, budget nudges, and edge cases

Notable implementation details confirmed:

  • Reserved finalization step correctly calculates agenticStepBudget = maxSteps - 1 when schema is active
  • Terminal re-check for aborts before finalization calls prevents unnecessary API requests
  • tool_choice: {type:"none"} correctly used instead of dropping tools (Anthropic API requirement)
  • finishReason exposed via both getter and metadata for wrapper compatibility

The fix addresses the root cause described in the PR: with tool_choice:"any", the text exit was unreachable, causing empty content when step budgets were exhausted. The reserved finalization step guarantees structured output delivery.

Recommendation: Approve for merge.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
test/continuous-test-suite-anthropic-cap.ts (1)

173-268: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Mocks never verify the abortSignal is actually forwarded to the SDK.

makeGenerateMock's create (lines 194-203) ignores the opts?.signal parameter its own type declares (lines 177-180), and makeStreamMock's controller.abort (lines 246-250) is a documented no-op. All abort tests (e.g. "generate: abort mid-turn", "stream: abort mid-turn") simulate cancellation purely by having onCall fire controller.abort() on the test's own AbortController and then unconditionally rejecting via behavior.reject. This means the tests would still pass even if the provider never actually passes options.abortSignal into client.messages.create()/client.messages.stream(), since the mock forces the abort-shaped rejection regardless of real wiring. Given the PR's stated goal is to "wire" abortSignal into both paths, this leaves that specific contract unverified.

♻️ Suggested tightening
 function makeGenerateMock(
   plan: (callIndex: number, params: AnthropicRequestParams) => CallBehavior,
 ): GenerateRecorder {
   const calls: AnthropicRequestParams[] = [];
+  const signals: (AbortSignal | undefined)[] = [];
   return {
     calls,
+    signals,
     client: {
       messages: {
-        create: async (p: AnthropicRequestParams) => {
+        create: async (
+          p: AnthropicRequestParams,
+          opts?: { signal?: AbortSignal },
+        ) => {
           const index = calls.length;
           calls.push(p);
+          signals.push(opts?.signal);
           const behavior = plan(index, p);
           behavior.onCall?.();
           if (behavior.reject) {
             throw behavior.reject;
           }
           return behavior.response as AnthropicResponse;
         },
       },
     },
   };
 }

Then assert mock.signals[i] === controller.signal in the abort tests to confirm real forwarding, instead of relying solely on the injected rejection.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/continuous-test-suite-anthropic-cap.ts` around lines 173 - 268, The
abort tests for the Anthropic paths are not actually verifying that the SDK
receives the AbortSignal, because `makeGenerateMock().client.messages.create`
and `makeStreamMock().client.messages.stream` ignore the `opts?.signal`/abort
wiring and only simulate cancellation via `behavior.reject`. Update the tests
around `generate` and `stream` to record the passed signal and assert it matches
the test controller’s signal, using the existing `makeGenerateMock`,
`makeStreamMock`, and the abort-related test cases to confirm real forwarding
instead of relying on injected rejection behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/providers/googleVertex.ts`:
- Around line 5117-5124: The capped-turn prose handling in googleVertex should
accumulate all mid-loop text instead of overwriting it with only the latest
non-empty chunk. Update the step-text tracking around the agentic loop to use
appendStepText() for lastStepText, and apply the same accumulation change in the
other Gemini-mirroring blocks mentioned so capped generate turns preserve
earlier prose alongside tool calls.
- Around line 4254-4261: The abort handling in googleVertex’s tool-call flow
leaves `stepStorageCalls` and `stepStorageResults` out of sync because the
aborted branch in the tool execution path (`options.abortSignal`,
`isAbortError`, `endToolSpan`) breaks before a result is recorded. Update the
abort path so it either appends a neutral cancellation result to
`stepStorageResults` or removes the corresponding pending call before
persisting, and apply the same pairing fix in the other abort sites mentioned in
the tool-loop helpers to keep replayed tool history consistent.
- Around line 4758-4764: Preserve the stream accessors when wrapping the result
in executeStream and wrapStreamResultWithLifecycle, because the current object
spread snapshots finishReason before the background loop updates
finishReasonRef. Update the wrapping logic to copy property descriptors instead
of spreading the result so finishReason remains a live getter for public
callers, and verify the same behavior still holds alongside usage and metadata
accessors.

---

Nitpick comments:
In `@test/continuous-test-suite-anthropic-cap.ts`:
- Around line 173-268: The abort tests for the Anthropic paths are not actually
verifying that the SDK receives the AbortSignal, because
`makeGenerateMock().client.messages.create` and
`makeStreamMock().client.messages.stream` ignore the `opts?.signal`/abort wiring
and only simulate cancellation via `behavior.reject`. Update the tests around
`generate` and `stream` to record the passed signal and assert it matches the
test controller’s signal, using the existing `makeGenerateMock`,
`makeStreamMock`, and the abort-related test cases to confirm real forwarding
instead of relying on injected rejection behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d8d42d55-a809-49aa-8178-e52a390e04b0

📥 Commits

Reviewing files that changed from the base of the PR and between 1191199 and d317ba9.

📒 Files selected for processing (4)
  • package.json
  • src/lib/providers/googleVertex.ts
  • src/lib/types/stream.ts
  • test/continuous-test-suite-anthropic-cap.ts

Comment thread src/lib/providers/googleVertex.ts
Comment thread src/lib/providers/googleVertex.ts
Comment thread src/lib/providers/googleVertex.ts Outdated
…ive Anthropic loop

Extends ed289b7 (9.79.3) and PR #1123 (both Gemini-only) to the native
Anthropic (Claude-on-Vertex) paths. Fixes the tool_choice:"any"
structured-output deadlock: with options.schema active, the model is
structurally forbidden from ever answering in plain text, so a turn whose
step budget was consumed by other tool calls (runaway loop, or every tool
failing) ended with content:"" and finishReason "stop" — surfaced to users
as the consumer's generic empty-content fallback (two production incidents
on 2026-07-01).

Changes, applied to BOTH executeNativeAnthropicGenerate and
executeNativeAnthropicStream:

- Reserve the last step of the budget as a forced finalization call with
  tool_choice {type:"tool", name:"final_result"} — same cache-breakpoint
  treatment as loop steps, usage counted into totals. If maxSteps <= 1 the
  finalization call is the only call.
- Soft budget nudge: when <= 3 agentic steps remain, a trailing text block
  on the tool_result user turn tells the model to consolidate and finish.
- Backstop — never return "": failed/malformed finalization falls back to
  buildToolLoopCapMessage; non-schema pure tool-loops get one
  tools-disabled synthesis call (tool_choice:"none" with tools retained —
  Anthropic rejects tool_use/tool_result history without a tools param —
  plus a system-suffix countermand), then the cap message. Both twins also
  carry an absolute empty-content guard for tool-running turns.
- Honest finishReason: "length" (max_tokens) takes precedence; capped
  turns without a clean/forced answer report "tool-calls" (aligned with
  the Gemini paths); clean and forced finishes report "stop". The stream
  twin exposes finishReason both as a result getter and on metadata (the
  mutable reference that survives wrapper result spreads, and the field
  neurolink.ts's stream span reads).
- Abort wiring in executeNativeAnthropicGenerate: loop-entry check,
  options.abortSignal passed as SDK request signal, per-step and tool-exec
  catches break gracefully via isAbortError instead of throwing into
  abortSignal-less fallback retries; aborted tool calls are not recorded
  as failures. Both twins re-check the signal at terminal-handling entry
  (covers aborts landing during the final budgeted step's tool exec),
  persist completed tools of an aborted step to conversation memory before
  breaking, and prefer prose already produced/delivered over the cap
  message (generate returns lastStepText; stream suppresses the cap chunk
  when live deltas already reached the consumer). The stream twin's abort
  now closes the stream with one graceful cap chunk instead of erroring
  the channel.

Adds test/continuous-test-suite-anthropic-cap.ts (pnpm run
test:anthropic-cap, 28 cases) covering the finalization, backstop, nudge,
abort, persistence, and finishReason matrix with a mocked Anthropic client.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pdogra1299

Copy link
Copy Markdown
Collaborator Author

Addressed all CodeRabbit feedback in the amended commit 6754f47 (single-commit policy — amended + force-pushed):

  • Paired aborted tool-call history (major): both twins now persist a neutral {aborted:true} result for the aborted call before breaking, keeping replayed tool history API-valid. Persistence test asserts calls/results pairing.
  • Stream accessors survive lifecycle wrapping (major): new preserveStreamResultAccessors() applied at both spread sites (wrapStreamResultWithLifecycle return + withClientStreamSpan setIterable), so public stream() callers see the resolved finishReason/structuredOutput after draining. metadata.finishReason stays as the wrapper-agnostic fallback. New test drains through the wrapped iterable.
  • Capped-turn prose accumulated (major): lastStepText → accumulatedStepText via the shared appendStepText() helper, mirroring the Gemini paths; earlier-step prose is no longer dropped on capped/aborted turns.
  • Abort-signal forwarding verified in tests (nitpick): makeGenerateMock now records the request-options signal and the abort test asserts options.abortSignal rides every create() call — the wiring itself is under test, not just the injected rejection.

Validation: test:anthropic-cap 29/29, test:gemini-abort 27/27, test:cache 20/20, test:providers-mocked 30/30, strict tsc + ESLint + Prettier clean.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

🤖 AI Review & Build Compliance ✅

Status: AI analysis complete • Build rules validated • Ready for review

📊 View detailed analysis results

🛡️ Analysis Complete

  • ✅ Security scan (vulnerabilities, API keys)
  • ✅ TypeScript safety & code quality
  • ✅ Error handling & best practices
  • ✅ Build rule enforcement validated
  • ✅ Commit format & compliance checks

📋 Ready for Merge When

  • All CI checks passing
  • Manual review approved
  • Any AI-flagged issues resolved

🤖 AI analysis complete - check individual code comments for specific feedback

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

I've completed a thorough review of PR #1125. This is a well-crafted fix that extends the Gemini-3 step-cap recovery pattern (PR #1123) to the native Anthropic (Claude-on-Vertex) paths.

What This PR Does Well

  1. Addresses Root Cause: Fixes the tool_choice:"any" structured-output deadlock where the text exit is structurally unreachable, preventing empty content returns when step budgets are exhausted.

  2. Comprehensive Implementation:

    • Reserved finalization step with forced final_result call
    • Proper abort signal wiring through the Anthropic SDK
    • Graceful cap recovery with buildToolLoopCapMessage
    • Tool history pairing for aborted calls
    • Accessor preservation through wrapper spreads
  3. Test Coverage: 28 tests covering schema/non-schema paths, abort scenarios, budget nudges, finish reason mapping, and lifecycle wrapping.

  4. Code Quality:

    • Follows existing patterns from the Gemini paths
    • Proper TypeScript types throughout
    • Comprehensive comments explaining the rationale
    • No hardcoded secrets or security issues

Verification

  • ✅ All existing review comments resolved
  • ✅ Imports correctly wired (isAbortError, appendStepText, buildToolLoopCapMessage, createTextChannel)
  • ✅ Abort signal properly passed to SDK requests
  • ✅ Tool call/result pairing maintained for Anthropic API compatibility
  • ✅ Getter preservation fixes the snapshot issue
  • ✅ No breaking changes to public SDK API

Recommendation

Approve - This PR is ready for merge. The fix addresses confirmed production incidents and maintains parity with the already-merged Gemini implementation.

@murdore
murdore merged commit ee44e60 into release Jul 2, 2026
16 checks passed
@murdore
murdore deleted the fix/vertex-anthropic-loop-cap branch July 2, 2026 06:59
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 9.80.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

This branch was successfully deployed

1 active deployment
Preview — 6754f474 Deployed Jul 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants