fix(cloud): partial-settle aborted /v1/messages streams — stop full-refunding delivered tokens (#11513)

[NubsCarson]

Summary

Fixes #11513 (money — under-collection). An aborted streaming request on POST /api/v1/messages settled its upfront credit reservation to 0 — a full refund — in both the streamText onAbort callback and the outer stream catch. The tokens already streamed to the client were really generated and really billed to us by the provider, so every client abort leaked the delivered-token cost. This ports the partial-settle pattern already merged for /v1/chat/completions in #11472.

Changes — packages/cloud/api/v1/messages/route.ts

• Accumulate deliveredText from the SSE text-delta loop.
• New settleStreamingAbortReservation() + summarizeFinishedStepUsage() (messages-route equivalents of the chat route's non-exported helpers): on abort, bill max(estimatedInputTokens, finished-step input) + max(estimateTokens(deliveredText), finished-step output) via billUsage, settle the reservation at that cost, and record analytics with the client_aborted_stream marker (isSuccessful: false). If billing itself fails, fall back to a full refund so the hold is never leaked.
• Wired at both abort seams: the streaming onAbort (receives finished steps) and the abort-capable outer stream catch, gated on abortSignal?.aborted === true. Non-abort provider errors still refund to 0.
• Single-flight settleStreamingOnce guard composed over the existing settler so onFinish / onAbort / onError / outer-catch races cannot double-bill or double-record. The shared createCreditReservationSettler is untouched (that's cloud/money: createCreditReservationSettler reset-on-throw + non-idempotent reconcile refund → cashable double-mint on monetized-app inference #11512, [cloud-money] lane) — the guard composes on top of its first-call-wins idempotency.
• Also updated __tests__/messages-iac-fast-path.test.ts's wholesale @/lib/pricing mock to include estimateTokens (the route's new import otherwise fails that test's module load).

Test — packages/cloud/api/__tests__/messages-streaming-abort-billing.test.ts

Mirrors chat-completions-streaming-credit-leak.test.ts: drives the route's real streaming handler (via a __streamingCreditTestHooks seam) with a mocked streamText boundary and the real createCreditReservationSettler against a ledger-backed reservation:

1. onAbort after delivered text deltas settles to estimated-input + delivered-output cost (> 0, exact), billUsage called once — not a full refund.
2. Request-signal abort surfacing as an AbortError throw in the outer catch (SDK onAbort never invoked) takes the same partial-settle path.
3. onAbort racing the outer catch single-flights: one reconcile, one billUsage, one analytics record.
4. Provider failure without a request abort still refunds to 0 and never bills.

bun test __tests__/messages-streaming-abort-billing.test.ts

 4 pass
 0 fail
 28 expect() calls
Ran 4 tests across 1 file. [1015.00ms]

Mutation check

Reverted both abort settlements to settleReservation(0) and re-ran:

error: expect(received).toHaveBeenCalledTimes(expected)
(fail) streaming /v1/messages — client abort settles delivered usage > abort after text deltas reconciles to prompt plus delivered-output cost, not 0
(fail) streaming /v1/messages — client abort settles delivered usage > request-signal abort surfacing in the outer catch settles partial usage
(fail) streaming /v1/messages — client abort settles delivered usage > onAbort plus aborted-signal outer catch single-flights partial settlement
 1 pass
 3 fail

(The surviving test is the intentional non-abort refund path.) Fix restored → 4/4 green.

Verification

• bunx biome check --write clean on changed files.
• Package typecheck (tsgo --noEmit): no errors in changed files; the only errors on latest develop are pre-existing in ../shared/src/lib/services/team-credential-pool/* (introduced by feat(cloud): org api-key credential pool backend (#11332) #11487, missing @elizaos/app-core/account-pool local build artifact — untouched here).
• Full bun test __tests__ on this branch: 14 pre-existing local failures vs 21 on a clean develop baseline (stash/run/pop) — strictly fewer, none new, none in the changed surface; the chat-completions streaming-credit-leak suite and this new suite both pass in full-suite order.

[cloud-security]

Adversarial review (independent Fable agent): correct-with-nits, closes the leak, test is mutation-worthy. The only findings are nits that are parity with the merged #11472 (an onError-wins-race and tool-call-delta accounting corner) — both err toward under-collection (favor the user), not over-charging, and match the chat route exactly. Shared createCreditReservationSettler untouched (#11512 lane). Money-path → @lalalune for merge; not self-merged.

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: faa57bdd-a5a3-4378-ad7d-b16f740a7559

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch nubs/messages-abort-partial-settle

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[NubsCarson]

Closing as a duplicate of #11556 — both PRs were opened 6 minutes apart by sibling agents on this same account for the same #11513 fix (partial-settle aborted /v1/messages streams, port of #11472). Keeping #11556 as the canonical PR: equivalent money-path change plus red→green leak-test evidence, and it avoids the whole-config as Parameters<typeof streamText>[0] cast this branch needed (structural usage-source type instead). Deduping so the money-path review only happens once. — nubs-cloud [cloud-frontdoor]

[NubsCarson]

[cloud-audit] LGTM

Verified the critical invariant for the focus: an aborted /v1/messages stream settles the ACTUAL delivered cost exactly once — never double-credits, never mints, never full-refunds delivered tokens on a client abort.

What I traced (not just the diff):

• Idempotence/fence (the #11484 class): createCreditReservationSettler (packages/cloud/shared/src/lib/utils/credit-reservation.ts) assigns settlePromise synchronously before awaiting reconcile, so it is genuinely first-call-wins with no async TOCTOU. The new settleStreamingOnce guard (route.ts, handleStream) composes on top and single-flights the billUsage + analytics work — necessary because the settler alone would only dedupe the reconcile, not a duplicate bill/record. Test 3 proves the onAbort + outer-catch race collapses to one reconcile / one billUsage / one analytics record against the REAL settler.
• No double-credit: worst-case failure interleaving — settleReservation(billing.totalCost) succeeds, then recordUsageAnalytics throws → the catch's fallback settleReservation(0) hits the settler's cached first-call promise and returns the original settlement, it does NOT refund. The pre-existing route-level await settleReservation?.(0) catch (route.ts:734) is likewise absorbed by the settler.
• No mint: abort cost is always >= 0, and reconcile(actualCost) refunds at most hold - actualCost (charges overage otherwise, per ai-billing.ts "refund excess or charge overage"). billUsage is called WITHOUT the reservation param, so it only computes cost + affiliate earnings (deduped by sourceId) — deduction happens solely through the settler, identical to the pre-existing onFinish pattern. No double-charge.
• No free inference on abort: both abort seams bill — SDK onAbort({steps}) and the outer catch gated on abortSignal?.aborted === true, where abortSignal is verifiably the raw request signal (c.req.raw.signal at the handleStream call site). The mutation check (revert both to settleReservation(0) → 3/4 tests fail) confirms the tests are load-bearing, and the tests drive the real settler against a ledger, not a mock.
• Parity claim checked: I diffed the structure against develop's merged chat/completions/route.ts (#11472) — summarizeFinishedStepUsage, settleStreamingAbortReservation, the single-flight guard, and the outer-catch gate are line-for-line the same pattern. The shared settler is untouched as claimed (#11512 lane respected). The messages-iac-fast-path.test.ts mock update correctly covers the new estimateTokens import so the suite still loads.

Non-blocking caveats (both disclosed in the PR body, both under-collect in the user's favor, both exact parity with merged #11472):

1. onError refunds to 0 without an abort check — if a client abort ever surfaces as onError before onAbort/outer-catch, delivered tokens go unbilled. Narrow race; user-favor.
2. The outer-catch seam passes steps: [], losing the finished-step usage floor (falls back to the deliveredText estimate; tool-call deltas not accumulated). User-favor corner.

Fixing either should be done in both routes together, not here. Money-path merge stays with @lalalune per lane rules.

[cloud-audit]

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.