fix: partially settle chat completions aborts - #11455
Merged
Merged
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
Contributor
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Jul 2, 2026
lalalune
pushed a commit
that referenced
this pull request
Jul 2, 2026
…ll-refund (#11513) A client abort on a streaming /api/v1/messages request settled the credit reservation to 0 (full refund) in both onAbort and the stream-catch backstop, even though the platform had already paid the upstream provider for the prompt and every token delivered before the disconnect — an uncollected-revenue leak. /v1/chat/completions was fixed for this in #11455/#11472; this ports the same mechanism to /v1/messages: - accumulate delivered text-delta output during the stream - on abort (onAbort AND the catch path when the request signal is aborted), bill max(estimated input, finished-step input) + max(estimateTokens of delivered text, finished-step output) and settle the reservation to that partial cost, recording the usage as client_aborted_stream - single-flight the terminal settlement across onFinish/onAbort/onError/catch so racing abort paths cannot double-bill (mirrors #11472) - provider errors (onError / catch without an aborted signal) still refund in full; a failed partial billing falls back to a full refund via the first-call-wins idempotent settler Regression test drives the REAL createCreditReservationSettler against a ledger-backed reservation through the exported handleStream seam: red on the old route (abort settled $0, billed nothing), green with the fix. Closes #11513 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NubsCarson
added a commit
that referenced
this pull request
Jul 2, 2026
…ll-refund (#11513) (#11556) A client abort on a streaming /api/v1/messages request settled the credit reservation to 0 (full refund) in both onAbort and the stream-catch backstop, even though the platform had already paid the upstream provider for the prompt and every token delivered before the disconnect — an uncollected-revenue leak. /v1/chat/completions was fixed for this in #11455/#11472; this ports the same mechanism to /v1/messages: - accumulate delivered text-delta output during the stream - on abort (onAbort AND the catch path when the request signal is aborted), bill max(estimated input, finished-step input) + max(estimateTokens of delivered text, finished-step output) and settle the reservation to that partial cost, recording the usage as client_aborted_stream - single-flight the terminal settlement across onFinish/onAbort/onError/catch so racing abort paths cannot double-bill (mirrors #11472) - provider errors (onError / catch without an aborted signal) still refund in full; a failed partial billing falls back to a full refund via the first-call-wins idempotent settler Regression test drives the REAL createCreditReservationSettler against a ledger-backed reservation through the exported handleStream seam: red on the old route (abort settled $0, billed nothing), green with the fix. Closes #11513 Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/api/v1/chat/completionsstreaming reservations on client abort instead of refunding the full hold withsettleReservation(0)after text deltas were already sent.0.Refs #11169. This targets part 4 only; the part 3 durable stranded-reservation sweep remains separate money-infra work.
Evidence
Artifact:
.github/issue-evidence/11169-completions-abort-partial-settle.mdVerified after rebasing onto latest
origin/develop:ELIZA_SKIP_ARTIFACT_SYNC=1 bun installbun test --conditions eliza-source --pass-with-no-tests __tests__/chat-completions-streaming-credit-leak.test.ts— 7 passbun test --conditions eliza-source --pass-with-no-tests __tests__/chat-stream-credit-leak.test.ts— 6 passbun test --conditions eliza-source --pass-with-no-tests __tests__/chat-completions-optimistic-billing.test.ts— 5 passbun test --conditions eliza-source --pass-with-no-tests __tests__/chat-completions-tool-choice.test.ts— 18 passbun run --cwd packages/cloud/api typecheck— passbun run --cwd packages/cloud/api build— passbunx @biomejs/biome check packages/cloud/api/v1/chat/completions/route.ts packages/cloud/api/__tests__/chat-completions-streaming-credit-leak.test.ts— passgit diff --check HEAD— passbun run verify— passN/A: UI screenshots/video and native capture; backend billing/stream settlement route only.