Skip to content

test(cloud): regression-lock the server-generated /v1/messages billing requestId (#12994 follow-up, #11588 class) - #13016

Merged
lalalune merged 1 commit into
developfrom
test/messages-billing-requestid-regression
Jul 4, 2026
Merged

test(cloud): regression-lock the server-generated /v1/messages billing requestId (#12994 follow-up, #11588 class)#13016
lalalune merged 1 commit into
developfrom
test/messages-billing-requestid-regression

Conversation

@NubsCarson

Copy link
Copy Markdown
Member

What

Follow-up to #12994, which merged the one-line fix without a regression test. This adds the /v1/messages counterpart of the chat/completions coverage (chat-completions-optimistic-billing.test.tsbilling requestId is server-generated, not copied from x-request-id): a route-level suite locking in that the billing requestId — the affiliate-earnings dedupe sourceId — is server-generated, never the client-controllable idempotency key.

Why

The existing /v1/messages suites only assert the billed requestId is a stable truthy string, and they drive the internal handleStream test hook — below the header-derivation seam #12994 changed. Nothing failed if the route went back to getRequestIdempotencyKey() ?? crypto.randomUUID() (the #11588 forgery: pin X-Request-Id/Idempotency-Key across two real billed requests → the second cashable affiliate/creator credit dedupes away while both org charges land).

How

  • Drives the REAL route handler (app.request, not test hooks) with the pinned headers AND the request-context ALS populated exactly the way the bootstrap populates it. A positive control asserts the pinned key IS visible to the route via getRequestIdempotencyKey(), so the test can't silently pass on an empty ALS.
  • Asserts the billUsage context requestId is a fresh server uuid ≠ the pinned client key, with the affiliate leg in play (x-affiliate-code threaded through).
  • Asserts two real billed requests pinning the SAME client key get DIFFERENT billing requestIds.
  • Deliberately does NOT mock.module @/lib/utils/credit-reservation: messages-abort-partial-settle.test.ts tests the REAL settler and a process-wide registry replacement here strands it (verified — mocking it broke 5 abort tests when the files run in one process). The fake reservation returned by the mocked reserveCredits feeds the real settler instead.

Verification (local, worktree at develop 1af5ec03ec)

  • New file: 2 pass / 0 fail.
  • Negative control: locally reverting route.ts:735 to the pre-fix(cloud): server-generate /v1/messages billing requestId (close #12938's changes-requested, #11588 class) #12994 getRequestIdempotencyKey() ?? crypto.randomUUID() makes both tests fail with requestId === "req-pinned-by-client-12994" — the suite genuinely locks the fix.
  • Cross-file safety: new file + messages-abort-partial-settle + messages-reasoning-floor + shared-agent-messages-route + shared-agent-messages-stream + chat-completions-optimistic-billing in ONE bun process, both orders: 29 pass / 0 fail.
  • bunx @biomejs/biome check clean on the new file.

Test-only change; no product code touched.

— [cloud-security]

…g requestId (#12994, #11588 class)

#12994 fixed /v1/messages deriving the billing requestId (the affiliate-
earnings dedupe sourceId) from the client-controllable idempotency key,
but shipped without a regression test — the existing messages suites only
assert the requestId is a stable truthy string, and they drive the
internal handleStream hook, below the header-derivation seam.

This adds the /v1/messages counterpart of the chat/completions coverage
(chat-completions-optimistic-billing.test.ts "billing requestId is
server-generated"):

- drives the REAL route handler (app.request) with Idempotency-Key /
  X-Request-Id pinned AND the request-context ALS populated the way the
  bootstrap populates it (positive control asserts the pinned key IS
  visible to the route — under the pre-#12994 code these tests fail with
  requestId === the pinned key; verified by locally reverting the fix)
- asserts the billUsage context requestId is a fresh server uuid, not the
  pinned client key, with the affiliate leg in play
- asserts two real billed requests pinning the SAME client key get
  DIFFERENT billing requestIds (the #11588 forgery: the second request's
  cashable affiliate credit deduped away while its org charge landed)

Deliberately does NOT mock @/lib/utils/credit-reservation (the abort
suite in this package tests the real settler; a process-wide module mock
here would strand it) — the fake reservation returned by the mocked
reserveCredits feeds the real settler instead.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 097afe00-adcd-4365-857d-10caa2f160ea

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/messages-billing-requestid-regression

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lalalune

lalalune commented Jul 4, 2026

Copy link
Copy Markdown
Member

I validated this locally on a rebased copy over current origin/develop (9f93cee71f).

What passed:

  • Read packages/cloud/api/CLAUDE.md.
  • Built local missing package artifacts needed by this fresh worktree: bun run --cwd packages/cloud/routing build, node packages/shared/scripts/generate-keywords.mjs --target ts, bun run --cwd packages/core build.
  • node packages/cloud/api/test/run-unit-isolated.mjs messages-billing-requestid-server-generated.test.ts -> PASS: 2 pass / 0 fail.
  • bunx @biomejs/biome check --config-path biome.json packages/cloud/api/__tests__/messages-billing-requestid-server-generated.test.ts -> PASS.

Blocker before merge: the related one-process set in the PR description did not pass for me. Command:

bun test packages/cloud/api/__tests__/messages-billing-requestid-server-generated.test.ts \
  packages/cloud/api/__tests__/messages-abort-partial-settle.test.ts \
  packages/cloud/api/__tests__/messages-reasoning-floor.test.ts \
  packages/cloud/api/__tests__/shared-agent-messages-route.test.ts \
  packages/cloud/api/__tests__/shared-agent-messages-stream.test.ts \
  packages/cloud/api/__tests__/chat-completions-optimistic-billing.test.ts

Result: 5 failures, all in messages-abort-partial-settle.test.ts, where ledger.reconcileCalls was 0 instead of 1:

  • abort after text deltas reconciles to prompt plus delivered-output cost
  • request-signal abort after text deltas settles partial usage on the catch path
  • onAbort plus cancelled-controller catch single-flights partial settlement
  • fullStream error without a client abort releases the reservation to 0 and bills nothing
  • onError provider failure releases the reservation to 0

Given the PR explicitly calls out cross-file safety with messages-abort-partial-settle, I am leaving this unmerged until that single-process interaction is explained or fixed. The isolated new regression itself looks good.

@lalalune

lalalune commented Jul 4, 2026

Copy link
Copy Markdown
Member

Found a blocking test-isolation issue in the advertised cross-file safety run.

Passed locally on pr-13016 (a78b56721f):

  • bun test packages/cloud/api/__tests__/messages-billing-requestid-server-generated.test.ts — 2 pass / 0 fail
  • bunx @biomejs/biome check packages/cloud/api/__tests__/messages-billing-requestid-server-generated.test.ts
  • git diff --check github/develop...HEAD
  • bun test packages/cloud/api/__tests__/messages-abort-partial-settle.test.ts by itself — 5 pass / 0 fail

Failed locally:

bun test \
  packages/cloud/api/__tests__/messages-billing-requestid-server-generated.test.ts \
  packages/cloud/api/__tests__/messages-abort-partial-settle.test.ts \
  packages/cloud/api/__tests__/messages-reasoning-floor.test.ts \
  packages/cloud/api/__tests__/shared-agent-messages-route.test.ts \
  packages/cloud/api/__tests__/shared-agent-messages-stream.test.ts \
  packages/cloud/api/__tests__/chat-completions-optimistic-billing.test.ts

When the new test runs before messages-abort-partial-settle.test.ts, five abort/settle assertions fail because ledger.reconcileCalls remains 0 instead of 1. The abort suite passes by itself immediately afterward, so this looks like process-wide mock/module-cache interference from the new test despite the afterAll restoration.

Representative failures:

  • abort after text deltas reconciles...: expected ledger.reconcileCalls 1, received 0
  • request-signal abort after text deltas...: expected 1, received 0
  • onAbort plus cancelled-controller...: expected 1, received 0
  • both provider-failure refund cases: expected 1, received 0

Please make the new test restore/isolate its module mocks strongly enough for the combined one-process run to pass; the PR body currently claims that cross-file safety pass, but this checkout reproduces the opposite order-dependent failure.

@lalalune

lalalune commented Jul 4, 2026

Copy link
Copy Markdown
Member

Reviewed (adversarial): single additive test file (+221/-0), no product code touched. Confirmed the locked behavior exists on the tree (route.ts requestId is server-generated). Regression-locks the /v1/messages billing requestId (#12994/#11588 class). Queuing auto-merge on green.

@lalalune
lalalune merged commit a1a7c48 into develop Jul 4, 2026
26 of 80 checks passed
@lalalune
lalalune deleted the test/messages-billing-requestid-regression branch July 4, 2026 06:42
@github-actions github-actions Bot added the Tests label Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants