Skip to content

fix(sse): prevent abandoned submissions from blocking the rate-limit queue - #18526

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.53from
shevar0:fix/cancel-received-storm
Nov 4, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.53from
shevar0:fix/cancel-received-storm

Conversation

@shevar0

@shevar0 shevar0 commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Why

A client abort can leave thousands of jobs in Bottleneck's RECEIVED submission tail. Waiting for that entire tail before removing an already-queued request makes live traffic wait behind dead work; the abandoned jobs can consume later reservoir tokens and scheduling slots even though their callers have gone away.

Change

  • Unlink a waiting RECEIVED submission synchronously and settle its submission promise.
  • Remove an already-QUEUED job under the registration lock, without waiting for unrelated submissions.
  • Preserve FIFO/list counters and the existing defensive fallback when the pinned Bottleneck internals are unavailable. Never remove running/executing work.
  • Settle the caller's original abort/timeout cause before immediate cleanup can expose Bottleneck's internal drop message.

Regression coverage

New tests cover a 2,000-submission cancellation storm, QUEUED cleanup behind a blocked submission tail, FIFO after removing head/middle/tail submissions, reservoir/minTime preservation, native/custom abort reasons, and queue-timeout cause preservation. Existing queue-wedge and per-connection limiter tests are included in the focused run.

Verification

  • TDD: the new RECEIVED-storm / blocked-submission-lock reproductions fail on the unmodified base, then pass with this patch.
  • Focused Node suite: 41/41 passed (storm, cause custody, abandoned queue, queue bounds/context, wedge recovery, per-connection execution and rate-limit-manager tests). Run with node --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=2 tests/unit/rate-limit-cancel-received-storm.test.ts tests/unit/rate-limit-cancel-cause-custody.test.ts tests/unit/rate-limit-abandoned-queued-job.test.ts tests/unit/rate-limit-manager-queue-bound.test.ts tests/unit/rate-limit-queue-context.test.ts tests/unit/rate-limit-wedge-recovery.test.ts tests/unit/rate-limit-execution-per-conn.test.ts tests/unit/rate-limit-manager.test.ts.
  • Complete Vitest backend: 56 files / 588 tests passed, via node node_modules/vitest/vitest.mjs run --config vitest.mcp.config.ts --maxWorkers=1 --fileParallelism=false.
  • Changed-file Prettier + ESLint, tsc -p tsconfig.typecheck-core.json, error-helper gate and all enabled commit hooks passed.
  • Node v22.23.2 on Windows. Test processes additionally loaded a local, uncommitted preloader that blocks external sockets (loopback allowed); no production code or assertions were stubbed for these runs.
  • The repository-wide Node suite, production build and coverage gate were not run for this focused port; this is not a claim that all base CI is green.

⚠️ base-red inherited: #17219 (release/v3.8.53). This PR does not attempt unrelated base repairs.

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@diegosouzapw
diegosouzapw merged commit bf6faf9 into cheaper-inference:release/v3.8.53 Nov 4, 2026
3 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants