Skip to content

fix(rate-limit): never start the work of a caller aborted before dispatch - #16566

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.52from
shipsfromrio:fix/rate-limit-aborted-caller-dispatch
Oct 16, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.52from
shipsfromrio:fix/rate-limit-aborted-caller-dispatch

Conversation

@shipsfromrio

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #15306

Summary

withRateLimit() could still run the scheduled work of a caller it had already rejected with
AbortError. The abort lands while withRateLimit awaits awaitProviderDefaultSlot().
onAbort() then tries to pull the job from the queue, but with limiter capacity free Bottleneck
has already dispatched it, so fn runs with an already-aborted signal. This PR adds the missing
check to wrappedFn, next to the existing queue-timeout check.

The visible symptom is in the test suite. tests/unit/rate-limit-manager.test.ts passes in about
30 s and then keeps the Node process alive for about 10 more minutes. That happens when the file is
run the way AGENTS.md documents a focused run (node --import tsx/esm --test <file>, without
--test-force-exit).

Symptom

Base 9ecddd15a1, file run on its own with --test-concurrency=1:

Platform Run Result
Windows 11, Node 24.19.0 as is 19/19 pass, the process exits after about 640 s
Windows 11, Node 24.19.0 last test skipped 18/18 pass, exits in 29 s
Windows 11, Node 24.19.0 --test-force-exit 19/19 pass, exits in 27 s
Linux (WSL2), Node 24.21.0 as is still running when timeout killed it at 195 s
Linux (WSL2), Node 24.21.0 last test skipped exits in 58 s
Linux (WSL2), Node 24.21.0 --test-force-exit exits in 34 s

An async_hooks preload dumped the live, ref'd resources right after the root after() hook. It
found one Timeout of 600000 ms, created in Bottleneck's _run() (through patchedRun in
open-sse/services/bottleneckPatch.ts). That is the expiration timer, executionMaxWaitMs
(10 min by default), of a job that never finished.

Root cause

  1. withRateLimit() checks signal.aborted once on entry, then awaits awaitProviderDefaultSlot().
  2. The last test in the file, "rejects cleanly when the caller aborts with the default
    DOMException reason", calls controller.abort() during that await.
  3. After the await, limiter.schedule() runs. onAbort() then runs at once, because the signal
    is already aborted: it calls abandonQueuedJob() and rejects the caller.
  4. cancelQueuedJob() (open-sse/services/rateLimitManager/queuedJobCancel.ts) waits for
    Bottleneck's _submitLock and removes the job only while it is still QUEUED. But
    _addToQueue() runs _drainAll() inside that same lock. With capacity free, the job has
    already been dispatched when the cancel gets the lock, so the cancel does nothing.
  5. wrappedFn only checked queueTimedOut, so it called fn with the aborted signal. The test's
    fn registers an abort listener on a signal that has already fired, so it never settles. The
    job stays EXECUTING, and Bottleneck's ref'd expiration timer keeps the process alive until the
    timer fires 10 minutes later.

In production the same path starts work for a caller that has already given up:

  • executeProviderRequest.ts marks the request rate_limit_slot_acquired and enters the executor;
  • modelTestRunner.ts enters the internal chat or responses pipeline.

Both get an aborted signal. Work that honors an already-aborted signal ends at once. Work that only
waits for a future abort event holds the limiter slot until executionMaxWaitMs expires the job.

Fix

One line in wrappedFn:

if (signal?.aborted) return Promise.reject(signal.reason);

It mirrors the existing queueTimedOut check. The rejection never reaches the caller:

  • scheduled.catch(() => {}) swallows it;
  • the race has already settled with the abort. onAbort() rejects abortPromise as soon as the
    signal fires, before Bottleneck can run wrappedFn.

The comment above the race now explains why a job can already be past QUEUED at that point.

Tests

  • New test: "withRateLimit never starts the work of a caller that aborted before dispatch". It
    aborts while withRateLimit awaits its provider slot, then asserts that:
    • the caller gets AbortError;
    • the limiter drains (RECEIVED + QUEUED + RUNNING + EXECUTING === 0);
    • fn was never called.

The drain assertion is the symptom check, scoped to the limiter: a drained limiter holds no
executing job, so no expiration timer is left to keep the process alive. There is no process-wide
check such as process.getActiveResourcesInfo() in the root after() hook. Under
test:unit:fast (--test-isolation=none) all files share one process, so that check would also
see the timers of every other file.

Validation

Run on GitHub Actions (ubuntu-24.04, Node 24.21.0 from .nvmrc) as a focused run,
node --import tsx/esm --test tests/unit/rate-limit-manager.test.ts, without
--test-force-exit. It ran twice: once with the branch, and once with
open-sse/services/rateLimitManager.ts restored to 9ecddd15a1 while the test file stays.

Tree Result Run ended after
this PR 20 tests, 20 pass 7.9 s
rateLimitManager.ts from 9ecddd15a1 20 tests, 19 pass; the new test fails, fn ran once (1 !== 0) 606 s

On the base, the new test fails about 5 s into the run. The remaining 600 s is the symptom: the
process waits for the 600000 ms expiration timer of the last test's job.

Earlier local runs used the same source change and an earlier form of the new test:

  • Linux (WSL2), Node 24.21.0: 20/20 in each of 3 rounds; the process exited in 28 s, 32 s and
    23 s.
  • Windows 11, Node 24.19.0: the process exited on its own in about 42 s, with 19 of 20 passing.
    The machine was under heavy load, and the timing-sensitive watchdog test "preserves a real
    Bottleneck minTime wait" failed. It also fails on the base under the same load, and it does not
    involve an aborted signal.

Static checks:

  • ESLint and Prettier on the changed files: clean.
  • tsc on the test file: only the 4 existing TS2503 errors (Cannot find namespace 'Bottleneck'),
    none from this change. npm run typecheck:core does not include these two files.
  • check:changelog-integrity against release/v3.8.52: OK.
  • check:file-size:
    • rateLimitManager.ts is 1313 lines (frozen at 1329);
    • the test file is 1014 lines (cap 1200);
    • the gate is red on the base for an untouched file
      (open-sse/services/combo/executeTargetAttempt.ts: 1288 > 1284).

Notes

…atch

withRateLimit() checks signal.aborted once on entry and then awaits
awaitProviderDefaultSlot(). An abort that lands during that await is
handled by onAbort(), which calls abandonQueuedJob() and rejects the
caller with AbortError. With limiter capacity free, though, Bottleneck
dispatches the job inside its submit lock, so the job has already left
QUEUED when cancelQueuedJob() gets that lock and the removal is a no-op.
wrappedFn then called fn with the already-aborted signal.

Work that honors an already-aborted signal ends at once. Work that only
listens for a later "abort" event keeps the limiter slot, and Bottleneck's
ref'd expiration timer, until executionMaxWaitMs (10 min by default)
expires the job. The last test in tests/unit/rate-limit-manager.test.ts
does exactly that, so without --test-force-exit the file passed in about
30 s and then kept the process alive for about 10 more minutes.

wrappedFn now rejects with the abort reason before calling fn, the same
way it already handled a queue-wait timeout. The caller never sees that
rejection: the race has already settled with the AbortError.

Tests: a new case aborts while withRateLimit awaits its provider slot and
asserts that fn never runs and that the limiter drains. A drained limiter
holds no executing job, so no expiration timer is left to keep the
process alive.
@diegosouzapw
diegosouzapw merged commit b3807a0 into cheaper-inference:release/v3.8.52 Oct 16, 2026
30 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants