Skip to content

test(sse): update three backoff assertions stale since the #8396 cooldown cap - #8539

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.49from
MumuTW:test/backoff-cap-8396-stale-assertions
Jul 25, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.49from
MumuTW:test/backoff-cap-8396-stale-assertions

Conversation

@MumuTW

@MumuTW MumuTW commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Problem

Three unit tests fail on release/v3.8.49 at its own HEAD (7a8f9156d), with no PR content applied. They show up on the Unit Tests fast-path shards:

Test File Expected Actual
502 transient: exponential backoff doubles until the configured max backoff step error-classification.test.ts 160000 120000
high transient backoff levels clamp to the configured maxBackoffSteps error-classification.test.ts 163840000 120000
Exponential backoff clamps to the configured maxBackoffLevel thundering-herd.test.ts 163840000 120000

Root cause: the tests assert the behavior #8396 deliberately removed

163840000 ms is 45.5 hours — transientInitial * 2^maxLevel. That unbounded growth is exactly what #8396 fixed. From the header of the module it added:

accountFallback/cooldownCap.ts — [...] Fixes #8396 — after a sustained 429 burst pushed backoffLevel high, baseCooldownMs * 2^level had no upper bound on this path and could black a connection out for hours, long past any real rate-limit reset window.

capScaledCooldownMs now bounds every scaled cooldown by the operator-configured profile.maxCooldownMs, falling back to BACKOFF_CONFIG.max when the profile does not set one. All three cases call checkFallbackError(502, "", level, null, null) with a null provider, so no profile resolves and the fallback ceiling applies — hence 120000 across the board.

So the production behavior is correct and intentional; the assertions are stale. #8396 landed the cap and its own test but did not update these three pre-existing cases. (The same PR also left its test unregistered in stryker.conf.json — that half is #8538.)

Change

Only the expected durations move. The backoff-level clamping each case was written to guard (newBackoffLevel === maxLevel / === level + 1) is untouched and still asserted.

  • Expectations keep the original formula wrapped in the cap — Math.min(COOLDOWN_MS.transientInitial * Math.pow(2, BACKOFF_CONFIG.maxLevel), BACKOFF_CONFIG.max) — rather than a bare 120000, so the relationship between the exponential series and the ceiling stays readable and survives a constant change.
  • The first case gains a precondition assertion:
assert.ok(
  COOLDOWN_MS.transientInitial * 32 > BACKOFF_CONFIG.max,
  "precondition: the 6th step must exceed the cap, or this test proves nothing"
);

so it cannot silently become vacuous if transientInitial or BACKOFF_CONFIG.max is retuned later.

Verification

node --import tsx/esm --test tests/unit/error-classification.test.ts tests/unit/thundering-herd.test.ts
# tests 33 / pass 33 / fail 0

eslint exit 0 and prettier --check clean on both files. No production code touched.

Note for maintainers

CLAUDE.md → Resilience Runtime State → Connection Cooldown still documents the pre-#8396 rule:

Repeated recoverable failures use exponential backoff: baseCooldownMs * 2 ** failureIndex

with no mention of the ceiling. That is now inaccurate. I left it alone to keep this PR to the test change — happy to send a docs follow-up if you want the cap documented there.

…pw#8396 cooldown cap

These three cases assert that a transient-error cooldown keeps doubling
to baseCooldownMs * 2^maxLevel — roughly 45.5h at the default constants.
That is precisely the blackout diegosouzapw#8396 removed: capScaledCooldownMs
(open-sse/services/accountFallback/cooldownCap.ts) now bounds every
scaled cooldown by profile.maxCooldownMs, falling back to
BACKOFF_CONFIG.max when the profile does not configure one. All three
call checkFallbackError with a null provider, so the fallback ceiling
applies and the observed value is BACKOFF_CONFIG.max.

The backoff-level clamping each case was written to guard is unchanged
and still asserted; only the expected duration moved. The expressions
keep the original formula wrapped in the cap so the relationship stays
readable, and the first case gains a precondition assertion so it cannot
silently become vacuous if the constants change.

Fixes the error-classification (x2) and thundering-herd (x1) failures
that are red on release/v3.8.49 at its own HEAD.
@diegosouzapw

Copy link
Copy Markdown
Owner

Reviewed and verified independently. On a clean checkout of release/v3.8.49 (current tip), the three cited tests fail exactly as described in the PR body (160000 vs actual 120000; 163840000 vs actual 120000, twice). With this PR's diff applied, all 33 tests in both files pass. I also read open-sse/services/accountFallback/cooldownCap.ts and its call site in accountFallback.ts — the 120000ms cap (BACKOFF_CONFIG.max, applied via capScaledCooldownMs when no provider profile sets maxCooldownMs) is the intentional, already-merged production behavior from #8396 (commit b64361d), not a regression. So this correctly updates stale pre-#8396 assertions rather than masking a bug — the expectations still assert the original doubling formula wrapped in Math.min(...) (not a bare literal), the backoff-level clamp in each case is untouched, and the precondition assertion added to the first case is a nice touch to keep it from going vacuous. ESLint and Prettier are clean on both files, and this diff doesn't touch production code, so no functional risk. Looks solid to me — merge decision is the maintainer's call.

@diegosouzapw
diegosouzapw merged commit d06d3fb into diegosouzapw:release/v3.8.49 Jul 25, 2026
5 checks passed
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…pw#8396 cooldown cap (diegosouzapw#8539)

These three cases assert that a transient-error cooldown keeps doubling
to baseCooldownMs * 2^maxLevel — roughly 45.5h at the default constants.
That is precisely the blackout diegosouzapw#8396 removed: capScaledCooldownMs
(open-sse/services/accountFallback/cooldownCap.ts) now bounds every
scaled cooldown by profile.maxCooldownMs, falling back to
BACKOFF_CONFIG.max when the profile does not configure one. All three
call checkFallbackError with a null provider, so the fallback ceiling
applies and the observed value is BACKOFF_CONFIG.max.

The backoff-level clamping each case was written to guard is unchanged
and still asserted; only the expected duration moved. The expressions
keep the original formula wrapped in the cap so the relationship stays
readable, and the first case gains a precondition assertion so it cannot
silently become vacuous if the constants change.

Fixes the error-classification (x2) and thundering-herd (x1) failures
that are red on release/v3.8.49 at its own HEAD.
@MumuTW
MumuTW deleted the test/backoff-cap-8396-stale-assertions branch September 5, 2026 10:19
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…pw#8396 cooldown cap (diegosouzapw#8539)

These three cases assert that a transient-error cooldown keeps doubling
to baseCooldownMs * 2^maxLevel — roughly 45.5h at the default constants.
That is precisely the blackout diegosouzapw#8396 removed: capScaledCooldownMs
(open-sse/services/accountFallback/cooldownCap.ts) now bounds every
scaled cooldown by profile.maxCooldownMs, falling back to
BACKOFF_CONFIG.max when the profile does not configure one. All three
call checkFallbackError with a null provider, so the fallback ceiling
applies and the observed value is BACKOFF_CONFIG.max.

The backoff-level clamping each case was written to guard is unchanged
and still asserted; only the expected duration moved. The expressions
keep the original formula wrapped in the cap so the relationship stays
readable, and the first case gains a precondition assertion so it cannot
silently become vacuous if the constants change.

Fixes the error-classification (x2) and thundering-herd (x1) failures
that are red on release/v3.8.49 at its own HEAD.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(resilience): combo fallback stays blacked out for hours after a 429 burst, past the real rate-limit window

2 participants