fix(token-refresh): exempt transient errors from exponential backoff - #9242
diegosouzapw merged 2 commits into
Conversation
A refresh that failed on a network timeout was treated exactly like one
that failed on a revoked token: the streak incremented and the circuit
backed off exponentially, up to four hours. A brief upstream blip could
therefore park a healthy account for the rest of the day.
Transient failures now take a flat two-minute retry window instead of
advancing the streak. Classification checks structured signals first
(err.name for AbortError/TimeoutError, then err.code and err.cause.code)
and only falls back to matching the message text, so it does not depend
on upstream wording. Everything else keeps the existing exponential path.
Two properties worth preserving on sight:
- A transient failure never shortens a longer permanent backoff. The
new window is only adopted when the existing one is not already
further out.
- testStatus is preserved on both paths, so a connection whose access
token is still valid keeps serving requests while its refresh
retries.
Only a successful refresh clears the circuit. A successful request does
not, because requests do not refresh tokens.
src/lib/tokenHealthCheck.ts lands at 1021 lines, above the 1000 cap. The file consolidates token-refresh health checking that was previously split across auth.ts and tokenRefresh.ts, and the refresh circuit state machine does not divide cleanly, so splitting it to satisfy the cap would cost more than it buys. Scoped to this file only. Baseline entries for files this branch does not touch are left at their upstream values.
a334454 to
463f0b9
Compare
|
Merged via local merge-train on 192.168.0.113 (32 cores) @ train tip Green:
For reference, |
b6bcc49
into
diegosouzapw:release/v3.8.50
…iegosouzapw#9242) * fix(token-refresh): exempt transient errors from exponential backoff A refresh that failed on a network timeout was treated exactly like one that failed on a revoked token: the streak incremented and the circuit backed off exponentially, up to four hours. A brief upstream blip could therefore park a healthy account for the rest of the day. Transient failures now take a flat two-minute retry window instead of advancing the streak. Classification checks structured signals first (err.name for AbortError/TimeoutError, then err.code and err.cause.code) and only falls back to matching the message text, so it does not depend on upstream wording. Everything else keeps the existing exponential path. Two properties worth preserving on sight: - A transient failure never shortens a longer permanent backoff. The new window is only adopted when the existing one is not already further out. - testStatus is preserved on both paths, so a connection whose access token is still valid keeps serving requests while its refresh retries. Only a successful refresh clears the circuit. A successful request does not, because requests do not refresh tokens. * chore(quality): rebaseline file-size for tokenHealthCheck.ts src/lib/tokenHealthCheck.ts lands at 1021 lines, above the 1000 cap. The file consolidates token-refresh health checking that was previously split across auth.ts and tokenRefresh.ts, and the refresh circuit state machine does not divide cleanly, so splitting it to satisfy the cap would cost more than it buys. Scoped to this file only. Baseline entries for files this branch does not touch are left at their upstream values.
Summary
OAuth token refresh failures were treated identically to permanent errors: any refresh failure (including network timeouts) triggered exponential backoff up to 4 hours. A transient network blip could take an account out of rotation for hours.
This fix classifies refresh failures and exempts transient network errors (timeout, connection reset, DNS failure) from the exponential backoff ladder. Transient errors get a flat 2-minute retry window instead.
Changes
buildTransientRefreshRetryUpdate()at tokenHealthCheck.ts:167: sets a flat 2-min retry window, preserves existing permanent streakuseTransient = existingUntil <= transientUntilat line 187: transient failure never shortens a longer permanent backofftransientflag always set on refreshCircuit for observabilityDesign decisions
Test plan
UNRESOLVED