fix(router): allow APIConnectionError to trigger cooldown for failover

[Jwrede]

Relevant issues

Fixes #27362

Pre-Submission checklist

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

• Bug Fix

Changes

_is_cooldown_required() in cooldown_handlers.py contained a hardcoded
ignored_strings = ["APIConnectionError"] that suppressed cooldown for any
exception whose string representation included "APIConnectionError". This
prevented the router from cooling down unreachable hosts, so all retries hit
the same dead deployment instead of failing over to healthy ones.

The fix removes that blanket exclusion. APIConnectionError indicates the
host is unreachable -- exactly the error that should trigger cooldown and
failover. Single-deployment model groups are already protected from
unnecessary cooldown by the traffic-based logic in
_should_cooldown_deployment(), which only cools down single-deployment
groups when 100% of requests fail under high traffic.

Files changed

• litellm/router_utils/cooldown_handlers.py -- removed ignored_strings
  check from _is_cooldown_required()
• tests/test_litellm/router_utils/test_cooldown_handlers.py -- added unit
  tests for _is_cooldown_required() covering APIConnectionError, 5xx, 429,
  4xx, and empty status scenarios

[greptile-apps]

Greptile Summary

This PR removes a hardcoded ignored_strings = ["APIConnectionError"] guard from _is_cooldown_required() that was preventing the router from cooling down unreachable deployments, causing all retries to hammer the same dead host instead of failing over. The fix is minimal and targeted, and single-deployment model groups retain protection via the rate-based thresholds in _should_cooldown_deployment().

• cooldown_handlers.py: Eight lines deleted — the ignored_strings list and its early-return check. No other logic is touched.
• test_cooldown_handlers.py: New mock-only test file that directly exercises _is_cooldown_required() for APIConnectionError (status 500), 5xx, 429, generic 4xx, and empty-status inputs, confirming the regression is covered.

Confidence Score: 4/5

Safe to merge — the change is a small, well-tested deletion that restores correct failover behavior for unreachable deployments.

The change is a clear bug fix with direct test coverage. litellm.APIConnectionError hard-codes status_code = 500, so the test's exception_status=500 accurately reflects real usage. Single-deployment groups are shielded by the rate-based gate in _should_cooldown_deployment(). The only non-trivial concern is that existing users relying on the old suppression behavior will silently see a behavior change, but this was the root cause of the reported failover regression.

No files require special attention; both changed files are straightforward.

Important Files Changed

Filename	Overview
litellm/router_utils/cooldown_handlers.py	Removes the ignored_strings = ["APIConnectionError"] guard from _is_cooldown_required(), allowing connection errors to trigger deployment cooldown and failover as intended.
tests/test_litellm/router_utils/test_cooldown_handlers.py	New unit-test file covering _is_cooldown_required() with mock-only tests for APIConnectionError, 5xx, 429, 400, and empty-status inputs. All tests use mocks; no real network calls.

Comments Outside Diff (1)

1. litellm/router_utils/cooldown_handlers.py, line 40-45 (link)

   After removing the ignored_strings block, exception_str is no longer read anywhere inside _is_cooldown_required() — it is accepted but silently ignored. Callers (e.g. _should_run_cooldown_logic) still pass it, so there is no breakage, but the dead parameter can mislead future contributors into thinking it affects the outcome. Consider removing the parameter from the signature (and the call sites) or adding a comment that it is reserved for future use.

_{Reviews (1): Last reviewed commit: "fix(router): allow APIConnectionError to..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Jwrede]

Friendly ping @ishaan-jaff @krrishdholakia -- this and several other fix PRs (#27426, #27540, #27542, #27548, #27551, #27558, #27593, #27597, #27598, #27599, #27600, #27601, #27608) have been open 3-6 days. All CI green, all targeting litellm_internal_staging. Happy to address any feedback.

[oss-pr-review-agent-shin]

🤖 litellm-agent: This PR was marked BLOCKED 7 days ago with no subsequent activity. Closing automatically.