fix(proxy): honor allow_requests_on_db_unavailable in /health/readiness

[devin-ai-integration]

TLDR

Problem this solves:

• /health/readiness returns 503 during a DB outage
• it ignores allow_requests_on_db_unavailable, pulling every pod out of rotation

How it solves it:

• gate the readiness 503 flip behind the same fail-open flag
• flag on plus unreachable DB now stays 200 with db: disconnected

Relevant issues

Fixes #34934

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Live proxy against a real Postgres (allow_requests_on_db_unavailable: true), then Postgres stopped mid-run to simulate the outage. The request layer already fails open on this flag; the probe now does too, so Kubernetes keeps the pod in the Service endpoints instead of yanking every replica

Before the fix (commit daf22ec871, the parent of this branch)

# DB up
$ curl -s -w " HTTP %{http_code}\n" http://localhost:4000/health/readiness
{"status":"healthy","db":"connected"} HTTP 200

# docker stop postgres, wait out the 15s readiness cache, then:
$ curl -s -m 30 -w " HTTP %{http_code}\n" http://localhost:4000/health/readiness
{"status":"healthy","db":"disconnected"} HTTP 503

After the fix (commit ec40a2bfb3)

# DB up
$ curl -s -w " HTTP %{http_code}\n" http://localhost:4000/health/readiness
{"status":"healthy","db":"connected"} HTTP 200

# docker stop postgres, wait out the 15s readiness cache, then:
$ curl -s -w " HTTP %{http_code}\n" http://localhost:4000/health/readiness
{"status":"healthy","db":"disconnected"} HTTP 200

Type

🐛 Bug Fix

Changes

allow_requests_on_db_unavailable is the HA switch that keeps the proxy serving when the database is unreachable; PrismaDBExceptionHandler.handle_db_exception consults it and fails open at the request layer. /health/readiness never did. When a Prisma DB was configured but unreachable, both _resolve_public_readiness_db and _get_health_readiness_details flipped the response to 503 unconditionally, so a ~30s outage marked every replica NotReady at once and Kubernetes pulled them all out of the Service endpoints; the request-layer fail-open then never got a chance to run. This was a regression: before #26134 the flag did gate readiness

The fix adds _db_unavailable_should_flip_readiness_to_503, which returns not PrismaDBExceptionHandler.should_allow_request_on_db_unavailable(), and both readiness paths now consult it before setting 503. Default behavior is unchanged (unreachable configured DB still 503, no DB configured still 200); only operators who explicitly set the flag now keep serving through an outage

Scope note: this PR fixes the primary bug (the flag gating readiness). The issue also raises secondary Helm-probe hardening items (/health/liveliness not restarting stuck pods, startupProbe sharing the readiness path); those are separate config-surface changes and are intentionally left out to keep this isolated

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/7fa143c1fd3740f592ba77469bd7acc6

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR makes both public and detailed readiness checks honor allow_requests_on_db_unavailable, preserving HTTP 200 during database outages when fail-open behavior is enabled, and adds regression coverage for enabled and default configurations.

Confidence Score: 5/5

The PR appears safe to merge because both readiness paths now consistently honor the existing database fail-open setting while retaining the default 503 behavior.

The helper uses the same configuration interpretation as the existing request-layer exception handling, both database-related readiness status assignments invoke it, and the added tests cover disconnected-database behavior with the setting enabled and absent.

Important Files Changed

Filename	Overview
litellm/proxy/health_endpoints/_health_endpoints.py	Centralizes the database-outage readiness decision and consistently applies it to both readiness response paths without changing default behavior.
tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py	Adds focused regression tests for public and detailed readiness behavior with fail-open enabled and disabled; other changes are formatting-only.

_{Reviews (1): Last reviewed commit: "fix(proxy): honor allow_requests_on_db_u..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_health_readiness_db_unavailable_flag (ec40a2b) with litellm_internal_staging (daf22ec)}