fix(proxy): self-heal Prisma read paths + harden reconnect state machine

[yuneng-berri]

Summary

Two related fixes layered on top of the existing Prisma reconnect plumbing.

1. Restore reconnect-and-retry on PrismaClient.get_generic_data (issue #25143)

#25143 reports that 1.83.x lost the transport-reconnect-and-retry-once branch that 1.82.6 had on get_generic_data. Transient httpx.ReadError flaps now surface immediately as db_exceptions alerts. _update_config_from_db fans out four concurrent get_generic_data reads, so a single transport blip used to mark four alerts and a stale config window.

This PR restores that self-heal — but instead of inlining the pattern (the auth path at auth_checks._fetch_key_object_from_db_with_reconnect and the readiness path at _health_endpoints already had two drifting inline copies), it factors the pattern into a single canonical helper:

```python
async def call_with_db_reconnect_retry(
prisma_client, coro_factory, *, reason,
timeout_seconds=None, lock_timeout_seconds=None,
):
```

placed next to PrismaDBExceptionHandler in litellm/proxy/db/exception_handler.py. get_generic_data now wraps its query block in this helper. Future read paths can opt in by passing a zero-arg coroutine factory.

2. Fix the `_engine_confirmed_dead` flag-reset bug in `_run_reconnect_cycle`

_run_reconnect_cycle cleared self._engine_confirmed_dead = False before awaiting _do_heavy_reconnect(). If the heavy reconnect raised (timeout, missing DATABASE_URL, recreate failure), the flag was left cleared and the next attempt could silently demote to the lightweight path even though the engine was genuinely dead.

Move the reset into the success branch so the flag stays True across heavy-reconnect failures, and the next attempt correctly re-enters the heavy branch.

Out of scope

• The event-loop-blocking `disconnect()` is addressed separately by [Fix] Proxy: reconnect Prisma DB without blocking the event loop #26225 — orthogonal change, this PR does not depend on or conflict with it.
• The auth + health inline copies of the retry pattern can be replaced with call_with_db_reconnect_retry in a follow-up PR; deferred to keep this change focused.

Test plan

• `pytest tests/test_litellm/proxy/db/test_exception_handler_reconnect_retry.py` — 9 helper tests pass
• `pytest tests/test_litellm/proxy/db/test_prisma_self_heal.py` — 16 tests pass (13 existing + 3 new)
• `pytest tests/test_litellm/proxy/db/ tests/litellm/proxy/test_prisma_engine_watchdog.py` — 197 pass
• `pytest tests/test_litellm/proxy/auth/ tests/test_litellm/proxy/db/test_exception_handler.py` — 535 pass
• `uv run black .` — clean
• `make lint-ruff` — clean

[greptile-apps]

Greptile Summary

This PR delivers two targeted fixes to the Prisma reconnect state machine: it restores the self-heal reconnect-and-retry branch on get_generic_data (lost in 1.83.x, issue #25143) by factoring a canonical call_with_db_reconnect_retry helper, and it moves the _engine_confirmed_dead = False reset to after a successful heavy reconnect so a mid-reconnect failure no longer silently demotes the next attempt to the lightweight path. The implementation is sound — error chaining, at-most-one-retry semantics, and the _coerce_timeout guard against MagicMock slots are all handled correctly, and the module-level traceback import in utils.py covers the removed inline import cleanly.

Confidence Score: 5/5

Safe to merge — both fixes are well-scoped, regression-tested, and introduce no breaking changes.

No P0 or P1 findings. Both changes are narrow, correct, and backed by comprehensive mock-only tests covering edge cases (reconnect-raises, double-transport-error, flag persistence across failures). The helper's exception chaining and bare-raise semantics are correct for Python 3's exception context rules.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/db/exception_handler.py	Adds call_with_db_reconnect_retry helper: well-structured, at-most-one-retry semantics, preserves original transport error when attempt_db_reconnect raises (chained as __cause__), _coerce_timeout correctly guards against MagicMock slots and booleans.
litellm/proxy/utils.py	Two clean fixes: get_generic_data wrapped in call_with_db_reconnect_retry (inline import traceback safely removed — module-level import on line 11 covers it), and _engine_confirmed_dead = False moved to after asyncio.wait_for succeeds so a heavy-reconnect failure no longer silently demotes the next attempt to the lightweight path.
tests/test_litellm/proxy/db/test_exception_handler_reconnect_retry.py	9 new unit tests for call_with_db_reconnect_retry; all cases mocked, no real network calls, covers happy path, transport retry, data-layer pass-through, reconnect-false, double-transport-error, no-attr guard, fresh-coroutine-on-retry, explicit timeouts, and reconnect-raises-preserves-original.
tests/test_litellm/proxy/db/test_prisma_self_heal.py	3 new integration-level tests: get_generic_data reconnect-and-retry for config table, propagation when reconnect fails, and regression test for _engine_confirmed_dead flag persisting across a failed heavy reconnect. All properly mocked.

_{Reviews (2): Last reviewed commit: "fix(proxy): preserve original transport ..." | Re-trigger Greptile}

[greptile-apps]

Exception from attempt_db_reconnect silently swallows original transport error

If attempt_db_reconnect itself raises (e.g. a timeout or lock acquisition error), the original first_exc transport error is lost and callers see the reconnect exception instead. The docstring says a "reconnect failure" corresponds to returning False, but an exception from the reconnect method bypasses the if not did_reconnect: raise branch entirely. This means the original transport error (e.g. httpx.ReadError) won't be what observers see in logs or the failure_handler.

Consider wrapping the reconnect call to preserve the original error:

try:
    did_reconnect = await prisma_client.attempt_db_reconnect(
        reason=reason,
        timeout_seconds=resolved_timeout,
        lock_timeout_seconds=resolved_lock_timeout,
    )
except Exception:
    raise first_exc
if not did_reconnect:
    raise first_exc

[yuneng-berri]

@greptile