test(proxy/utils): pin PrismaClient and spend-update behavior

[yuneng-berri]

Relevant issues

PR2 of the proxy/utils.py behavior-pinning plan. Sibling to the proxy_server.py pinning work that landed earlier; this PR pins the second of three regions of litellm/proxy/utils.py so we can refactor it without regressions slipping in.

Linear ticket

n/a

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

• Branch creation CI run
  Link:
• CI run for the last commit
  Link:
• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

182 tests pass against a real uv-installed proxy environment, exercising every PR2 pin against a fully-mocked Prisma stack (no DB, no network).

$ uv run pytest tests/test_litellm/proxy/utils/prisma_and_spend/ -q
182 passed, 3 warnings in 133.88s

The tests run against a conftest that patches prisma.Prisma and PrismaWrapper before instantiating PrismaClient, swaps in a MagicMock table layer for ~30 db.litellm_* tables, captures SMTP traffic in memory for send_email, and drives the spend monitor loop with a controllable clock and a _fast_sleep shim.

Type

✅ Test

Changes

Adds tests/test_litellm/proxy/utils/prisma_and_spend/ with happy + error pins for every symbol in the PR2 pin list (the PR2 page):

• _ConfigRow, _config_cache_key, _pack_config_row, _unpack_config_row, get_config_param, invalidate_config_param, prefetch_config_params
• PrismaClient.__init__, writer_db, connect, disconnect
• hash_token, jsonify_object, jsonify_team_object, check_view_exists, get_request_status, get_generic_data, _query_first_with_cached_plan_fallback, get_data, insert_data, update_data, delete_data
• Engine watcher: _get_engine_pid, _is_engine_alive, _reap_all_zombies, _try_waitpid_watch, _waitpid_thread_func, _on_engine_death_from_thread, _try_pidfd_watch, _on_pidfd_readable, _poll_engine_proc, _cleanup_engine_watcher, _start_engine_watcher, _stop_engine_watcher
• Reconnect: _run_reconnect_cycle, _attempt_reconnect_inside_lock, attempt_db_reconnect, start_db_health_watchdog_task, stop_db_health_watchdog_task, _db_health_watchdog_loop
• Health: health_check, _get_spend_logs_row_count, _set_spend_logs_row_count_in_proxy_state, _validate_response_time, _clean_details, save_health_check_result, get_health_check_history, get_all_latest_health_checks, _is_sha256_hex
• _cache_user_row, send_email
• Password / token helpers: hash_token, hash_password, verify_password, migrate_passwords_to_scrypt_async, _hash_token_if_needed
• ProxyUpdateSpend.update_end_user_spend, update_spend_logs, disable_spend_updates
• Module-level spend functions: update_spend, update_daily_tag_spend, update_spend_logs_job, _monitor_spend_logs_queue, _raise_failed_update_spend_exception

The workflow shard tests/test_litellm/proxy/utils is added to the existing proxy-endpoints job in .github/workflows/test-unit-proxy-endpoints.yml.

Plan: https://www.notion.so/37343b8acdab81f68f39f66915f62bcf
Pin list: https://www.notion.so/37343b8acdab81e3a172c2af8cdbd11d

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR adds a comprehensive behavior-pinning test suite for litellm/proxy/utils.py (the "PR2" region), covering PrismaClient lifecycle, engine watcher, reconnect/watchdog, health checks, config cache, spend functions, password helpers, send_email, and ProxyUpdateSpend. No production code is changed.

• ~3,800 lines of new mock-only tests across 12 test files, organized by symbol cluster and driven by a shared conftest.py with fixtures for the Prisma stack, a controllable clock, an in-memory SMTP capture, and a DualCache stand-in.
• CI shard updated to run the new tests/test_litellm/proxy/utils directory alongside the existing proxy-endpoints job (2 workers, 2 reruns).

Confidence Score: 5/5

Pure test addition — no production code is touched; all 182 new tests are mock-only with no real DB or network traffic.

Every changed file is either a new test module or a one-line CI shard addition. The mocking strategy is thorough: Prisma, SMTP, asyncio.sleep, os.waitpid/pidfd_open, and all DB table methods are stubbed before any production code runs. No existing tests are modified and no production behaviour is changed.

No files require special attention.

Important Files Changed

Filename	Overview
tests/test_litellm/proxy/utils/prisma_and_spend/conftest.py	Shared fixture library: Prisma mock, dual-cache stub, patched prisma import, controllable clock, in-memory SMTP, spend-log row factory — all well-scoped via monkeypatch.
tests/test_litellm/proxy/utils/prisma_and_spend/test_prisma_client_engine_watcher.py	Pins all 12 engine-watcher methods; platform-specific calls (waitpid, pidfd_open) are fully mocked; pidfd tests use raising=False so they are portable to macOS.
tests/test_litellm/proxy/utils/prisma_and_spend/test_spend_functions.py	Pins update_spend, update_daily_tag_spend, update_spend_logs_job, _monitor_spend_logs_queue, and _raise_failed_update_spend_exception; error/backoff paths covered via CancelledError injection and bad-lock simulation.
tests/test_litellm/proxy/utils/prisma_and_spend/test_proxy_update_spend.py	Pins ProxyUpdateSpend.update_end_user_spend, update_spend_logs, and disable_spend_updates; retry/backoff, writer-URL routing, and flag toggling all covered.
tests/test_litellm/proxy/utils/prisma_and_spend/test_prisma_client_lifecycle.py	Pins PrismaClient.init (defaults + env overrides + missing-generated-client error), writer_db property, connect/disconnect; manual try/finally restore pattern is safe.
tests/test_litellm/proxy/utils/prisma_and_spend/test_password_helpers.py	Pins hash_token, hash_password, verify_password, migrate_passwords_to_scrypt_async, and _hash_token_if_needed with happy + error + scrypt-vs-sha256 migration paths.
tests/test_litellm/proxy/utils/prisma_and_spend/test_prisma_client_reconnect.py	Pins reconnect cycle (direct/heavy path), _attempt_reconnect_inside_lock, watchdog start/stop, and watchdog loop; uses monkeypatched asyncio.sleep for determinism.
tests/test_litellm/proxy/utils/prisma_and_spend/test_config_param_cache.py	Pins the full config-cache layer (_ConfigRow, cache key, pack/unpack, get/invalidate/prefetch); autouse fixture swaps the module-level cache for a clean store per test.
tests/test_litellm/proxy/utils/prisma_and_spend/test_send_email.py	Pins send_email; SMTP is captured in-memory with no real network calls; TLS, login, missing-env, and SMTP-failure-swallowing paths covered.
.github/workflows/test-unit-proxy-endpoints.yml	Adds the new test shard (tests/test_litellm/proxy/utils) to the existing proxy-endpoints matrix job; one-line addition, correct indentation.
tests/test_litellm/proxy/utils/prisma_and_spend/_harness_smoke_test.py	Self-tests for conftest fixtures; imports normalize directly from conftest, which is non-standard but works due to the sys.path insertion.
tests/test_litellm/proxy/utils/prisma_and_spend/test_prisma_client_health.py	Pins health_check, spend-logs row counter helpers, save/get health-check history, and _is_sha256_hex; all paths exercised against mocked query_raw.
tests/test_litellm/proxy/utils/prisma_and_spend/test_prisma_client_get_data.py	Pins read-side PrismaClient methods including get_data, get_generic_data, _query_first_with_cached_plan_fallback, jsonify_object, and check_view_exists.
tests/test_litellm/proxy/utils/prisma_and_spend/test_prisma_client_writes.py	Pins insert_data, update_data, and delete_data write paths with retry and error propagation coverage.
tests/test_litellm/proxy/utils/prisma_and_spend/test_cache_user_row.py	Pins _cache_user_row behavior including cache hit/miss paths and error handling.

_{Reviews (2): Last reviewed commit: "test(proxy/utils): drop committed pin-li..." | Re-trigger Greptile}

[greptile-apps]

Missing .pin_list.txt prevents pin check from running

The script requires a --list argument pointing to tests/test_litellm/proxy/utils/prisma_and_spend/.pin_list.txt, and the PR description's proof-of-fix commands reference this file explicitly. However, .pin_list.txt is not included in the PR's changeset. Anyone cloning the repo and following the documented steps would immediately hit:

FAIL: pin list not found at tests/test_litellm/proxy/utils/prisma_and_spend/.pin_list.txt

The file needs to be committed alongside the script that depends on it.

[greptile-apps]

Substring matching causes false-positive coverage for overlapping symbol names

_references_pin does a raw in substring check, so the pin hash_token matches any test file that references _hash_token_if_needed, and update_spend matches files referencing update_spend_logs_job. This lets the gate report a pin as covered when only a differently-named related symbol is exercised. Word-boundary matching avoids the overlap.

Suggested change

	def _references_pin(tf: TestFunction, pin: str) -> bool:
	return pin in tf.source
	def _references_pin(tf: TestFunction, pin: str) -> bool:
	import re as _re
	return bool(_re.search(r"\b" + _re.escape(pin) + r"\b", tf.source))

[greptile-apps]

_harness_smoke_test.py exclusion is dead code

The loop already only processes files matching test_*.py, so _harness_smoke_test.py (which starts with _) is never yielded by the glob in the first place. The if guard can't be reached and silently misleads readers into thinking it does something.

Suggested change

	for path in sorted(test_dir.glob("test_*.py")):
	if path.name == "_harness_smoke_test.py":
	continue
	for path in sorted(test_dir.glob("test_*.py")):

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Hardcoded line numbers will silently drift as utils.py grows

_PR2_LINE_START = 2668 and _PR2_LINE_END = 5541 are absolute line numbers in litellm/proxy/utils.py. Any code inserted before line 2668 (new helpers, imports, etc.) will shift the window without any indication, causing the gate to measure the wrong region and potentially print PASS when the target symbols are actually uncovered. Anchoring by function/class names — or storing the anchors as a comment in the source file and resolving them at check time — would be more robust.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yuneng-berri]

@greptile