fix(migrations): keep the toolchain heal from raising on an unreadable nodeenv cache

[yassin-berriai]

TLDR

Problem this solves:

• Migrations crash under any non-root uid before Prisma runs
• A bare Path.is_dir() lets EACCES escape instead of answering False
• image-scan is red on every PR that triggers it

How it solves it:

• Tolerate OSError while inspecting the nodeenv cache
• An unreachable cache means nothing to heal, not a crash
• Restores the never-raises contract the module already documents
• Lets changes to that package trigger the image job at all

Relevant issues

Linear ticket

Resolves LIT-5243

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Before at 54f83b2614, after at d98ef29b5b

1. Before. CI's own migrations-image job, which builds migrations/Dockerfile and runs the migration as an arbitrary uid on a network with no egress, fails before Prisma is invoked (job 92417519889):

2026-08-05 19:22:32,883 - litellm_proxy_extras - INFO - Using custom Prisma CLI at /opt/prisma/binaries/node_modules/.bin/prisma
Traceback (most recent call last):
  File "/app/run.py", line 55, in main
    ok = ProxyExtrasDBManager.setup_database(
  File "/app/.venv/.../litellm_proxy_extras/utils.py", line 738, in setup_database
    ensure_prisma_toolchain(
  File "/app/.venv/.../litellm_proxy_extras/prisma_toolchain.py", line 150, in ensure_prisma_toolchain
    healed = heal_incomplete_nodeenv_cache()
  File "/app/.venv/.../litellm_proxy_extras/prisma_toolchain.py", line 124, in heal_incomplete_nodeenv_cache
    if cache_dir is None or not cache_dir.is_dir():
  File "/usr/lib/python3.13/pathlib/_local.py", line 515, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/home/nonroot/.cache/prisma-python/nodeenv'

FAILED tests/proxy_migration_tests/test_offline_image_migration.py::test_migration_offline_as_non_root_uid

2. After. Same image built from this branch, same invocation the workflow uses:

$ docker build -f migrations/Dockerfile -t lit5243-migrations .
$ LITELLM_IMAGE=lit5243-migrations \
  LITELLM_MIGRATION_INTERPRETER=python3 \
  LITELLM_MIGRATION_SCRIPT=/app/run.py \
  python -m pytest tests/proxy_migration_tests/test_offline_image_migration.py -v

tests/proxy_migration_tests/test_offline_image_migration.py::test_migration_offline_as_non_root_uid PASSED [ 50%]
tests/proxy_migration_tests/test_offline_image_migration.py::test_runtime_cache_env_not_read_only   PASSED [100%]
2 passed in 13.53s

That test runs the migration entrypoint as uid 12345:0 against a brand-new Postgres on an --internal network, then asserts the schema was actually created, so it fails both when the entrypoint dies and when it exits 0 having applied nothing

3. The added unit test fails without the fix. The pre-fix module was restored from HEAD into a scratch copy so nothing in the working tree moved:

$ git show HEAD:litellm-proxy-extras/litellm_proxy_extras/prisma_toolchain.py > $SCRATCH/litellm_proxy_extras/prisma_toolchain.py
$ PYTHONPATH=$SCRATCH python -m pytest tests/proxy_migration_tests/test_prisma_toolchain.py -k unreadable

E   PermissionError: [Errno 13] Permission denied: '.../test_unreadable_cache_dir_is_n0/nodeenv'
FAILED tests/proxy_migration_tests/test_prisma_toolchain.py::test_unreadable_cache_dir_is_not_an_error
1 failed, 18 deselected

$ python -m pytest tests/proxy_migration_tests/test_prisma_toolchain.py
19 passed in 8.67s

Same errno and same call as the container failure

Type

🐛 Bug Fix

Changes

heal_incomplete_nodeenv_cache() stats a $HOME-derived path with a bare Path.is_dir(). pathlib only swallows the ENOENT-shaped errnos in _IGNORED_ERRNOS, so a directory the process cannot search raises PermissionError rather than answering False. The images bake that cache under the build user's home, and the base image ships that home at mode 0700, so a container started under any other uid dies there before the Prisma CLI is ever invoked and the migration never runs. ensure_prisma_toolchain() documents itself as never raising; this made that untrue

The fix tolerates OSError while inspecting the cache, so an unreachable cache means there is nothing to heal. One try covers both stats rather than two separate guards, since a cache directory whose bin/ is unreadable is the same class of failure. The guard already existed one function above: nodeenv_cache_dir() catches OSError on its other branch, so this brings the two into line. The warning fires only on the OSError path, leaving a legitimately absent cache silent while telling an operator why healing was skipped

Worth stating plainly, because it is the argument for fixing it here rather than elsewhere: this is the same defect class as an unguarded pathlib existence check in prisma-client-py itself, reproduced inside the code we wrote to make prisma's toolchain more robust. An unguarded stat is easy to write and invisible until someone runs as a different uid

Why not just fix where the cache is baked, which is the other obvious remedy: the two are independent and both are worth having. The non_root image already bakes at /opt/prisma, so its $HOME-derived nodeenv path stays reachable and test_migration_offline_as_non_root_uid passes there; the migrations image still bakes under the build user's home, so the same line raises. Moving a bake removes one unreadable path, while this change makes the stat tolerate an unreadable path wherever anything is baked. Fixing only the bake location would leave this raising the next time any path is unreachable, so the bake move is being handled separately rather than folded in here

The test drives the cache location through the existing PRISMA_NODEENV_CACHE_DIR override and the file's own fixture rather than patching anything, which matters here because the bug is precisely about which path gets stat-ed. It is skipped as root and on Windows, since neither honours a 0o000 directory, and the skip carries that reason inline so it does not get deleted later as unexplained

One line of CI wiring comes with it, because the fix is otherwise untested by the job it repairs. image-scan triggers on a paths list that names migrations/Dockerfile, migrations/run.py and one test file, but not litellm-proxy-extras/**, which is where the code the migration Job actually executes lives. The change that introduced this behaviour touched only that package, so the job never ran on it:

$ gh pr checks 35832 | grep -i image-scan
(no output; the job was never queued on that PR)

That is the whole explanation for how a two-line defect shipped green. Nothing was watching, rather than someone missing it. The single-file entry is replaced with litellm-proxy-extras/** and tests/proxy_migration_tests/**, so a change to the migration code path exercises the migration image from now on. To be precise about what this PR proves: image-scan runs here because the PR edits the workflow file, which was already a trigger, so its green is not itself evidence that the new entries work. What the new entries change is the next PR that touches only litellm-proxy-extras/**, which is the exact shape that shipped this bug

The cost is real and worth stating: image-scan now triggers on any change under either path, and it builds two images at roughly nine minutes. PRs touching that package previously skipped it and no longer will

A note on formatting, so the untouched reflow does not read as an oversight: ruff format wants to reformat both files at HEAD as well, since the package is written at 88 columns while the repo config is 120, so that drift is pre-existing. Bundling a whole-file reflow into a two-line urgent fix would bury the change; the added lines conform and ruff check is clean

Related and deliberately not fixed here: tests/proxy_migration_tests/test_prisma_toolchain.py is not referenced by any workflow, and assert-shard-coverage only enumerates tests/proxy_unit_tests/, so those unit tests have never run in CI. Picking a shard for them, or extending the coverage assertion so the next unlisted directory is caught automatically, is a broader change than this fix and belongs on its own

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes the migration entrypoint’s Prisma toolchain prep on a security-sensitive startup path, but behavior is limited to swallowing stat failures and is covered by new tests; CI will run heavier image jobs on more PRs.

Overview
Fixes migrations aborting before Prisma runs when the container uid cannot read the baked-in nodeenv cache under another user’s home.

heal_incomplete_nodeenv_cache() now wraps cache inspection in OSError handling so PermissionError from Path.is_dir() is treated as “nothing to heal” (with a warning) instead of violating ensure_prisma_toolchain()’s never-raise contract. A unit test covers an unreadable parent directory (skipped on root/Windows).

image-scan PR path filters now include litellm-proxy-extras/** and tests/proxy_migration_tests/** (replacing a single migration test file), so edits to the migration execution path trigger the migration image checks.

^{Reviewed by Cursor Bugbot for commit d98ef29. Bugbot is set up for automated code reviews on this repo. Configure here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR prevents migration startup from crashing when the configured nodeenv cache cannot be inspected.

• Wraps nodeenv cache filesystem probes in OSError handling and logs when inspection is skipped.
• Adds a regression test covering an inaccessible cache directory under non-root Unix execution.

Confidence Score: 5/5

The PR appears safe to merge, with the inaccessible-cache failure handled consistently with the toolchain’s never-raises contract.

The changed guard catches filesystem inspection failures before cache cleanup, logs the condition, and returns without mutating an inaccessible cache; the regression test directly exercises the reported non-root permission failure.

Important Files Changed

Filename	Overview
litellm-proxy-extras/litellm_proxy_extras/prisma_toolchain.py	Safely treats filesystem inspection errors as a non-healable cache state while preserving later Prisma error reporting.
tests/proxy_migration_tests/test_prisma_toolchain.py	Adds a focused permission-denied regression test with cleanup and appropriate platform/root skips.

_{Reviews (1): Last reviewed commit: "fix(migrations): keep the toolchain heal..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d98ef29. Configure here.}

[mateo-berri]

LGTM. Thanks