[Feature] Download Prisma binaries at build time instead of at runtime for Security Restricted environments by mdiloreto · Pull Request #17695 · BerriAI/litellm

mdiloreto · 2025-12-09T04:31:30Z

Description

This PR makes the LiteLLM proxy image safe for hardened, restricted egress clusters by downloading Prisma engines and the required Node toolchain into the image at build time. The goal is to keep runtime completely offline (no calls to npm or prisma domains), while still supporting rootless + read‑only filesystems and Prisma‑based migrations via litellm_proxy_extras.

Related Issues

This PR addresses a long-standing pain point for users deploying LiteLLM in enterprise/hardened Kubernetes environments. Multiple issues have been reported where Prisma's runtime dependency on npm/Node.js downloads causes failures in egress-restricted clusters, and where the non-root image fails due to permission issues when writing to read-only filesystems.

Open Issues (this PR should fix):

Closed Issues (same root cause):

Summary of changes

Refactor docker/Dockerfile.non_root:
- Build all dependency wheels in the builder stage and run the UI build in a single RUN.
- Add optional PROXY_EXTRAS_SOURCE=local to test local litellm-proxy-extras changes.
- Warm the Prisma cache (prisma generate, p.ensure_cached(), engines + CLI) and copy it into the runtime layer.
Ship Prisma + Node inside the image:
- Add nodejs-bin==18.4.0a4 and Prisma 0.11.0 to requirements.txt.
- Copy Prisma CLI + engines + node modules into the runtime image.
- Set offline-friendly defaults (PRISMA_OFFLINE_MODE=true, PRISMA_BINARY_CACHE_DIR=/app/.cache/prisma-python/binaries, NPM_CONFIG_CACHE=/app/.cache/npm, etc.) so the proxy never hits npm / prisma at startup.
Make litellm_proxy_extras offline-aware:
- New helpers _get_prisma_env() and _get_prisma_command() respect PRISMA_OFFLINE_MODE and an optional PRISMA_CLI_PATH.
- All Prisma subprocess calls (migrate diff, migrate deploy, db execute, resolve, etc.) now use the cached CLI path when offline.
Add docker-compose.hardened.yml:
- Rootless, read‑only stack with only two tmpfs mounts (/app/cache, /app/migrations), all caps dropped, no-new-privileges, and a Squid proxy that denies outbound traffic.
- Makes it easy to validate the proxy in the same posture many enterprise platforms enforce.
Small cleanups:
- Reorganize Dockerfile sections for readability.
- Set success_callback: ["prometheus"] in proxy_server_config.yaml for out‑of‑the‑box metrics in the hardened stack.
- Pin runtime PyJWT to 2.10.1 without requiring any network access during install.

Runtime env vars for read‑only rootfs

LITELLM_NON_ROOT=true – keep runtime on the non‑root path/layout assumptions baked into the image.
LITELLM_MIGRATION_DIR=/app/migrations – point Prisma migrations at a writable tmpfs/emptyDir.

Tmpfs / emptyDir requirements

/app/cache (uid/gid 101, mode 1777, ~128MiB) – backing store for PRISMA_BINARY_CACHE_DIR, NPM_CONFIG_CACHE, and XDG_CACHE_HOME under read‑only rootfs.
/app/migrations (uid/gid 101, mode 1777, ~64MiB) – workspace for Prisma migrate diff output and baseline SQL when DBs are pre‑populated.

Why we block Prisma egress

The Prisma CLI + npm normally download Node toolchains and engine binaries on first prisma generate / prisma migrate. In many enterprise clusters,
outbound access to registry.npmjs.org and binaries.prisma.sh is prohibited at the pod level, so the proxy never becomes Ready with the current image.
Even where egress is allowed, runtime downloads make deployments non‑deterministic (unversioned engines) and add seconds of startup latency.
Pre‑baking engines at build time and running Prisma in offline mode gives hardened environments (no egress, read‑only rootfs, no‑new‑privileges) a deterministic, auditable startup path.

Follow‑ups / known gaps

Prisma still logs a warning about Wolfi and falls back to Debian engines; expected, but noisy. We may want to follow up with Prisma upstream once this is merged.

Error evidence:

subprocess.CalledProcessError: Command '['/usr/bin/npm', 'install', 'prisma@5.4.2']' returned non-zero exit status 1.
, e: Installing Prisma CLI
An error ocurred while installing the Prisma CLI; npm install log: npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm.
npm warn Unknown builtin config "python". This will stop working in the next major version of npm.
npm error code E403
npm error 403 403 Forbidden - GET https://registry.npmjs.org/prisma
npm error 403 In most cases, you or one of your dependencies are requesting
npm error 403 a package version that is forbidden by your security policy, or
npm error 403 on a server you do not have access to.
npm error A complete log of this run can be found in: /.npm/_logs/2025-12-08T03_43_32_121Z-debug-0.log

In egress restricted environments:

➜ litellm git:(feat/download-prisma-binaries-at-buildtime) ✗ docker logs c4c4e2332c2f -f 2025-12-08 03:43:26,002 - litellm_proxy_extras - INFO - Running prisma migrate deploy 2025-12-08 03:43:28,167 - litellm_proxy_extras - INFO - prisma db error: Traceback (most recent call last): File "/usr/bin/prisma", line 7, in sys.exit(main()) ~~~~^^ File "/usr/lib/python3.13/site-packages/prisma/cli/cli.py", line 39, in main sys.exit(prisma.run(args[1:])) ~~~~~~~~~~^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 36, in run entrypoint = ensure_cached().entrypoint ~~~~~~~~~~~~~^^ File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 104, in ensure_cached proc.check_returncode() ~~~~~~~~~~~~~~~~~~~~~^^ File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode raise CalledProcessError(self.returncode, self.args, self.stdout, self.stderr) subprocess.CalledProcessError: Command '['/usr/bin/npm', 'install', 'prisma@5.4.2']' returned non-zero exit status 1. , e: Installing Prisma CLI An error ocurred while installing the Prisma CLI; npm install log: npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm. npm warn Unknown builtin config "python". This will stop working in the next major version of npm. npm error code E403 npm error 403 403 Forbidden - GET https://registry.npmjs.org/prisma npm error 403 In most cases, you or one of your dependencies are requesting npm error 403 a package version that is forbidden by your security policy, or npm error 403 on a server you do not have access to. npm error A complete log of this run can be found in: /.npm/_logs/2025-12-08T03_43_27_681Z-debug-0.log

2025-12-08 03:43:28,168 - litellm_proxy_extras - INFO - Running prisma migrate deploy
2025-12-08 03:43:30,424 - litellm_proxy_extras - INFO - prisma db error: Traceback (most recent call last):
File "/usr/bin/prisma", line 7, in
sys.exit(main())
~~~~^^
File "/usr/lib/python3.13/site-packages/prisma/cli/cli.py", line 39, in main
sys.exit(prisma.run(args[1:]))
~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 36, in run
entrypoint = ensure_cached().entrypoint
~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 104, in ensure_cached
proc.check_returncode()
~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
self.stderr)
subprocess.CalledProcessError: Command '['/usr/bin/npm', 'install', 'prisma@5.4.2']' returned non-zero exit status 1.
, e: Installing Prisma CLI
An error ocurred while installing the Prisma CLI; npm install log: npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm.
npm warn Unknown builtin config "python". This will stop working in the next major version of npm.
npm error code E403
npm error 403 403 Forbidden - GET https://registry.npmjs.org/prisma
npm error 403 In most cases, you or one of your dependencies are requesting
npm error 403 a package version that is forbidden by your security policy, or
npm error 403 on a server you do not have access to.
npm error A complete log of this run can be found in: /.npm/_logs/2025-12-08T03_43_29_946Z-debug-0.log

2025-12-08 03:43:30,424 - litellm_proxy_extras - INFO - Running prisma migrate deploy
2025-12-08 03:43:32,593 - litellm_proxy_extras - INFO - prisma db error: Traceback (most recent call last):
File "/usr/bin/prisma", line 7, in
sys.exit(main())
~~~~^^
File "/usr/lib/python3.13/site-packages/prisma/cli/cli.py", line 39, in main
sys.exit(prisma.run(args[1:]))
~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 36, in run
entrypoint = ensure_cached().entrypoint
~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 104, in ensure_cached
proc.check_returncode()
~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
self.stderr)
subprocess.CalledProcessError: Command '['/usr/bin/npm', 'install', 'prisma@5.4.2']' returned non-zero exit status 1.
, e: Installing Prisma CLI
An error ocurred while installing the Prisma CLI; npm install log: npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm.
npm warn Unknown builtin config "python". This will stop working in the next major version of npm.
npm error code E403
npm error 403 403 Forbidden - GET https://registry.npmjs.org/prisma
npm error 403 In most cases, you or one of your dependencies are requesting
npm error 403 a package version that is forbidden by your security policy, or
npm error 403 on a server you do not have access to.
npm error A complete log of this run can be found in: /.npm/_logs/2025-12-08T03_43_32_121Z-debug-0.log

2025-12-08 03:43:32,593 - litellm_proxy_extras - INFO - Running prisma migrate deploy
2025-12-08 03:43:34,792 - litellm_proxy_extras - INFO - prisma db error: Traceback (most recent call last):
File "/usr/bin/prisma", line 7, in
sys.exit(main())
~~~~^^
File "/usr/lib/python3.13/site-packages/prisma/cli/cli.py", line 39, in main
sys.exit(prisma.run(args[1:]))
~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 36, in run
entrypoint = ensure_cached().entrypoint
~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/prisma/cli/prisma.py", line 104, in ensure_cached
proc.check_returncode()
~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
self.stderr)
subprocess.CalledProcessError: Command '['/usr/bin/npm', 'install', 'prisma@5.4.2']' returned non-zero exit status 1.
, e: Installing Prisma CLI
An error ocurred while installing the Prisma CLI; npm install log: npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm.
npm warn Unknown builtin config "python". This will stop working in the next major version of npm.
npm error code E403
npm error 403 403 Forbidden - GET https://registry.npmjs.org/prisma
npm error 403 In most cases, you or one of your dependencies are requesting
npm error 403 a package version that is forbidden by your security policy, or
npm error 403 on a server you do not have access to.
npm error A complete log of this run can be found in: /.npm/_logs/2025-12-08T03_43_34_281Z-debug-0.log

INFO: Started server process [1]
INFO: Waiting for application startup.

██╗ ██╗████████╗███████╗██╗ ██╗ ███╗ ███╗
██║ ██║╚══██╔══╝██╔════╝██║ ██║ ████╗ ████║
██║ ██║ ██║ █████╗ ██║ ██║ ██╔████╔██║
██║ ██║ ██║ ██╔══╝ ██║ ██║ ██║╚██╔╝██║
███████╗██║ ██║ ███████╗███████╗███████╗██║ ╚═╝ ██║
╚══════╝╚═╝ ╚═╝ ╚══════╝╚══════╝╚══════╝╚═╝ ╚═╝

Task exception was never retrieved
future: <Task finished name='Task-5' coro=<PrismaClient.check_view_exists() done, defined at /usr/lib/python3.13/site-packages/backoff/_async.py:130> exception=NotConnectedError('Not connected to the query engine')>
Traceback (most recent call last):
File "/usr/lib/python3.13/site-packages/backoff/_async.py", line 151, in retry
ret = await target(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/litellm/proxy/utils.py", line 1760, in check_view_exists
ret = await self.db.query_raw(
^^^^^^^^^^^^^^^^^^^^^^^^
...<13 lines>...
)
^
File "/usr/lib/python3.13/site-packages/prisma/client.py", line 487, in query_raw
resp = await self._execute(
^^^^^^^^^^^^^^^^^^^^
...<6 lines>...
)
^
File "/usr/lib/python3.13/site-packages/prisma/client.py", line 591, in _execute
return await self._engine.query(builder.build(), tx_id=self._tx_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/prisma/engine/query.py", line 244, in query
return await self.request(
^^^^^^^^^^^^^^^^^^^
...<4 lines>...
)
^
File "/usr/lib/python3.13/site-packages/prisma/engine/http.py", line 97, in request
raise errors.NotConnectedError('Not connected to the query engine')
prisma.engine.errors.NotConnectedError: Not connected to the query engine

Type of change

Tests results

Fix implemented: prisma binaries downloaded at build time

➜ litellm git:(feat/download-prisma-binaries-at-buildtime) ✗ d logs litellm-litellm-1 -f 2025-12-09 01:44:05,858 - litellm_proxy_extras - INFO - Running prisma migrate deploy 2025-12-09 01:44:05,858 - litellm_proxy_extras - INFO - Using cached Prisma CLI at /app/.cache/prisma-python/binaries/node_modules/.bin/prisma 2025-12-09 01:44:06,211 - litellm_proxy_extras - INFO - prisma migrate deploy stdout: Prisma schema loaded from schema.prisma Datasource "client": PostgreSQL database "litellm", schema "public" at "db:5432"

54 migrations found in prisma/migrations

No pending migrations to apply.

2025-12-09 01:44:06,211 - litellm_proxy_extras - INFO - prisma migrate deploy completed
2025-12-09 01:44:06,211 - litellm_proxy_extras - INFO - Running post-migration sanity check...
2025-12-09 01:44:06,211 - litellm_proxy_extras - INFO - Generating migration diff between DB and schema.prisma...
2025-12-09 01:44:06,211 - litellm_proxy_extras - INFO - Using cached Prisma CLI at /app/.cache/prisma-python/binaries/node_modules/.bin/prisma
prisma:warn Prisma doesn't know which engines to download for the Linux distro "wolfi". Falling back to Prisma engines built "debian".
Please report your experience by creating an issue at https://github.com/prisma/prisma/issues so we can add your distro to the list of known supported distros.
prisma:warn Prisma doesn't know which engines to download for the Linux distro "wolfi". Falling back to Prisma engines built "debian".
Please report your experience by creating an issue at https://github.com/prisma/prisma/issues so we can add your distro to the list of known supported distros.
2025-12-09 01:44:06,626 - litellm_proxy_extras - INFO - Migration diff created at /app/migrations/migrations/20251209014406_baseline_diff/migration.sql
2025-12-09 01:44:06,626 - litellm_proxy_extras - INFO - Running prisma db execute to apply the migration diff...
2025-12-09 01:44:06,626 - litellm_proxy_extras - INFO - Using cached Prisma CLI at /app/.cache/prisma-python/binaries/node_modules/.bin/prisma
2025-12-09 01:44:06,954 - litellm_proxy_extras - INFO - prisma db execute stdout: Script executed successfully.

2025-12-09 01:44:06,954 - litellm_proxy_extras - INFO - ✅ Migration diff applied successfully
2025-12-09 01:44:06,954 - litellm_proxy_extras - INFO - ✅ Post-migration sanity check completed
INFO: Started server process [1]
INFO: Waiting for application startup.

██╗ ██╗████████╗███████╗██╗ ██╗ ███╗ ███╗
██║ ██║╚══██╔══╝██╔════╝██║ ██║ ████╗ ████║
██║ ██║ ██║ █████╗ ██║ ██║ ██╔████╔██║
██║ ██║ ██║ ██╔══╝ ██║ ██║ ██║╚██╔╝██║
███████╗██║ ██║ ███████╗███████╗███████╗██║ ╚═╝ ██║
╚══════╝╚═╝ ╚═╝ ╚══════╝╚══════╝╚══════╝╚═╝ ╚═╝

INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)

#------------------------------------------------------------#

'This feature doesn't meet my needs because...'

https://github.com/BerriAI/litellm/issues/new

#------------------------------------------------------------#

Thank you for using LiteLLM! - Krrish & Ishaan

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new

Initialized Success Callbacks - ['prometheus']
LiteLLM: Proxy initialized with Config, Set models:
gpt-3.5-turbo-end-user-test
gpt-3.5-turbo-end-user-test
gpt-3.5-turbo
gpt-3.5-turbo-large
gpt-4
sagemaker-completion-model
text-embedding-ada-002
dall-e-2
openai-dall-e-3
fake-openai-endpoint
fake-openai-endpoint-2
fake-openai-endpoint-3
fake-openai-endpoint-4
fake-openai-endpoint-3
bad-model
good-model
*
realtime-v1
realtime-beta
anthropic/*
bedrock/*
groq/*
mistral-embed
gpt-instruct
fake-openai-endpoint-5
badly-configured-openai-endpoint
gemini-1.5-flash
gpt-4o
INFO: 172.19.0.4:34386 - "GET /metrics HTTP/1.1" 307 Temporary Redirect
INFO: 172.19.0.4:34386 - "GET /metrics/ HTTP/1.1" 200 OK
INFO: 172.19.0.1:42160 - "GET /metrics HTTP/1.1" 307 Temporary Redirect
INFO: 172.19.0.1:42160 - "GET /metrics/ HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /swagger/favicon.png HTTP/1.1" 200 OK
INFO: 172.19.0.4:36432 - "GET /metrics HTTP/1.1" 307 Temporary Redirect
INFO: 172.19.0.4:36432 - "GET /metrics/ HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "POST /v2/login HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /ui?login=success HTTP/1.1" 307 Temporary Redirect
INFO: 172.19.0.1:42176 - "GET /ui/?login=success HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /litellm-asset-prefix/_next/static/media/e4af272ccee01ff0-s.p.woff2 HTTP/1.1" 200 OK
INFO: 172.19.0.1:42188 - "GET /litellm-asset-prefix/_next/static/css/349654da14372cd9.css HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /litellm-asset-prefix/_next/static/css/1c3ed8e3d4570ab5.css HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /litellm/.well-known/litellm-ui-config HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /get/ui_theme_settings HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /user/available_users HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /sso/get/ui_settings HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /models?include_model_access_groups=True HTTP/1.1" 200 OK
INFO: 172.19.0.1:42214 - "GET /prompts/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /user/available_roles HTTP/1.1" 200 OK
INFO: 172.19.0.1:42188 - "GET /v1/mcp/access_groups HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /v2/guardrails/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /models?include_model_access_groups=True&return_wildcard_routes=True HTTP/1.1" 200 OK
INFO: 172.19.0.1:42214 - "GET /key/aliases HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /team/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "POST /v2/key/info HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /organization/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42214 - "GET /models?include_model_access_groups=True HTTP/1.1" 200 OK
INFO: 172.19.0.1:42188 - "GET /key/list?page=1&size=100&return_full_object=true&include_team_keys=true&include_created_by_keys=true HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /user/list?page=1&page_size=100 HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /index.txt?_rsc=19zvn HTTP/1.1" 404 Not Found
INFO: 172.19.0.1:42192 - "GET /models?include_model_access_groups=True&return_wildcard_routes=True HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /organization/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /team/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /user/list?page=1&page_size=100 HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /credentials HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /v2/model/info?include_team_models=true HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /public/providers/fields HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /models?include_model_access_groups=True&only_model_access_groups=True HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /model_group/info HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /team/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /credentials HTTP/1.1" 200 OK
INFO: 172.19.0.1:42214 - "GET /v2/guardrails/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /config/pass_through_endpoint HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /models?include_model_access_groups=True&only_model_access_groups=True HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /health/latest HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /model/metrics HTTP/1.1" 200 OK
INFO: 172.19.0.1:42188 - "GET /tag/list HTTP/1.1" 200 OK
INFO: 172.19.0.1:42214 - "GET /schedule/model_cost_map_reload/status HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /public/litellm_model_cost_map HTTP/1.1" 200 OK
INFO: 172.19.0.1:42220 - "GET /model/settings HTTP/1.1" 200 OK
INFO: 172.19.0.1:42214 - "GET /ui/assets/logos/mistral.svg HTTP/1.1" 200 OK
INFO: 172.19.0.1:42192 - "GET /health/latest HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /model/streaming_metrics HTTP/1.1" 200 OK
INFO: 172.19.0.1:42176 - "GET /model/metrics?_selected_model_group=text-embedding-ada-002&startTime=2025-12-02T01:44:33.884Z&endTime=2025-12-09T01:44:33.884Z&api_key=undefined&customer=null HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /model/metrics/exceptions HTTP/1.1" 200 OK
INFO: 172.19.0.1:42204 - "GET /model/streaming_metrics?_selected_model_group=text-embedding-ada-002&startTime=2025-12-02T01:44:33.884Z&endTime=2025-12-09T01:44:33.884Z HTTP/1.1" 200 OK

Basic checklist

This reverts commit 15ae36e.

vercel · 2025-12-09T04:31:36Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Dec 9, 2025 0:38am

ghost · 2025-12-09T15:40:12Z

cc: @rajatvig does this look okay to you?

mdiloreto · 2025-12-13T21:31:16Z

cc: @rajatvig does this look okay to you?

Hi @krrishdholakia let me know if you need anything from my side!

…e for Security Restricted environments (BerriAI#17695) * Use config file to enable prometheus metrics * Revert "Use config file to enable prometheus metrics" This reverts commit 15ae36e. * Improve hardened stack and Prisma offline flow * Document hardened compose usage * Remove undesired change in fastapi-sso * Restore dashboard lockfile * Remove unecessary tempdirs * Document hardened/offline Docker validation flow

…ix (#62) * [Feature] Download Prisma binaries at build time instead of at runtime for Security Restricted environments (BerriAI#17695) * Use config file to enable prometheus metrics * Revert "Use config file to enable prometheus metrics" This reverts commit 15ae36e. * Improve hardened stack and Prisma offline flow * Document hardened compose usage * Remove undesired change in fastapi-sso * Restore dashboard lockfile * Remove unecessary tempdirs * Document hardened/offline Docker validation flow * Trigger pipelines * Circle CI e2e test fix (BerriAI#16100) * fix: align PyJWT version to 2.9.0 to match requirements.txt * fix: update streaming chunk ID test for CARTO encoding behavior CARTO fork encodes chunk IDs for Redis session consistency. Updated test to match this behavior instead of upstream's raw IDs. * fix: skip pre-existing failing tests in CARTO fork - test_mock_create_audio_file: requires LITELLM_LICENSE for Enterprise features - test_anthropic_web_search_in_model_info: model_prices_and_context_window.json out of sync These should be addressed in separate PRs. * fix: update mock helper function signatures to match upstream API changes The encrypt_value_helper and decrypt_value_helper functions now accept additional keyword arguments (key, new_encryption_key, return_original_value). Updated the mock functions to accept these parameters to prevent test failures. * fix: revert test_main.py to origin/main version The test_url_with_format_param tests were broken by changes introduced in a previous merge conflict resolution. Reverting to the working origin/main version. --------- Co-authored-by: Mateo Di Loreto <101841200+mdiloreto@users.noreply.github.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>

…ilds PRISMA_CLI_BINARY_TARGETS="debian-openssl-3.0.x" was hardcoded in docker/Dockerfile.non_root by BerriAI#17695. On a buildx linux/arm64 leg this forces prisma to download the amd64 schema-engine into an arm64 image, so 'prisma migrate deploy' fails at startup with 'Could not find schema-engine binary'. Removing the env lets prisma auto-detect per build platform: amd64 builds still resolve to debian-openssl-3.0.x (Wolfi falls back to debian, same binary as before), and arm64 builds now correctly fetch linux-arm64-openssl-3.0.x. The offline-cache pre-warm goal of BerriAI#17695 is preserved — only which binaries fill the cache changes. Fixes BerriAI#19458

…e for Security Restricted environments (BerriAI#17695) * Use config file to enable prometheus metrics * Revert "Use config file to enable prometheus metrics" This reverts commit 15ae36e. * Improve hardened stack and Prisma offline flow * Document hardened compose usage * Remove undesired change in fastapi-sso * Restore dashboard lockfile * Remove unecessary tempdirs * Document hardened/offline Docker validation flow

…ilds PRISMA_CLI_BINARY_TARGETS="debian-openssl-3.0.x" was hardcoded in docker/Dockerfile.non_root by BerriAI#17695. On a buildx linux/arm64 leg this forces prisma to download the amd64 schema-engine into an arm64 image, so 'prisma migrate deploy' fails at startup with 'Could not find schema-engine binary'. Removing the env lets prisma auto-detect per build platform: amd64 builds still resolve to debian-openssl-3.0.x (Wolfi falls back to debian, same binary as before), and arm64 builds now correctly fetch linux-arm64-openssl-3.0.x. The offline-cache pre-warm goal of BerriAI#17695 is preserved — only which binaries fill the cache changes. Fixes BerriAI#19458

mdiloreto added 7 commits December 8, 2025 22:48

Use config file to enable prometheus metrics

15ae36e

Revert "Use config file to enable prometheus metrics"

6484f1d

This reverts commit 15ae36e.

Improve hardened stack and Prisma offline flow

b2f977f

Document hardened compose usage

e7af877

Remove undesired change in fastapi-sso

c3926df

Restore dashboard lockfile

c2e9a72

Remove unecessary tempdirs

ff2d4ae

vercel Bot deployed to Preview December 9, 2025 04:32 View deployment

mdiloreto changed the title ~~Bake Prisma binaries at build time for hardened offline deploys~~ Download Prisma binaries at build time instead of at runtime for Security Restricted environments Dec 9, 2025

Document hardened/offline Docker validation flow

f1bfb26

vercel Bot deployed to Preview December 9, 2025 12:38 View deployment

mdiloreto changed the title ~~Download Prisma binaries at build time instead of at runtime for Security Restricted environments~~ [Feature] Download Prisma binaries at build time instead of at runtime for Security Restricted environments Dec 9, 2025

mdiloreto marked this pull request as ready for review December 9, 2025 12:58

ghost merged commit 107ea90 into BerriAI:main Dec 16, 2025
6 of 7 checks passed

mateo-di mentioned this pull request Jan 12, 2026

fix(aiproxy): Update helm chart for new LiteLLM image with egress fix [sc-526258] CartoDB/carto-selfhosted-helm#812

Merged

8 tasks

joshgarnett mentioned this pull request Jan 21, 2026

[Bug]: Multi-arch builds broken: Hardcoded PRISMA_CLI_BINARY_TARGETS forces AMD64 binaries on ARM64 #19458

Closed

1 task

yuneng-berri mentioned this pull request May 5, 2026

[Fix] Docker: Remove Hardcoded Prisma Binary Target For Multi-Arch Builds #27170

Merged

5 tasks

This pull request was closed.

Navigation Menu

Uh oh!

Uh oh!

[Feature] Download Prisma binaries at build time instead of at runtime for Security Restricted environments - #17695

[Feature] Download Prisma binaries at build time instead of at runtime for Security Restricted environments#17695
8 commits merged into
BerriAI:mainfrom
mdiloreto:feat/download-prisma-binaries-at-buildtime

mdiloreto commented Dec 9, 2025 •

edited

Loading

Uh oh!

vercel Bot commented Dec 9, 2025 •

edited

Loading

Uh oh!

ghost commented Dec 9, 2025

Uh oh!

mdiloreto commented Dec 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

mdiloreto commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Summary of changes

Runtime env vars for read‑only rootfs

Tmpfs / emptyDir requirements

Why we block Prisma egress

Follow‑ups / known gaps

Error evidence:

Type of change

Tests results

'This feature doesn't meet my needs because...'

https://github.com/BerriAI/litellm/issues/new

Basic checklist

Uh oh!

vercel Bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Dec 9, 2025

Uh oh!

mdiloreto commented Dec 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mdiloreto commented Dec 9, 2025 •

edited

Loading

vercel Bot commented Dec 9, 2025 •

edited

Loading