feat(docker): add FIPS 140-3 image with build-time crypto verification

[ojensen-berri]

TLDR

Problem this solves:

• FIPS-gated prospects cannot deploy LiteLLM at all
• No image uses a CMVP-validated crypto module
• No visibility into dependencies bundling their own crypto

How it solves it:

• New docker/Dockerfile.fips on chainguard-base-fips
• openssl-fips-test runs as a hard build gate
• ELF allowlist scan fails the build on unreviewed native binaries
• Data-only scan report ships inside the image for auditors

User Flow

Before: a platform operator whose compliance team requires FIPS 140-3 validated cryptography has no LiteLLM image they are allowed to run.

After: we now have a base image on which to produce a fips-compliant functional LiteLLM deployment

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Setup: Chainguard org access to chainguard-base-fips and its apk repo, chainctl logged in, and a throwaway Postgres for the boot test:

docker network create fips-demo
docker run -d --rm --name fips-demo-pg --network fips-demo -e POSTGRES_PASSWORD=fipsdemo -e POSTGRES_DB=litellm postgres:17-alpine

Before (4b368bf)

There is no FIPS image.

After (e43e984)

Image builds with both gates enforced

1. CGR_TOKEN=$(chainctl auth token --audience apk.cgr.dev) docker build --secret id=cgr-token,env=CGR_TOKEN -f docker/Dockerfile.fips -t litellm-fips:dev .
2. Build log shows the verification gate passing:

FIPS cryptographic module provider details (fips.so):
      name:           Chainguard FIPS Provider for OpenSSL
      version:        3.4.0
      build:          3.4.0-r5
Locate applicable certificate(s) at: CMVP #5132 (with entropy #E191)
Lifecycle assurance satisfied.

3. Followed by the binary scan gate: PASS: all 124 ELF binaries accounted for (inventoried, system-linked, or waived)

Note that this is NOT PRODUCTION READY until we remove the waivers.

Scan gate blocks unreviewed native binaries

1. During development the gate caught real offenders missing from the known list, failing the build:

VIOLATION /app/.venv/lib/python3.13/site-packages/hf_xet/hf_xet.abi3.so [BoringSSL] bundled crypto with no waiver
FAIL: 1 of 124 ELF binaries unaccounted for (see above)
ERROR: failed to build: ... exit code: 1

Runtime verification and audit evidence

1. docker run -d --name fips-demo-litellm --network fips-demo -e DATABASE_URL='postgresql://postgres:fipsdemo@fips-demo-pg:5432/litellm' -e LITELLM_MASTER_KEY='sk-fips-demo-1234' -p 4001:4000 litellm-fips:dev --port 4000
2. Proxy applies all 177 migrations and serves: Uvicorn running on http://0.0.0.0:4000
3. docker exec fips-demo-litellm openssl-fips-test ends with Lifecycle assurance satisfied.
4. docker exec fips-demo-litellm cat /fips-scan-report.txt returns the per-binary inventory, e.g.:

CLEAN /app/.venv/lib/python3.13/site-packages/aiohttp/_http_parser.cpython-313-aarch64-linux-gnu.so
BUNDLED-CRYPTO /app/.venv/lib/python3.13/site-packages/cryptography/hazmat/bindings/_rust.abi3.so [bundled-OpenSSL]
BUNDLED-CRYPTO /app/.venv/lib/python3.13/site-packages/nacl/_sodium.abi3.so [libsodium]
scanned 124 ELF binaries

FIPS enforcement is live (non-approved algorithm rejected end to end)

1. create a new user
2. access the invitation link
3. attempt to set a password
4. observe "Internal Server Error" -- password setting fails
5. observe ValueError: [digital envelope routines] unsupported in the app logs stacktrace

Type

🆕 New Feature
🚄 Infrastructure

Caveats (if any)

Severe

• Application code still calls scrypt; setting user passwords fails by design
  • Password hashing replacement is the next workstream
• Plaintext password startup migration fails silently in FIPS mode, rows stay plaintext

Medium

• 25 binaries carry bundled non-FIPS crypto under explicit waivers
  • Each waiver will need to be removed / deleted before we can ship this
• Prisma engines embed rustls; which stack serves DB TLS needs verification
• Image requires a Chainguard entitlement to build; CI needs a pull token secret

Low

• ELF inventory must be regenerated on reviewed dependency changes (command in script header)
• No -fips entry in the publish matrix yet; CI/release wiring is a follow-up

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

Bugbot on #42899 flagged two things in these files: the libcrypto-* waiver glob misses psycopg_binary.libs/libcrypto.so.3, and normalize strips the hash from libgfortran-daac5196.so.5.0.0 while the inventory keeps it