fix(helm): render pod-level securityContext on the migration Job

[yassin-berriai]

TLDR

Problem this solves:

• Migration Job pod omits pod-level securityContext
• Hardened clusters deny it at admission, blocking install and upgrade
• Main Deployment is admitted, so only the Job fails

How it solves it:

• Job pod now renders .Values.podSecurityContext, like the Deployment
• Default installs render the same empty context as before

Relevant issues

Linear ticket

Resolves LIT-4928

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The proof is a real Kubernetes cluster, not helm template, because the failure is an admission decision. A kind cluster (k8s v1.36.1) enforces a ValidatingAdmissionPolicy that mirrors the reported Gatekeeper K8sPSPAllowedUsers constraint (fsGroup rule MustRunAs, ranges 1-65535), so no Gatekeeper install is needed and the denial message is reproduced verbatim.

Before: 704b9da8ab (origin/litellm_internal_staging). After: daa3eaf443

Each leg runs in its own namespace and is bracketed by a gate that creates an unhardened pod and requires it to be denied, both immediately before the install and immediately after; a leg whose gate does not deny is discarded rather than reported. This caught two would-be false passes: one where the policy had not finished propagating to a freshly created namespace, and one where another cluster on the same machine had taken over the shared kubeconfig's current-context, so every run is pinned to an explicit KUBECONFIG written by kind get kubeconfig --name lit4928.

Cluster and policy setup

kind create cluster --name lit4928

cat <<'EOF' | kubectl apply -f -
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: psp-pods-allowed-user-ranges
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
  validations:
    - expression: >-
        has(object.spec.securityContext) &&
        has(object.spec.securityContext.fsGroup) &&
        object.spec.securityContext.fsGroup >= 1 &&
        object.spec.securityContext.fsGroup <= 65535
      message: >-
        is attempting to run without a required securityContext/fsGroup.
        Allowed fsGroup: {"ranges":[{"max":65535,"min":1}],"rule":"MustRunAs"}
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: psp-pods-allowed-user-ranges
spec:
  policyName: psp-pods-allowed-user-ranges
  validationActions: ["Deny"]
  matchResources:
    namespaceSelector:
      matchLabels:
        psp-fsgroup: enforce
EOF

Values used for both legs, matching the reported configuration (a plain Postgres pod runs in each namespace as the migration target):

podSecurityContext:
  fsGroup: 10000
  runAsUser: 10000
  runAsNonRoot: true
securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
migrationJob:
  enabled: true
  hooks:
    helm: {enabled: true}
    argocd: {enabled: false}
replicaCount: 0
masterkey: sk-lit4928
db:
  deployStandalone: false
  useExisting: false
  url: postgresql://litellm:litellm@pg:5432/litellm

Before the fix, at 704b9da8ab; install is blocked

$ helm install litellm ./helm/litellm-helm -n lit4928-b1 -f values-hardened.yaml --timeout 120s
  [gate:pre] policy ENFORCING in lit4928-b1
Error: INSTALLATION FAILED: failed pre-install: resource Job/lit4928-b1/litellm-migrations not ready. status: InProgress, message: Job in progress
  migration Job pods : 0
  pod fsGroup        :
  admission denials  : 7
  release status     : failed
  denial: Error creating: pods "litellm-migrations-l6x5d" is forbidden: ValidatingAdmissionPolicy 'psp-pods-allowed-user-ranges' with binding 'psp-pods-allowed-user-ranges' denied request: is attempting to run without a required securityContext/fsGroup. Allowed fsGroup: {"ranges":[{"max":65535,"min":1}],"rule":"MustRunAs"}
  [gate:post] policy ENFORCING in lit4928-b1

Not one migration pod is ever created, and the denial text matches the report.

After the fix, at daa3eaf443; install completes

$ helm install litellm ./helm/litellm-helm -n lit4928-b2 -f values-hardened.yaml --timeout 120s
  [gate:pre] policy ENFORCING in lit4928-b2
NAME: litellm
STATUS: deployed
  migration Job pods : 1
  pod fsGroup        : 10000
  admission denials  : 0
  release status     : deployed
  [gate:post] policy ENFORCING in lit4928-b2

The Job pod is admitted carrying the configured fsGroup, the pre-install hook completes, and the release reaches deployed under the same policy that rejected it before.

Rendered output matches the Deployment

$ helm template lit ./helm/litellm-helm -f values-hardened.yaml -s templates/migrations-job.yaml   # before
    spec:
      serviceAccountName: default
      containers:

$ helm template lit ./helm/litellm-helm -f values-hardened.yaml -s templates/migrations-job.yaml   # after
    spec:
      serviceAccountName: default
      securityContext:
        fsGroup: 10000
        runAsNonRoot: true
        runAsUser: 10000
      containers:

$ helm template lit ./helm/litellm-helm -f values-hardened.yaml -s templates/deployment.yaml       # unchanged control
      serviceAccountName: default
      securityContext:
        fsGroup: 10000
        runAsNonRoot: true
        runAsUser: 10000
      containers:

With podSecurityContext left at its default the Job renders securityContext: {}, which is exactly what the Deployment already emitted, so existing installs see no change.

Chart gates

$ helm lint ./helm/litellm-helm
1 chart(s) linted, 0 chart(s) failed

$ helm unittest -f 'tests/*.yaml' helm/litellm-helm
Test Suites: 11 passed, 11 total
Tests:       90 passed, 90 total

Mutation check on the two new tests

Against the chart at 704b9da8ab with only the new tests copied in (fix absent), and against a copy whose pod-level context reads .Values.securityContext instead, each mutation asserted as applied before running:

MUTANT 1: fix absent            -> Tests: 2 failed, 14 passed, 16 total
MUTANT 2: wrong values key      -> Tests: 2 failed, 14 passed, 16 total
   asserts[0] `equal` fail: -fsGroup: 10000 / +allowPrivilegeEscalation: false

Both new tests fail on both mutants and pass on the fix.

Type

🐛 Bug Fix

Changes

helm/litellm-helm/templates/migrations-job.yaml renders spec.template.spec.securityContext from .Values.podSecurityContext, the same expression and position the proxy Deployment already uses. Two regression tests land in the existing helm/litellm-helm/tests/migrations-job_tests.yaml suite, which the Helm unit test workflow already runs: one pins the pod-level context to the configured value, the other sets pod-level and container-level contexts to different values and pins both, so a fix applied at the wrong level or from the wrong values key fails.

Two things deliberately left out of scope. The helm/litellm chart has no securityContext support in any template, so it has no inconsistency to correct here; adding hardening to it is a feature rather than this regression. Separately, running the litellm-database image under runAsUser: 10000 leaves HOME=/ unwritable, so Prisma logs a PermissionError on /.cache and exits 0 without applying the schema. That reproduces from a bare pod with no chart involved and applies equally to the proxy Deployment, so it is an image-level limitation rather than anything this template change introduces or can fix.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[yassin-berriai]

@greptileai

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR applies the chart’s pod-level security context to the migration Job so hardened Kubernetes clusters can admit migration pods.

• Renders .Values.podSecurityContext on the migration Job pod using the same pattern as the Deployment.
• Adds regression coverage for configured pod-level context and separation from container-level context.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
helm/litellm-helm/templates/migrations-job.yaml	Correctly renders the configured pod security context on the migration Job using the established Deployment pattern.
helm/litellm-helm/tests/migrations-job_tests.yaml	Adds focused assertions for pod-level security context rendering and separation from container-level settings.

_{Reviews (2): Last reviewed commit: "fix(helm): render pod-level securityCont..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai please review the current head daa3eaf443

Rebased onto current litellm_internal_staging to clear a lint failure that was base drift, an F811 duplicate Sequence import in litellm/proxy/management_endpoints/team_endpoints.py, a file this PR does not touch (PR #35483 failed identically). The chart diff is unchanged at 38 insertions across the two files; the proof section was re-captured against the new base and head so every commit hash and pod name in it is live.