build: add Dockerfile + render blueprint for rust ai-gateway

[ishaan-berri]

Relevant issues

Adds the deploy artifacts for the Rust ai-gateway (realtime WebSocket proxy) so it can be built as a container and run on Render. Stacks on litellm_rust_realtime_ai_gateway since the Rust code only lives there.

Pre-Submission checklist

• My PR's scope is as isolated as possible; it only solves 1 specific problem

Type

🚄 Infrastructure

Changes

• litellm-rust/crates/ai-gateway/Dockerfile — multi-stage build. Rust builder compiles only litellm-ai-gateway with --release --locked and default features (no python-config, so libpython is never linked). Runtime is debian:bookworm-slim with CA certs, HOST=0.0.0.0, and the binary as entrypoint. Build context is the litellm-rust/ workspace root so the workspace Cargo.lock is used.
• litellm-rust/crates/ai-gateway/render.yaml — Render blueprint for a single-instance Docker web service. Health check on /health/readiness, non-secret env vars (OPENAI_REALTIME_MODEL, HOST) inline, secrets (LITELLM_MASTER_KEY, OPENAI_API_KEY) marked sync: false.

No secrets in either file. Image builds clean locally from the litellm-rust/ context; the gateway serves wss://.../v1/realtime over Render's TLS endpoint.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[ishaan-berri]

@greptileai

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds deploy artifacts for the Rust AI Gateway realtime WebSocket proxy: a multi-stage Dockerfile (cargo-chef dependency caching, python:3.11-slim-bookworm runtime, non-root user, litellm installed from source), a BuildKit-compatible Dockerfile-specific dockerignore, and a Render blueprint.

• Dockerfile uses cargo-chef correctly (matching --features python-config in both the cook and build steps), drops to appuser uid 10001 before the entrypoint, and installs CA certs and litellm proxy extras so config loading works at boot.
• render.yaml sets correct repo-root dockerContext: . and dockerfilePath, marks secrets sync: false, and wires the health check to /health/readiness.
• config.yaml uses openai/gpt-realtime (the GA model alias confirmed valid in prior review discussion) with os.environ/OPENAI_API_KEY — no inlined secrets.

Confidence Score: 5/5

Infrastructure-only change adding Docker and Render deploy artifacts; no production code paths are modified.

All files are new additions with no existing behavior altered. The Dockerfile multi-stage build is correctly structured with cargo-chef, non-root runtime user, and matching feature flags across cook and build steps. Render blueprint paths and secret handling are correct. No credentials are baked into any file.

No files require special attention; the render.yaml and README are worth a quick glance to confirm the auto-deploy trigger setting matches your intended deployment policy.

Important Files Changed

Filename	Overview
litellm-rust/crates/ai-gateway/Dockerfile	Well-structured multi-stage build: chef→planner→builder (cargo-chef with --locked and matching --features python-config in both cook and build) + python:3.11-slim-bookworm runtime with CA certs, non-root appuser, and litellm installed from source.
litellm-rust/crates/ai-gateway/Dockerfile.dockerignore	Correctly uses BuildKit's Dockerfile-specific dockerignore convention; allowlist strategy re-includes only litellm/, litellm-rust/, and packaging metadata while pruning target/, pycache, node_modules, and tests.
litellm-rust/crates/ai-gateway/render.yaml	Render blueprint with correct repo-root dockerContext and dockerfilePath; secrets marked sync: false. autoDeployTrigger: off is absent — deploys will fire automatically on every commit despite documentation stating manual-only.
litellm-rust/crates/ai-gateway/config.yaml	Sample config using gpt-realtime (confirmed valid GA model alias) and os.environ/OPENAI_API_KEY secret reference; no inlined secrets.
litellm-rust/crates/ai-gateway/README.md	Comprehensive deployment guide; the deploy section still references the deprecated autoDeploy field name rather than the current autoDeployTrigger.

_{Reviews (11): Last reviewed commit: "ci: re-trigger flaky otel fake-openai-en..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR adds the container and cloud-deployment artifacts for the Rust realtime WebSocket AI gateway: a multi-stage Dockerfile and a Render blueprint. The Dockerfile itself is well-structured, but the render.yaml has path values that Render resolves relative to the repository root, which will cause the deployment to fail before it starts.

• render.yaml paths are broken: dockerfilePath: ./crates/ai-gateway/Dockerfile and dockerContext: . are both relative to the repo root per Render's spec — they need to be ./litellm-rust/crates/ai-gateway/Dockerfile and ./litellm-rust respectively, or Render will not find the Dockerfile and cargo build --locked will fail against the wrong workspace root.
• Dockerfile hardening: The runtime stage has no USER directive, so the gateway process runs as root inside the container.
• Build caching: COPY . . before any fetch step busts the dependency layer on every source change; separating manifest and source copies would make rebuilds significantly faster.

Confidence Score: 3/5

The Dockerfile builds correctly in isolation, but the Render blueprint has wrong path values that will prevent any deployment from completing — the service will never start on Render as written.

The dockerfilePath and dockerContext in render.yaml point to locations that do not exist when resolved from the repository root, which is how Render evaluates these fields. This means every attempt to deploy via the blueprint will fail at the Docker build step. The Dockerfile itself is sound, so the fix is small and isolated to two lines in render.yaml.

litellm-rust/crates/ai-gateway/render.yaml — the path fields must be corrected before the blueprint can deploy successfully.

Important Files Changed

Filename	Overview
litellm-rust/crates/ai-gateway/render.yaml	Render blueprint for the Rust AI gateway; dockerfilePath and dockerContext are both wrong relative to the repository root, which will cause the Render build to fail to locate the Dockerfile and pass the wrong Docker build context.
litellm-rust/crates/ai-gateway/Dockerfile	Multi-stage Rust build with a slim Debian runtime; structurally correct, but lacks a non-root USER directive and has no dependency-caching layer separation, which will slow rebuilds.

_{Reviews (2): Last reviewed commit: "build: add render blueprint for rust ai-..." | Re-trigger Greptile}

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review

[ishaan-berri]

@greptile review