feat(ai): deploy omniroute AI gateway and expose in litellm - #4307
Merged
Conversation
Self-hosted OmniRoute gateway (docker.io/diegosouzapw/omniroute) for free-tier multi-provider LLM routing, wired into litellm as the "omniroute" model via its zero-config "auto" routing.
/simplify pass: use the shared kopiur backup/restore component for the data PVC (matches every other stateful ai/ app) instead of an unbacked inline PVC, and cut memory/storage requests to what a stateless routing proxy actually needs.
📝 WalkthroughWalkthroughAdds Omniroute deployment manifests, encrypted credentials, Flux reconciliation, persistent storage, health settings, and a LiteLLM route to the internal Omniroute service. ChangesOmniroute deployment
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Flux
participant AppKustomization
participant HelmRelease
participant OmnirouteService
participant LiteLLM
Flux->>AppKustomization: reconcile Omniroute resources
AppKustomization->>HelmRelease: apply HelmRelease and secret
HelmRelease->>OmnirouteService: create Omniroute service
LiteLLM->>OmnirouteService: route openai/auto requests
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
 |
|
Overall Grade |
Security Reliability Complexity Hygiene |
Code Review Summary
| Analyzer | Status | Updated (UTC) | Details |
|---|---|---|---|
| JavaScript |  |
Aug 2, 2026 4:10p.m. | Review ↗ |
| Shell | |
Aug 2, 2026 4:10p.m. | Review ↗ |
Important
AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.
@@ (root level) @@
# v1/ServiceAccount/ai/omniroute
! + one document added:
+ apiVersion: v1
+ kind: ServiceAccount
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ app.kubernetes.io/instance: omniroute
+ app.kubernetes.io/managed-by: Helm
+ app.kubernetes.io/name: omniroute
+ helm.toolkit.fluxcd.io/name: omniroute
+ helm.toolkit.fluxcd.io/namespace: ai
@@ (root level) @@
# v1/Service/ai/omniroute
! + one document added:
+ apiVersion: v1
+ kind: Service
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ app.kubernetes.io/instance: omniroute
+ app.kubernetes.io/managed-by: Helm
+ app.kubernetes.io/name: omniroute
+ app.kubernetes.io/service: omniroute
+ helm.toolkit.fluxcd.io/name: omniroute
+ helm.toolkit.fluxcd.io/namespace: ai
+ spec:
+ type: ClusterIP
+ selector:
+ app.kubernetes.io/controller: omniroute
+ app.kubernetes.io/instance: omniroute
+ app.kubernetes.io/name: omniroute
+ ports:
+ - name: http
+ port: 20128
+ protocol: TCP
+ targetPort: 20128
@@ (root level) @@
# apps/v1/Deployment/ai/omniroute
! + one document added:
+ apiVersion: apps/v1
+ kind: Deployment
+ metadata:
+ name: omniroute
+ namespace: ai
+ annotations:
+ reloader.stakater.com/auto: "true"
+ labels:
+ app.kubernetes.io/controller: omniroute
+ app.kubernetes.io/instance: omniroute
+ app.kubernetes.io/managed-by: Helm
+ app.kubernetes.io/name: omniroute
+ helm.toolkit.fluxcd.io/name: omniroute
+ helm.toolkit.fluxcd.io/namespace: ai
+ spec:
+ replicas: 1
+ revisionHistoryLimit: 3
+ selector:
+ matchLabels:
+ app.kubernetes.io/controller: omniroute
+ app.kubernetes.io/instance: omniroute
+ app.kubernetes.io/name: omniroute
+ strategy:
+ type: Recreate
+ template:
+ metadata:
+ labels:
+ app.kubernetes.io/controller: omniroute
+ app.kubernetes.io/instance: omniroute
+ app.kubernetes.io/name: omniroute
+ spec:
+ automountServiceAccountToken: false
+ containers:
+ - name: app
+ image: "docker.io/diegosouzapw/omniroute:3.8.49"
+ env:
+ - name: DATA_DIR
+ value: /app/data
+ - name: PORT
+ value: "20128"
+ - name: REQUIRE_API_KEY
+ value: "false"
+ - name: TZ
+ value: Europe/Brussels
+ livenessProbe:
+ failureThreshold: 3
+ initialDelaySeconds: 0
+ periodSeconds: 10
+ tcpSocket:
+ port: 20128
+ timeoutSeconds: 1
+ readinessProbe:
+ failureThreshold: 3
+ initialDelaySeconds: 0
+ periodSeconds: 10
+ tcpSocket:
+ port: 20128
+ timeoutSeconds: 1
+ resources:
+ limits:
+ cpu: 1000m
+ memory: 512Mi
+ requests:
+ cpu: 100m
+ memory: 128Mi
+ volumeMounts:
+ - name: data
+ mountPath: /app/data
+ envFrom:
+ - secretRef:
+ name: omniroute-secret
+ dnsPolicy: ClusterFirst
+ enableServiceLinks: false
+ hostIPC: false
+ hostNetwork: false
+ hostPID: false
+ securityContext:
+ fsGroup: 1000
+ runAsGroup: 1000
+ runAsNonRoot: true
+ runAsUser: 1000
+ serviceAccountName: omniroute
+ volumes:
+ - name: data
+ persistentVolumeClaim:
+ claimName: omniroute
|
@@ (root level) @@
# litellm.home-operations.com/v1alpha1/LiteLLMModel/ai/omniroute
! + one document added:
+ apiVersion: litellm.home-operations.com/v1alpha1
+ kind: LiteLLMModel
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: litellm
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ spec:
+ info:
+ mode: chat
+ modelName: omniroute
+ params:
+ additional:
+ num_retries: 0
+ timeout: 300
+ apiBase: "http://omniroute.ai.svc.cluster.local:20128/v1"
+ apiKey: sk-omniroute-noauth
+ model: openai/auto
+ proxyRef: litellm
@@ (root level) @@
# helm.toolkit.fluxcd.io/v2/HelmRelease/ai/omniroute
! + one document added:
+ apiVersion: helm.toolkit.fluxcd.io/v2
+ kind: HelmRelease
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: omniroute
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ spec:
+ chartRef:
+ name: app-template
+ kind: OCIRepository
+ install:
+ crds: CreateReplace
+ interval: 30m
+ rollback:
+ cleanupOnFail: true
+ upgrade:
+ cleanupOnFail: true
+ crds: CreateReplace
+ remediation:
+ remediateLastFailure: true
+ retries: 2
+ strategy:
+ name: RemediateOnFailure
+ values:
+ controllers:
+ omniroute:
+ annotations:
+ reloader.stakater.com/auto: "true"
+ containers:
+ app:
+ resources:
+ limits:
+ cpu: 1000m
+ memory: 512Mi
+ requests:
+ cpu: 100m
+ memory: 128Mi
+ env:
+ DATA_DIR: /app/data
+ PORT: 20128
+ REQUIRE_API_KEY: "false"
+ TZ: Europe/Brussels
+ image:
+ repository: docker.io/diegosouzapw/omniroute
+ tag: "3.8.49"
+ probes:
+ liveness:
+ enabled: true
+ readiness:
+ enabled: true
+ envFrom:
+ - secretRef:
+ name: "{{ .Release.Name }}-secret"
+ defaultPodOptions:
+ securityContext:
+ fsGroup: 1000
+ runAsGroup: 1000
+ runAsNonRoot: true
+ runAsUser: 1000
+ persistence:
+ data:
+ existingClaim: omniroute
+ globalMounts:
+ - path: /app/data
+ service:
+ app:
+ ports:
+ http:
+ port: 20128
@@ (root level) @@
# kopiur.home-operations.com/v1alpha1/SnapshotPolicy/ai/omniroute
! + one document added:
+ apiVersion: kopiur.home-operations.com/v1alpha1
+ kind: SnapshotPolicy
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: omniroute
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ spec:
+ compression:
+ compressor: zstd
+ credentialProjection:
+ enabled: true
+ mover:
+ securityContext:
+ capabilities:
+ add: []
+ runAsGroup: 1000
+ runAsUser: 1000
+ repository:
+ name: kopia-nas
+ kind: ClusterRepository
+ retention:
+ keepDaily: 7
+ keepHourly: 24
+ sources:
+ - pvc:
+ name: omniroute
+ sourcePathOverride: /data
+ verification:
+ quick:
+ schedule:
+ cron: "0 3 * * 0"
+ jitter: 3h
+ volumeSnapshotClassName: csi-ceph-blockpool
@@ (root level) @@
# kopiur.home-operations.com/v1alpha1/SnapshotSchedule/ai/omniroute
! + one document added:
+ apiVersion: kopiur.home-operations.com/v1alpha1
+ kind: SnapshotSchedule
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: omniroute
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ spec:
+ policyRef:
+ name: omniroute
+ schedule:
+ cron: "H * * * *"
@@ (root level) @@
# v1/PersistentVolumeClaim/ai/omniroute
! + one document added:
+ apiVersion: v1
+ kind: PersistentVolumeClaim
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: omniroute
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ spec:
+ resources:
+ requests:
+ storage: 2Gi
+ storageClassName: ceph-block
+ accessModes:
+ - ReadWriteOnce
+ dataSourceRef:
+ name: omniroute-restore
+ apiGroup: kopiur.home-operations.com
+ kind: Restore
@@ (root level) @@
# kopiur.home-operations.com/v1alpha1/Restore/ai/omniroute-restore
! + one document added:
+ apiVersion: kopiur.home-operations.com/v1alpha1
+ kind: Restore
+ metadata:
+ name: omniroute-restore
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: omniroute
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ spec:
+ source:
+ fromPolicy:
+ name: omniroute
+ target:
+ populator: {}
+ credentialProjection:
+ enabled: true
+ policy:
+ onMissingSnapshot: Continue
+ mover:
+ securityContext:
+ runAsGroup: 1000
+ runAsUser: 1000
@@ (root level) @@
# kustomize.toolkit.fluxcd.io/v1/Kustomization/ai/omniroute
! + one document added:
+ apiVersion: kustomize.toolkit.fluxcd.io/v1
+ kind: Kustomization
+ metadata:
+ name: omniroute
+ namespace: ai
+ labels:
+ kustomize.toolkit.fluxcd.io/name: cluster-apps
+ kustomize.toolkit.fluxcd.io/namespace: flux-system
+ spec:
+ components:
+ - ../../../../components/kopiur
+ decryption:
+ provider: sops
+ secretRef:
+ name: sops-age
+ deletionPolicy: WaitForTermination
+ interval: 30m
+ patches:
+ - patch: |
+ apiVersion: helm.toolkit.fluxcd.io/v2
+ kind: HelmRelease
+ metadata:
+ name: _
+ spec:
+ install:
+ crds: CreateReplace
+ rollback:
+ cleanupOnFail: true
+ upgrade:
+ cleanupOnFail: true
+ crds: CreateReplace
+ strategy:
+ name: RemediateOnFailure
+ remediation:
+ remediateLastFailure: true
+ retries: 2
+ target:
+ kind: HelmRelease
+ group: helm.toolkit.fluxcd.io
+ path: ./kubernetes/apps/ai/omniroute/app
+ postBuild:
+ substitute:
+ APP: omniroute
+ KOPIUR_PGID: "1000"
+ KOPIUR_PUID: "1000"
+ PVC_CAPACITY: 2Gi
+ substituteFrom:
+ - name: cluster-settings
+ kind: ConfigMap
+ optional: false
+ - name: cluster-secrets
+ kind: Secret
+ optional: false
+ prune: true
+ retryInterval: 2m
+ sourceRef:
+ name: flux-system
+ kind: GitRepository
+ namespace: flux-system
+ targetNamespace: ai
+ timeout: 5m
+ wait: true
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
kubernetes/apps/ai/omniroute/app/helmrelease.yaml (3)
19-22: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winPin the image with a digest.
The image tag
3.8.49has no digest pin. The hermes HelmRelease in this repo pins images by tag and digest (tag: v2026.7.30@sha256:...). A mutable tag on Docker Hub can be repointed to a different image without a corresponding manifest change, weakening supply-chain guarantees.Add the
@sha256:...digest fordocker.io/diegosouzapw/omniroute:3.8.49.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kubernetes/apps/ai/omniroute/app/helmrelease.yaml` around lines 19 - 22, Update the image tag in the omniroute HelmRelease to append the verified SHA-256 digest for docker.io/diegosouzapw/omniroute:3.8.49, preserving the existing repository and tag while pinning the deployment by tag and digest.
34-38: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winConsider a custom health check instead of the default probe.
Liveness and readiness only set
enabled: true, without acustom/httpGetspec, unlike the hermes convention of an explicithttpGetcheck on a status endpoint. This likely falls back to a bare TCP check on the container port, which confirms the port is open but not that the app is actually healthy.OmniRoute documents internal health-check plumbing (health endpoints, DB connectivity checks), so an HTTP-based probe may be available.
Local Health Check: Monitors internal service availability and database connectivity.Verify whether OmniRoute exposes a stable health endpoint (for example under
/api/healthor similar) and use it forcustom: trueprobes, matching the hermes pattern.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kubernetes/apps/ai/omniroute/app/helmrelease.yaml` around lines 34 - 38, Update the liveness and readiness probe configuration in the HelmRelease to use OmniRoute’s documented stable health endpoint, confirming the endpoint path first. Configure both probes with custom HTTP GET checks matching the hermes convention, including the appropriate port and health path, instead of relying on enabled-only default probes.
14-51: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winAdd the missing pod and container security controls.
omniroutedoes not disable service-account-token mounting, setseccompProfile.typetoRuntimeDefault, or setallowPrivilegeEscalation: false. Add these controls to match the repository baseline. Do not enablereadOnlyRootFilesystembased only on the Docker guidance for read-only volume mounts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kubernetes/apps/ai/omniroute/app/helmrelease.yaml` around lines 14 - 51, Update the omniroute pod security configuration under defaultPodOptions to disable service-account-token mounting and set seccompProfile.type to RuntimeDefault, and add allowPrivilegeEscalation: false to the app container’s securityContext. Do not enable readOnlyRootFilesystem.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kubernetes/apps/ai/omniroute/app/helmrelease.yaml`:
- Around line 27-30: Add a CiliumNetworkPolicy for the omniroute workload that
restricts ingress to only the litellm pods, while preserving the required
traffic for the unauthenticated /v1 API. Use selectors matching omniroute as the
protected endpoint and litellm as the sole allowed client, and include the
necessary namespace and port constraints.
---
Nitpick comments:
In `@kubernetes/apps/ai/omniroute/app/helmrelease.yaml`:
- Around line 19-22: Update the image tag in the omniroute HelmRelease to append
the verified SHA-256 digest for docker.io/diegosouzapw/omniroute:3.8.49,
preserving the existing repository and tag while pinning the deployment by tag
and digest.
- Around line 34-38: Update the liveness and readiness probe configuration in
the HelmRelease to use OmniRoute’s documented stable health endpoint, confirming
the endpoint path first. Configure both probes with custom HTTP GET checks
matching the hermes convention, including the appropriate port and health path,
instead of relying on enabled-only default probes.
- Around line 14-51: Update the omniroute pod security configuration under
defaultPodOptions to disable service-account-token mounting and set
seccompProfile.type to RuntimeDefault, and add allowPrivilegeEscalation: false
to the app container’s securityContext. Do not enable readOnlyRootFilesystem.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 119a0732-3fab-4a69-b117-d5906866b3bf
📒 Files selected for processing (6)
kubernetes/apps/ai/kustomization.yamlkubernetes/apps/ai/litellm/instance/models.yamlkubernetes/apps/ai/omniroute/app/helmrelease.yamlkubernetes/apps/ai/omniroute/app/kustomization.yamlkubernetes/apps/ai/omniroute/app/secret.sops.yamlkubernetes/apps/ai/omniroute/ks.yaml
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
Tanguille/LLMKube(auto-detected)Tanguille/toolhive(auto-detected)Tanguille/pr-reviewer-action(auto-detected)Tanguille/2x-R9700-RDNA4-GFX1201-sglang-inference(auto-detected)
Comment on lines
+27
to
+30
| # Internal-only ClusterIP — litellm is the sole caller, same | ||
| # unauthenticated-backend + non-empty-apiKey convention as the | ||
| # qwen-3.6 SGLang model (LiteLLMModel requires apiKey non-empty). | ||
| REQUIRE_API_KEY: "false" |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- target manifest ---'
cat -n kubernetes/apps/ai/omniroute/app/helmrelease.yaml | sed -n '1,90p'
printf '%s\n' '--- NetworkPolicy manifests and references ---'
find kubernetes -type f \( -name '*.yaml' -o -name '*.yml' \) -print0 |
xargs -0 grep -nH -E 'kind:[[:space:]]*NetworkPolicy|networkpolicies|omniroute|litellm' || true
printf '%s\n' '--- app tree ---'
find kubernetes/apps/ai -maxdepth 4 -type f -print | sortRepository: Tanguille/cluster
Length of output: 12622
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
for p in Path("kubernetes").rglob("*.yaml"):
text = p.read_text(errors="replace")
if "kind: NetworkPolicy" in text or "networking.k8s.io/v1" in text:
print(f"{p}:")
for i, line in enumerate(text.splitlines(), 1):
if "NetworkPolicy" in line or "networking.k8s.io/v1" in line or "namespaceSelector" in line or "podSelector" in line or "policyTypes" in line or "ingress:" in line:
print(f" {i}: {line}")
PY
printf '%s\n' '--- Flux/Kustomization context for AI apps ---'
find kubernetes/apps/ai -maxdepth 5 -type f -print | sortRepository: Tanguille/cluster
Length of output: 5748
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- cluster-wide network controls and policy references ---'
rg -n -i --glob '!*.sops.yaml' \
'NetworkPolicy|network-policy|networkpolicy|default.?deny|CiliumNetworkPolicy|GlobalNetworkPolicy|calico|cilium|kube.?proxy|network.?policy' \
kubernetes talos .github 2>/dev/null || true
printf '%s\n' '--- LiteLLM and OmniRoute workload/service configuration ---'
cat -n kubernetes/apps/ai/litellm/instance/proxy.yaml | sed -n '1,110p'
cat -n kubernetes/apps/ai/litellm/instance/models.yaml | sed -n '55,78p'
cat -n kubernetes/apps/ai/omniroute/app/kustomization.yaml
cat -n kubernetes/apps/ai/omniroute/ks.yamlRepository: Tanguille/cluster
Length of output: 12843
🏁 Script executed:
#!/bin/bash
set -e
for f in \
kubernetes/apps/kube-system/network-policies/app/deny-apiserver-egress.yaml \
kubernetes/apps/kube-system/network-policies/app/database-ingress.yaml \
kubernetes/apps/kube-system/network-policies/app/kustomization.yaml \
kubernetes/apps/kube-system/network-policies/ks.yaml \
kubernetes/apps/kube-system/kustomization.yaml
do
if [ -f "$f" ]; then
printf '\n--- %s ---\n' "$f"
cat -n "$f"
fi
done
printf '\n--- all policy selectors and namespace selectors ---\n'
rg -n -A8 -B3 'kind: CiliumNetworkPolicy|endpointSelector:|namespaceSelector:|matchLabels:|enableDefaultDeny:' \
kubernetes/apps/kube-system/network-policies kubernetes --glob '*.yaml' 2>/dev/null || trueRepository: Tanguille/cluster
Length of output: 50374
Add a CiliumNetworkPolicy for omniroute.
No policy applies to the ai namespace. With REQUIRE_API_KEY set to "false", any pod that can reach the omniroute Service can call its unauthenticated /v1 API. Restrict ingress to the litellm pods.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@kubernetes/apps/ai/omniroute/app/helmrelease.yaml` around lines 27 - 30, Add
a CiliumNetworkPolicy for the omniroute workload that restricts ingress to only
the litellm pods, while preserving the required traffic for the unauthenticated
/v1 API. Use selectors matching omniroute as the protected endpoint and litellm
as the sole allowed client, and include the necessary namespace and port
constraints.
AI Automated ReviewAnalysis engine: qwen-3.6-fast@http://litellm.ai.svc.cluster.local/v1 (openai) Recommendation: Approve with minor notes for post-merge verification. This PR adds the Change-by-change findingskubernetes/apps/ai/kustomization.yaml
kubernetes/apps/ai/litellm/instance/models.yaml
kubernetes/apps/ai/omniroute/app/helmrelease.yaml
kubernetes/apps/ai/omniroute/app/kustomization.yaml
kubernetes/apps/ai/omniroute/app/secret.sops.yaml
kubernetes/apps/ai/omniroute/ks.yaml
Standards Compliance
Must-Check Items
Unknowns / Needs Verification
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docker.io/diegosouzapw/omniroute:3.8.49) as a newai/app-template HelmRelease — internal ClusterIP only, no ingress.ai/app).LiteLLMModel(omniroute) pointing atopenai/auto— OmniRoute's zero-config smart-routing model, works with no upstream provider keys.Notes
REQUIRE_API_KEY=falseinternally (ClusterIP-only, litellm is the sole caller) — same unauthenticated-backend convention as the existingsk-sglang-noauthqwen-3.6 entries.INITIAL_PASSWORD, random, inomniroute-secret) needed once to connect any paid/keyed providers beyond the freeautotier.Test plan
kustomize build kubernetes/apps/ai— builds cleankustomize build kubernetes/apps/ai/omniroute/app— builds cleancurl/v1/chat/completionsvia litellmomniroutemodel aliasSummary by CodeRabbit