feat(deploy): make coordination redis a first-class chart and terraform surface #32662
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,22 @@ | ||
| {{- if .Values.proxyConfigMap.create }} | ||
| {{- $config := deepCopy .Values.proxy_config }} | ||
| {{- if and .Values.redis.enabled (dig "coordination" "enabled" true .Values.redis) }} | ||
| {{- $generalSettings := (get $config "general_settings") | default dict }} | ||
| {{- if not (hasKey $generalSettings "coordination_redis") }} | ||
| {{- $coordinationRedis := dict "host" "os.environ/REDIS_HOST" "port" "os.environ/REDIS_PORT" "password" "os.environ/REDIS_PASSWORD" }} | ||
| {{- if .Values.redis.sentinel.enabled }} | ||
| {{- $sentinelNode := list (include "litellm.redis.serviceName" .) (include "litellm.redis.port" . | int) }} | ||
| {{- $coordinationRedis = dict "sentinel_nodes" (list $sentinelNode) "service_name" (default "mymaster" .Values.redis.sentinel.masterSet) "password" "os.environ/REDIS_PASSWORD" }} | ||
| {{- end }} | ||
| {{- $_ := set $generalSettings "coordination_redis" $coordinationRedis }} | ||
|
Contributor
There was a problem hiding this comment. Medium: Redis coordination config is ignored
|
||
| {{- $_ := set $config "general_settings" $generalSettings }} | ||
| {{- end }} | ||
| {{- end }} | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{ include "litellm.fullname" . }}-config | ||
| data: | ||
| config.yaml: | | ||
| {{ $config | toYaml | indent 6 }} | ||
| {{- end }} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| suite: test coordination redis | ||
| templates: | ||
| - configmap-litellm.yaml | ||
| - deployment.yaml | ||
| tests: | ||
| - it: should not render coordination_redis when redis is disabled | ||
| template: configmap-litellm.yaml | ||
| set: | ||
| redis.enabled: false | ||
| asserts: | ||
| - notMatchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: coordination_redis | ||
|
|
||
| - it: should not emit redis env vars when redis is disabled | ||
| template: deployment.yaml | ||
| set: | ||
| redis.enabled: false | ||
| asserts: | ||
| - notContains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: RELEASE-NAME-redis-master | ||
| any: true | ||
|
|
||
| - it: should render coordination_redis pointing at the bundled redis when enabled | ||
| template: configmap-litellm.yaml | ||
| set: | ||
| redis.enabled: true | ||
| asserts: | ||
| - matchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "coordination_redis:\n host: os.environ/REDIS_HOST\n password: os.environ/REDIS_PASSWORD\n port: os.environ/REDIS_PORT\n" | ||
| - matchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "master_key: os.environ/PROXY_MASTER_KEY" | ||
|
|
||
| - it: should emit redis env vars backing the coordination_redis os.environ refs | ||
| template: deployment.yaml | ||
| set: | ||
| redis.enabled: true | ||
| asserts: | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: RELEASE-NAME-redis-master | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PORT | ||
| value: "6379" | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PASSWORD | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: RELEASE-NAME-redis | ||
| key: redis-password | ||
|
|
||
| - it: should not render coordination_redis when coordination is opted out | ||
| template: configmap-litellm.yaml | ||
| set: | ||
| redis.enabled: true | ||
| redis.coordination.enabled: false | ||
| asserts: | ||
| - notMatchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: coordination_redis | ||
|
|
||
| - it: should keep emitting redis env vars when coordination is opted out | ||
| template: deployment.yaml | ||
| set: | ||
| redis.enabled: true | ||
| redis.coordination.enabled: false | ||
| asserts: | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: RELEASE-NAME-redis-master | ||
|
|
||
| - it: should not clobber a user supplied coordination_redis block | ||
| template: configmap-litellm.yaml | ||
| set: | ||
| redis.enabled: true | ||
| proxy_config.general_settings.coordination_redis: | ||
| url: os.environ/COORDINATION_REDIS_URL | ||
| asserts: | ||
| - matchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "coordination_redis:\n url: os.environ/COORDINATION_REDIS_URL\n" | ||
| - notMatchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "host: os.environ/REDIS_HOST" | ||
|
|
||
| - it: should render sentinel_nodes and service_name in sentinel mode | ||
| template: configmap-litellm.yaml | ||
| set: | ||
| redis.enabled: true | ||
| redis.architecture: replication | ||
| redis.sentinel.enabled: true | ||
| asserts: | ||
| # The sentinel Service the redis subchart renders is "<release>-redis", and a | ||
| # plain client cannot speak the sentinel protocol, so host/port must not appear | ||
| - matchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "coordination_redis:\n password: os.environ/REDIS_PASSWORD\n sentinel_nodes:\n - - RELEASE-NAME-redis\n - 26379\n service_name: mymaster\n" | ||
| - notMatchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "host: os.environ/REDIS_HOST" | ||
|
|
||
| - it: should carry a custom sentinel masterSet into service_name | ||
| template: configmap-litellm.yaml | ||
| set: | ||
| redis.enabled: true | ||
| redis.architecture: replication | ||
| redis.sentinel.enabled: true | ||
| redis.sentinel.masterSet: litellm-master | ||
| asserts: | ||
| - matchRegex: | ||
| path: data["config.yaml"] | ||
| pattern: "service_name: litellm-master" | ||
|
|
||
| - it: should point REDIS_HOST at the sentinel service in sentinel mode | ||
| template: deployment.yaml | ||
| set: | ||
| redis.enabled: true | ||
| redis.architecture: replication | ||
| redis.sentinel.enabled: true | ||
| asserts: | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: RELEASE-NAME-redis | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PORT | ||
| value: "26379" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| suite: test redis coordination env vars | ||
| templates: | ||
| - gateway/deployment.yaml | ||
| - gateway/configmap.yaml | ||
| - backend/deployment.yaml | ||
| values: | ||
| - ./values/required.yaml | ||
| tests: | ||
| - it: gateway omits redis env vars when no host is configured | ||
| template: gateway/deployment.yaml | ||
| asserts: | ||
| - notContains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: redis.example.com | ||
| any: true | ||
| - notContains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_CLUSTER_NODES | ||
| any: true | ||
|
|
||
| - it: gateway emits host, port and password when redis is configured | ||
| template: gateway/deployment.yaml | ||
| set: | ||
| redis.host: redis.example.com | ||
| redis.port: 6380 | ||
| redis.passwordSecret.name: redis-secret | ||
| asserts: | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: redis.example.com | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PORT | ||
| value: "6380" | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PASSWORD | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: redis-secret | ||
| key: password | ||
|
|
||
| - it: backend emits the same redis env vars so both pods coordinate on one redis | ||
| template: backend/deployment.yaml | ||
| set: | ||
| redis.host: redis.example.com | ||
| redis.passwordSecret.name: redis-secret | ||
| redis.passwordSecret.passwordKey: redis-password | ||
| asserts: | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: redis.example.com | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PASSWORD | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: redis-secret | ||
| key: redis-password | ||
|
|
||
| - it: gateway omits REDIS_PASSWORD for an auth-less redis | ||
| template: gateway/deployment.yaml | ||
| set: | ||
| redis.host: redis.example.com | ||
| asserts: | ||
| - notContains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_PASSWORD | ||
| any: true | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_HOST | ||
| value: redis.example.com | ||
|
|
||
| - it: gateway seeds REDIS_CLUSTER_NODES from host and port in cluster mode | ||
| template: gateway/deployment.yaml | ||
| set: | ||
| redis.host: redis.example.com | ||
| redis.port: 6380 | ||
| redis.cluster: true | ||
| asserts: | ||
| - contains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_CLUSTER_NODES | ||
| value: '[{"host":"redis.example.com","port":6380}]' | ||
|
|
||
| - it: gateway omits REDIS_CLUSTER_NODES when cluster mode is off | ||
| template: gateway/deployment.yaml | ||
| set: | ||
| redis.host: redis.example.com | ||
| asserts: | ||
| - notContains: | ||
| path: spec.template.spec.containers[0].env | ||
| content: | ||
| name: REDIS_CLUSTER_NODES | ||
| any: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both the standalone and sentinel branches unconditionally add
password: os.environ/REDIS_PASSWORDto thecoordination_redisblock. If a user deploys the bundled Redis withredis.auth.enabled: false(disabling Bitnami's auth), the Bitnami subchart will not create the<release>-redissecret and theREDIS_PASSWORDenv var will not be set in the pod. At startup, litellm will attempt to resolveos.environ/REDIS_PASSWORDand fail with a key error, preventing the proxy from starting.Consider guarding the password field on
redis.auth.enabled(which defaults totruein the Bitnami subchart):{{- if ne (dig "auth" "enabled" true .Values.redis) false }}. The same guard would apply in the sentinel branch.