Skip to content

[release-1.3] changed httproute creation to be behind a flag. #2129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ahg-g merged 1 commit into from
Jan 12, 2026
+16 −5
Merged

[release-1.3] changed httproute creation to be behind a flag. #2129

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion config/charts/inferencepool/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,11 @@ The following table list the configurable parameters of the chart.
| `inferenceExtension.tracing.otelExporterEndpoint` | OpenTelemetry collector endpoint. |
| `inferenceExtension.tracing.sampling.sampler` | The trace sampler to use. Currently, only `parentbased_traceidratio` is supported. This sampler respects the parent span’s sampling decision when present, and applies the configured ratio for root spans. |
| `inferenceExtension.tracing.sampling.samplerArg` | Sampler-specific argument. For `parentbased_traceidratio`, this defines the base sampling rate for new traces (root spans), as a float string in the range [0.0, 1.0]. For example, "0.1" enables 10% sampling. |
| `inferenceExtension.baseModel` | Base model used in the current instance of the epp. When this value is set the HttpRoute will be set to match the pool based on `X-Gateway-Base-Model-Name` header. Optional. |
| `inferenceExtension.volumes` | List of volumes to mount in the EPP deployment as free-form YAML. Optional. |
| `inferenceExtension.volumeMounts` | List of volume mounts for the EPP container as free-form YAML. Optional. |
| `experimentalGateway.enabled` | Boolean flag to indicate whether the helm chart should create HttpRoute. Defaults to `False`. Optional. |
| `experimentalGateway.name` | Name of the inference-gateway to be used when creating the HttpRoute. Used only if `experimentalGateway` is enabled. Optional. |
| `experimentalGateway.baseModel` | Base model used in the current instance of the epp. When this value is set the HttpRoute will be set to match the pool based on `X-Gateway-Base-Model-Name` header. Optional. |
| `inferenceExtension.sidecar.enabled` | Enables or disables the sidecar container in the EPP deployment. Defaults to `false`. |
| `inferenceExtension.sidecar.name` | Name of the sidecar container. Required when the sidecar is enabled. |
| `inferenceExtension.sidecar.image` | Image for the sidecar container. Required when the sidecar is enabled. |
Expand Down
8 changes: 5 additions & 3 deletions config/charts/inferencepool/templates/httproute.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{- if .Values.experimentalGateway.enabled }}
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
Expand All @@ -7,7 +8,7 @@ spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: {{ .Values.inferenceGateway.name }}
name: {{ .Values.experimentalGateway.name }}
rules:
- backendRefs:
- group: inference.networking.k8s.io
Expand All @@ -17,11 +18,12 @@ spec:
- path:
type: PathPrefix
value: /
{{- if .Values.inferenceExtension.baseModel }}
{{- if .Values.experimentalGateway.baseModel }}
headers:
- type: Exact
name: X-Gateway-Base-Model-Name
value: {{ .Values.inferenceExtension.baseModel }}
value: {{ .Values.experimentalGateway.baseModel }}
{{- end }}
timeouts:
request: 300s
{{- end }}
5 changes: 4 additions & 1 deletion config/charts/inferencepool/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,10 @@ provider:
# http:
# maxRequestsPerConnection: 256000

inferenceGateway:
# experimentalGateway section is used to deploy httproute as part of the epp helm chart.
# this section should be extracted to a separate chart.
experimentalGateway:
enabled: false # a flag to indicate whether to create the httproute as part of the chart or not.
name: inference-gateway

# DEPRECATED and will be removed in v1.3. Instead, use `provider.istio.*`.
Expand Down
4 changes: 4 additions & 0 deletions site-src/_includes/epp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
helm install vllm-llama3-8b-instruct \
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
--set provider.name=$GATEWAY_PROVIDER \
--set experimentalGateway.enabled=true \
--version $IGW_CHART_VERSION \
oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool
```
Expand All @@ -16,6 +17,7 @@
helm install vllm-llama3-8b-instruct \
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
--set provider.name=$GATEWAY_PROVIDER \
--set experimentalGateway.enabled=true \
--version $IGW_CHART_VERSION \
oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool
```
Expand All @@ -27,6 +29,7 @@
helm install vllm-llama3-8b-instruct \
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
--set provider.name=$GATEWAY_PROVIDER \
--set experimentalGateway.enabled=true \
--version $IGW_CHART_VERSION \
oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool
```
Expand All @@ -38,6 +41,7 @@
helm install vllm-llama3-8b-instruct \
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
--set provider.name=$GATEWAY_PROVIDER \
--set experimentalGateway.enabled=true \
--version $IGW_CHART_VERSION \
oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool
```
Loading