diff --git a/docs/fern/pages/reference/components/gateway-api-routing.mdx b/docs/fern/pages/reference/components/gateway-api-routing.mdx index 2a18720088e0..c7bbb8a24781 100644 --- a/docs/fern/pages/reference/components/gateway-api-routing.mdx +++ b/docs/fern/pages/reference/components/gateway-api-routing.mdx @@ -9,6 +9,28 @@ Use this reference for a `DynamoGraphDeployment` (DGD) that routes requests thro Inference Extension (GAIE) and the Dynamo Endpoint Picker Plugin (EPP). For the procedure and topology, see [Using GAIE with Dynamo](../../kubernetes/kv-aware-routing/gateway-api.mdx). +## EPP Modes + +The native Rust EPP runs in one of two modes, selected by the `DYN_EPP_MODE` environment variable. +This reference covers Dynamo mode only. The deprecated Go EPP, kept by `eppConfig` with the image +pinned to 1.4, does not read `DYN_EPP_MODE` and has no standalone mode. + +| Concern | Dynamo mode (`dynamo`, default) | Standalone mode (`standalone`) | +|---|---|---| +| Deployed by | Dynamo operator, from a DGD `epp` component | User-applied manifests | +| Dynamo runtime | Required. It uses whichever discovery and event planes the platform is configured for; etcd and NATS are needed only when selected | Not used | +| Worker discovery | Dynamo runtime discovery | Kubernetes pod watch driven by `InferencePool.spec.selector` | +| EPP ServiceAccount and RBAC | Cluster-wide mode: the operator creates the ServiceAccount and binds it to the `dynamo-platform` chart's EPP ClusterRole. Namespace-restricted mode: the chart creates the ServiceAccount, Role, and RoleBinding | Supplied with the deployment manifests | +| EPP replication | Not available; exactly one replica | Available through `DYN_EPP_PEER_SERVICE` | + +A standalone EPP is not managed by the Dynamo operator, so the operator generates no resources for +it. It uses neither the EPP ServiceAccount nor the `dynamo-platform` chart's EPP RBAC. Standalone +mode targets clusters without the Dynamo platform chart; do not install it for this path. A +standalone EPP carries its own ServiceAccount and Role, which grant the permissions that mode needs. +Do not bind a standalone EPP to the chart's EPP ClusterRole; its rules cover only the resources a +Dynamo-mode EPP reads. For the standalone path, see +[Using GAIE with vanilla vLLM](../../kubernetes/kv-aware-routing/vanilla-vllm-onramp.mdx). + ## DGD EPP Fields The EPP is a DGD component with `type: epp`. The Dynamo operator creates the EPP workload, Service, @@ -31,7 +53,9 @@ and is configured through `DYN_*` environment variables. It implements the GAIE - Number of EPP pods. Increase this value when endpoint-picker request volume requires more EPP capacity. + Number of EPP pods. Must be exactly `1`. Admission rejects any other value with the message + `EPP component must have exactly 1 replica`. EPP replication is not available in Dynamo mode. To run + several EPP replicas that share active-load state, use standalone mode. @@ -89,7 +113,8 @@ Gateway topology. Publishes and consumes best-effort active-sequence state across EPP replicas through the Dynamo event - plane. + plane. A DGD EPP component runs a single replica, so this setting has no peer to synchronize with + there.