From 0399a40f40021700a10fd8561c462582cac2be87 Mon Sep 17 00:00:00 2001 From: Anna Tchernych Date: Fri, 11 Sep 2026 16:02:36 -0700 Subject: [PATCH] docs: clarify EPP mode boundary and single-replica Dynamo mode The Gateway API routing reference described operator-generated EPP resources without stating which EPP mode they apply to, and documented spec.components[].replicas as something to increase for more endpoint-picker capacity. Admission rejects any value other than 1, and EPP replication is reachable only in standalone mode, where DYN_EPP_PEER_SERVICE enables peer discovery and cross-replica active-load sync. Readers could therefore take the operator chart's EPP RBAC as the EPP's RBAC in general and bind a standalone EPP to it. That ClusterRole covers only what a Dynamo-mode EPP reads; a standalone deployment ships its own ServiceAccount and Role. Add an EPP Modes section contrasting the two modes and stating that standalone mode runs no operator, correct the replicas field to the value admission enforces, and note that DYN_ROUTER_REPLICA_SYNC has no peer under the single-replica cap. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Anna Tchernych --- .../components/gateway-api-routing.mdx | 29 +++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/docs/fern/pages/reference/components/gateway-api-routing.mdx b/docs/fern/pages/reference/components/gateway-api-routing.mdx index 2a18720088e0..c7bbb8a24781 100644 --- a/docs/fern/pages/reference/components/gateway-api-routing.mdx +++ b/docs/fern/pages/reference/components/gateway-api-routing.mdx @@ -9,6 +9,28 @@ Use this reference for a `DynamoGraphDeployment` (DGD) that routes requests thro Inference Extension (GAIE) and the Dynamo Endpoint Picker Plugin (EPP). For the procedure and topology, see [Using GAIE with Dynamo](../../kubernetes/kv-aware-routing/gateway-api.mdx). +## EPP Modes + +The native Rust EPP runs in one of two modes, selected by the `DYN_EPP_MODE` environment variable. +This reference covers Dynamo mode only. The deprecated Go EPP, kept by `eppConfig` with the image +pinned to 1.4, does not read `DYN_EPP_MODE` and has no standalone mode. + +| Concern | Dynamo mode (`dynamo`, default) | Standalone mode (`standalone`) | +|---|---|---| +| Deployed by | Dynamo operator, from a DGD `epp` component | User-applied manifests | +| Dynamo runtime | Required. It uses whichever discovery and event planes the platform is configured for; etcd and NATS are needed only when selected | Not used | +| Worker discovery | Dynamo runtime discovery | Kubernetes pod watch driven by `InferencePool.spec.selector` | +| EPP ServiceAccount and RBAC | Cluster-wide mode: the operator creates the ServiceAccount and binds it to the `dynamo-platform` chart's EPP ClusterRole. Namespace-restricted mode: the chart creates the ServiceAccount, Role, and RoleBinding | Supplied with the deployment manifests | +| EPP replication | Not available; exactly one replica | Available through `DYN_EPP_PEER_SERVICE` | + +A standalone EPP is not managed by the Dynamo operator, so the operator generates no resources for +it. It uses neither the EPP ServiceAccount nor the `dynamo-platform` chart's EPP RBAC. Standalone +mode targets clusters without the Dynamo platform chart; do not install it for this path. A +standalone EPP carries its own ServiceAccount and Role, which grant the permissions that mode needs. +Do not bind a standalone EPP to the chart's EPP ClusterRole; its rules cover only the resources a +Dynamo-mode EPP reads. For the standalone path, see +[Using GAIE with vanilla vLLM](../../kubernetes/kv-aware-routing/vanilla-vllm-onramp.mdx). + ## DGD EPP Fields The EPP is a DGD component with `type: epp`. The Dynamo operator creates the EPP workload, Service, @@ -31,7 +53,9 @@ and is configured through `DYN_*` environment variables. It implements the GAIE - Number of EPP pods. Increase this value when endpoint-picker request volume requires more EPP capacity. + Number of EPP pods. Must be exactly `1`. Admission rejects any other value with the message + `EPP component must have exactly 1 replica`. EPP replication is not available in Dynamo mode. To run + several EPP replicas that share active-load state, use standalone mode. @@ -89,7 +113,8 @@ Gateway topology. Publishes and consumes best-effort active-sequence state across EPP replicas through the Dynamo event - plane. + plane. A DGD EPP component runs a single replica, so this setting has no peer to synchronize with + there.