Skip to content

docs: clarify EPP mode boundary and single-replica Dynamo mode fixes [DYN-4310] #14756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
atchernych merged 2 commits into from
Sep 16, 2026
Merged

docs: clarify EPP mode boundary and single-replica Dynamo mode fixes [DYN-4310] #14756

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0399a40
docs: clarify EPP mode boundary and single-replica Dynamo mode
atchernych Sep 11, 2026
c1e76dd
Merge remote-tracking branch 'origin/main' into fix/epp-rbac-services…
atchernych Sep 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions docs/fern/pages/reference/components/gateway-api-routing.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,28 @@ Use this reference for a `DynamoGraphDeployment` (DGD) that routes requests thro
Inference Extension (GAIE) and the Dynamo Endpoint Picker Plugin (EPP). For the procedure and
topology, see [Using GAIE with Dynamo](../../kubernetes/kv-aware-routing/gateway-api.mdx).

## EPP Modes

The native Rust EPP runs in one of two modes, selected by the `DYN_EPP_MODE` environment variable.
This reference covers Dynamo mode only. The deprecated Go EPP, kept by `eppConfig` with the image
pinned to 1.4, does not read `DYN_EPP_MODE` and has no standalone mode.

| Concern | Dynamo mode (`dynamo`, default) | Standalone mode (`standalone`) |
|---|---|---|
| Deployed by | Dynamo operator, from a DGD `epp` component | User-applied manifests |
| Dynamo runtime | Required. It uses whichever discovery and event planes the platform is configured for; etcd and NATS are needed only when selected | Not used |
| Worker discovery | Dynamo runtime discovery | Kubernetes pod watch driven by `InferencePool.spec.selector` |
| EPP ServiceAccount and RBAC | Cluster-wide mode: the operator creates the ServiceAccount and binds it to the `dynamo-platform` chart's EPP ClusterRole. Namespace-restricted mode: the chart creates the ServiceAccount, Role, and RoleBinding | Supplied with the deployment manifests |
| EPP replication | Not available; exactly one replica | Available through `DYN_EPP_PEER_SERVICE` |

A standalone EPP is not managed by the Dynamo operator, so the operator generates no resources for
it. It uses neither the EPP ServiceAccount nor the `dynamo-platform` chart's EPP RBAC. Standalone
mode targets clusters without the Dynamo platform chart; do not install it for this path. A
standalone EPP carries its own ServiceAccount and Role, which grant the permissions that mode needs.
Do not bind a standalone EPP to the chart's EPP ClusterRole; its rules cover only the resources a
Dynamo-mode EPP reads. For the standalone path, see
[Using GAIE with vanilla vLLM](../../kubernetes/kv-aware-routing/vanilla-vllm-onramp.mdx).

## DGD EPP Fields

The EPP is a DGD component with `type: epp`. The Dynamo operator creates the EPP workload, Service,
Expand All @@ -31,7 +53,9 @@ and is configured through `DYN_*` environment variables. It implements the GAIE
</ParamField>

<ParamField path="spec.components[].replicas" type="integer" default="1">
Number of EPP pods. Increase this value when endpoint-picker request volume requires more EPP capacity.
Number of EPP pods. Must be exactly `1`. Admission rejects any other value with the message
`EPP component must have exactly 1 replica`. EPP replication is not available in Dynamo mode. To run
several EPP replicas that share active-load state, use standalone mode.
</ParamField>

<ParamField path="spec.components[].frontendSidecar" type="string" required={true}>
Expand Down Expand Up @@ -89,7 +113,8 @@ Gateway topology.

<ParamField path="DYN_ROUTER_REPLICA_SYNC" type="boolean" default="false">
Publishes and consumes best-effort active-sequence state across EPP replicas through the Dynamo event
plane.
plane. A DGD EPP component runs a single replica, so this setting has no peer to synchronize with
there.
</ParamField>

<ParamField path="DYN_ROUTER_TRACK_ACTIVE_BLOCKS" type="boolean" default="true">
Expand Down
Loading