Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ This writes to `docs/reference/api/openapi.json`.

## Service Discovery and Messaging

Dynamo uses TCP for inter-component communication. On Kubernetes, native resources ([CRDs + EndpointSlices](docs/fern/pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/service-discovery.md)) handle service discovery. External services are optional for most deployments:
Dynamo uses TCP for inter-component communication. On Kubernetes, native resources ([CRDs + EndpointSlices](docs/fern/pages/developer-guide/knowledge-base/concepts/system-architecture/architecture.md#discovery-plane)) handle service discovery. External services are optional for most deployments:

| Deployment | etcd | NATS | Notes |
|------------|------|------|-------|
Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ cargo run -p dynamo-llm --bin generate-frontend-openapi

## 服务发现与消息传递

Dynamo 使用 TCP 进行组件间通信。在 Kubernetes 上,原生资源([CRDs + EndpointSlices](docs/fern/pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/service-discovery.md))负责服务发现。对大多数部署来说,外部服务是可选的:
Dynamo 使用 TCP 进行组件间通信。在 Kubernetes 上,原生资源([CRDs + EndpointSlices](docs/fern/pages/developer-guide/knowledge-base/concepts/system-architecture/architecture.md#discovery-plane))负责服务发现。对大多数部署来说,外部服务是可选的:

| 部署 | etcd | NATS | 说明 |
|------|------|------|------|
Expand Down
140 changes: 89 additions & 51 deletions docs/fern/docs.yml

Large diffs are not rendered by default.

51 changes: 0 additions & 51 deletions docs/fern/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,6 @@ navigation:
- section: Model Loading
collapsed: true
contents:
- page: Model Caching
path: pages/kubernetes/model-deployment/model-loading/model-caching.mdx
- page: Fast Model Loading
path: pages/kubernetes/model-deployment/model-loading/fast-model-loading.mdx

Expand All @@ -170,9 +168,6 @@ navigation:
contents:
- page: Overview
path: pages/kubernetes/kv-aware-routing/overview.md
- page: Gateway API Routing Architecture
slug: gateway-api-routing-architecture
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/gateway-api-routing.md
- page: Using the Dynamo Frontend
path: pages/kubernetes/kv-aware-routing/dynamo-frontend.md
- page: Using GAIE with Dynamo
Expand All @@ -187,29 +182,10 @@ navigation:
contents:
- page: Overview
path: pages/kubernetes/disaggregated-serving/overview.md
- page: Disaggregated Communication
slug: disaggregated-communication
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/disagg-communication.md
- page: Sizing with AIConfigurator
slug: size-with-ai-configurator
path: pages/kubernetes/disaggregated-serving/sizing-with-aiconfigurator.mdx

- section: Multinode
slug: multinode
icon: network-wired
collapsed: true
contents:
- page: Multinode Deployments
path: pages/kubernetes/model-deployment/multinode-deployments.md
- page: Grove
path: pages/developer-guide/knowledge-base/kubernetes/multinode/grove.md
- page: Topology Aware Scheduling
path: pages/developer-guide/knowledge-base/kubernetes/multinode/topology-aware-scheduling.md
- page: Topology-Aware KV Transfer
path: pages/developer-guide/knowledge-base/kubernetes/multinode/topology-aware-kv-transfer.md
- page: LWS
path: pages/developer-guide/knowledge-base/kubernetes/multinode/lws.md

- section: KV Cache Offloading
icon: database
collapsed: true
Expand All @@ -231,8 +207,6 @@ navigation:
path: pages/kubernetes/fault-tolerance/request-rejection.md
- page: Graceful Shutdown
path: pages/kubernetes/fault-tolerance/graceful-shutdown.md
- page: Shadow Engine Failover
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/shadow-engine-failover.md

- section: Auto Deployment
icon: wand-magic-sparkles
Expand All @@ -248,35 +222,10 @@ navigation:
- page: Dynamo Planner
path: pages/kubernetes/auto-deployment/dynamo-planner.mdx

- section: Kubernetes Operator
slug: kubernetes-operator
icon: gears
collapsed: true
contents:
- page: Dynamo Operator
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/dynamo-operator.md
- page: Service Discovery
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/service-discovery.md
- page: Webhooks
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/webhooks.md
- page: Managing Models (DynamoModel)
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/managing-models-dynamomodel.md
- page: Autoscaling
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/autoscaling.md
- page: Rolling Update
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/rolling-update.md

- section: Operations
icon: screwdriver-wrench
collapsed: open-by-default
contents:
- section: Cold Start Optimizations
collapsed: true
contents:
- page: Dynamo Snapshot
path: pages/developer-guide/knowledge-base/kubernetes/kubernetes-operator/snapshot.md
- page: Model Express
path: pages/developer-guide/knowledge-base/kubernetes/model-loading/modelexpress.md
- page: Benchmarking with AIPerf
path: pages/kubernetes/operations/benchmarking-with-aiperf.mdx
- page: Performance Tuning
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ for [vLLM](../../recipes/cli-templates/vllm.mdx),
[TensorRT-LLM](../../recipes/cli-templates/tensorrt-llm.mdx).

If one worker requires more GPUs than a node provides, use a
[Kubernetes multinode deployment](../../kubernetes/model-deployment/multinode-deployments.md) instead of treating the GPU budget as one flat local device list.
[Kubernetes multinode deployment](../../kubernetes/installation/multinode-orchestration.md) instead of treating the GPU budget as one flat local device list.

</Step>
<Step title="Validate the recommendation" id="validate-the-recommendation">
Expand Down
2 changes: 1 addition & 1 deletion docs/fern/pages/cli/operations/lora-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ LoRA (Low-Rank Adaptation) serves specialized model variants without duplicating
# customer-support-lora 2 2 30s
```

See [Managing Models with DynamoModel](../../developer-guide/knowledge-base/kubernetes/kubernetes-operator/managing-models-dynamomodel.md) for the full CRD workflow.
See the [DynamoModel API reference](../../reference/kubernetes-api/full-api-reference.mdx#dynamomodel) for the CRD fields.

<AccordionGroup>
<Accordion title="Load imperatively with the worker system API">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -191,10 +191,9 @@ built-in cert controller (based on OPA cert-controller). On startup it:
3. Injects the CA bundle into `ValidatingWebhookConfiguration` and
`MutatingWebhookConfiguration` resources.

This matches production behavior and requires no external tooling. For
alternative certificate management (cert-manager or external certs), see the
[webhook documentation](../knowledge-base/kubernetes/kubernetes-operator/webhooks.md) and configure via
`helm_values` in `tilt-settings.yaml`.
This matches production behavior and requires no external tooling. Configure alternative certificate
management, such as cert-manager or external certificates, through `helm_values` in
`tilt-settings.yaml`.

## Typical Workflows

Expand Down
Loading
Loading