Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/api/nixl-connect/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ flowchart LR

### Multimodal Example

In the case of the [Dynamo Multimodal Disaggregated Example](../../features/multimodal/multimodal-vllm.md):
In the case of the [Dynamo Multimodal Disaggregated Example](https://github.com/ai-dynamo/dynamo/blob/main/docs/features/multimodal/multimodal-vllm.md):

1. The HTTP frontend accepts a text prompt and a URL to an image.

Expand Down
12 changes: 6 additions & 6 deletions docs/backends/mocker_backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ title: Mocker Backend (Rust)

Reference Rust backend for Dynamo. Wraps the `dynamo-mocker` scheduler
in the `LLMEngine` contract from
[`dynamo-backend-common`](../../../lib/backend-common/), giving a
[`dynamo-backend-common`](https://github.com/ai-dynamo/dynamo/tree/main/lib/backend-common), giving a
full-fidelity mocked engine — one shared forward-pass loop paces all
in-flight requests (continuous batching), not independent per-request
timers. Use it as a template for writing your own Rust backend, a
Expand All @@ -17,7 +17,7 @@ forcing function that keeps the mocker scheduler tied to the
`LLMEngine` contract.

See the `LLMEngine` trait's doc comments in
[`engine.rs`](../../../lib/backend-common/src/engine.rs) for the
[`engine.rs`](https://github.com/ai-dynamo/dynamo/blob/main/lib/backend-common/src/engine.rs) for the
authoritative contract.

## Quick demo (docker compose)
Expand Down Expand Up @@ -73,7 +73,7 @@ reachable via `NATS_SERVER` / `ETCD_ENDPOINTS` env vars.

1. New crate depending on `dynamo-backend-common`; place under `lib/`.
2. Implement
[`LLMEngine`](../../../lib/backend-common/src/engine.rs)
[`LLMEngine`](https://github.com/ai-dynamo/dynamo/blob/main/lib/backend-common/src/engine.rs)
plus an inherent
`from_args(argv) -> Result<(Self, WorkerConfig), DynamoError>`.
3. Mirror the mocker example's three-line `main.rs`.
Expand Down Expand Up @@ -109,6 +109,6 @@ lib/backend-common/examples/mocker/

## References

- Crate: [`lib/backend-common/`](../../../lib/backend-common/)
- Example source: [`lib/backend-common/examples/mocker/`](../../../lib/backend-common/examples/mocker/)
- Conformance kit: [`lib/backend-common/src/testing.rs`](../../../lib/backend-common/src/testing.rs)
- Crate: [`lib/backend-common/`](https://github.com/ai-dynamo/dynamo/tree/main/lib/backend-common)
- Example source: [`lib/backend-common/examples/mocker/`](https://github.com/ai-dynamo/dynamo/tree/main/lib/backend-common/examples/mocker)
- Conformance kit: [`lib/backend-common/src/testing.rs`](https://github.com/ai-dynamo/dynamo/blob/main/lib/backend-common/src/testing.rs)
2 changes: 1 addition & 1 deletion docs/backends/sglang/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ docker run \
| [**Disaggregated Serving**](../../design-docs/disagg-serving.md) | ✅ | Prefill/decode separation with NIXL KV transfer |
| [**KV-Aware Routing**](../../components/router/README.md) | ✅ | |
| [**SLA-Based Planner**](../../components/planner/planner-guide.md) | ✅ | |
| [**Multimodal Support**](../../features/multimodal/multimodal-sglang.md) | ✅ | Image via EPD, E/PD, E/P/D patterns |
| [**Multimodal Support**](https://github.com/ai-dynamo/dynamo/blob/main/docs/features/multimodal/multimodal-sglang.md) | ✅ | Image via EPD, E/PD, E/P/D patterns |
| [**Diffusion Models**](sglang-diffusion.md) | ✅ | LLM diffusion, image, and video generation |
| [**Request Cancellation**](../../fault-tolerance/request-cancellation.md) | ✅ | Aggregated full; disaggregated decode-only |
| [**Graceful Shutdown**](../../fault-tolerance/graceful-shutdown.md) | ✅ | Discovery unregister + grace period |
Expand Down
4 changes: 2 additions & 2 deletions docs/backends/sglang/sglang-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ curl http://localhost:8000/v1/chat/completions \

### Multimodal with Disaggregated Components

For advanced multimodal deployments with separate encoder, prefill, and decode workers (E/PD and E/P/D patterns), see the dedicated [SGLang Multimodal](../../features/multimodal/multimodal-sglang.md) documentation.
For advanced multimodal deployments with separate encoder, prefill, and decode workers (E/PD and E/P/D patterns), see the dedicated [SGLang Multimodal](https://github.com/ai-dynamo/dynamo/blob/main/docs/features/multimodal/multimodal-sglang.md) documentation.

| Pattern | Script | Description |
| ------- | ------------------------------- | --------------------------------------------- |
Expand Down Expand Up @@ -187,7 +187,7 @@ Ensure both prefill and decode workers can reach each other over TCP. The bootst

- **[SGLang README](README.md)**: Quick start and feature overview
- **[Reference Guide](sglang-reference-guide.md)**: Architecture, configuration, and operational details
- **[SGLang Multimodal](../../features/multimodal/multimodal-sglang.md)**: Vision model deployment patterns
- **[SGLang Multimodal](https://github.com/ai-dynamo/dynamo/blob/main/docs/features/multimodal/multimodal-sglang.md)**: Vision model deployment patterns
- **[SGLang HiCache](../../integrations/sglang-hicache.md)**: Hierarchical cache integration
- **[Benchmarking](../../benchmarks/benchmarking.md)**: Performance benchmarking tools
- **[Tuning Disaggregated Performance](../../performance/tuning.md)**: P/D tuning guide
2 changes: 1 addition & 1 deletion docs/backends/sglang/sglang-reference-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ These arguments are added by Dynamo on top of SGLang's native arguments.
| `--dyn-reasoning-parser` | `DYN_REASONING_PARSER` | `None` | [Reasoning](../../agents/reasoning.md#supported-reasoning-parsers) parser for chain-of-thought models |
| `--custom-jinja-template` | `DYN_CUSTOM_JINJA_TEMPLATE` | `None` | Custom chat template path (incompatible with `--use-sglang-tokenizer`) |
| `--embedding-worker` | `DYN_SGL_EMBEDDING_WORKER` | `false` | Run as embedding worker (also sets SGLang's `--is-embedding`) |
| `--multimodal-encode-worker` | `DYN_SGL_MULTIMODAL_ENCODE_WORKER` | `false` | Run as [multimodal](../../features/multimodal/multimodal-sglang.md) encode worker (frontend-facing) |
| `--multimodal-encode-worker` | `DYN_SGL_MULTIMODAL_ENCODE_WORKER` | `false` | Run as [multimodal](https://github.com/ai-dynamo/dynamo/blob/main/docs/features/multimodal/multimodal-sglang.md) encode worker (frontend-facing) |
| `--multimodal-worker` | `DYN_SGL_MULTIMODAL_WORKER` | `false` | Run as multimodal LLM worker |
| `--image-diffusion-worker` | `DYN_SGL_IMAGE_DIFFUSION_WORKER` | `false` | Run as [image diffusion](sglang-diffusion.md#image-diffusion) worker |
| `--video-generation-worker` | `DYN_SGL_VIDEO_GENERATION_WORKER` | `false` | Run as [video generation](sglang-diffusion.md#video-generation) worker |
Expand Down
32 changes: 16 additions & 16 deletions docs/backends/trtllm/multinode/trtllm-multinode-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,29 @@ For general TensorRT-LLM features and engine configuration, see the
## Recommended Path

For multinode TensorRT-LLM deployments, start from the checked-in Kubernetes
recipes under [`recipes/`](../../../../recipes/README.md). Those manifests are
recipes under [`recipes/`](https://github.com/ai-dynamo/dynamo/blob/main/recipes/README.md). Those manifests are
the supported entrypoints for launching multi-node workers, frontend services,
and related routing components.

The main TRT-LLM recipe entrypoints are:

- [DeepSeek-R1 WideEP on GB200](../../../../recipes/deepseek-r1/trtllm/disagg/wide_ep/gb200/deploy.yaml)
- [Qwen3-235B-A22B-FP8 aggregated](../../../../recipes/qwen3-235b-a22b-fp8/trtllm/agg/deploy.yaml)
- [Qwen3-235B-A22B-FP8 disaggregated](../../../../recipes/qwen3-235b-a22b-fp8/trtllm/disagg/deploy.yaml)
- [Qwen3-32B-FP8 aggregated](../../../../recipes/qwen3-32b-fp8/trtllm/agg/deploy.yaml)
- [Qwen3-32B-FP8 disaggregated](../../../../recipes/qwen3-32b-fp8/trtllm/disagg/deploy.yaml)
- [GPT-OSS-120B aggregated](../../../../recipes/gpt-oss-120b/trtllm/agg/deploy.yaml)
- [GPT-OSS-120B disaggregated](../../../../recipes/gpt-oss-120b/trtllm/disagg/deploy.yaml)
- [Nemotron-3-Super-FP8 disaggregated](../../../../recipes/nemotron-3-super-fp8/trtllm/disagg/deploy.yaml)
- [DeepSeek-R1 WideEP on GB200](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-r1/trtllm/disagg/wide_ep/gb200/deploy.yaml)
- [Qwen3-235B-A22B-FP8 aggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-235b-a22b-fp8/trtllm/agg/deploy.yaml)
- [Qwen3-235B-A22B-FP8 disaggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-235b-a22b-fp8/trtllm/disagg/deploy.yaml)
Comment on lines +20 to +21

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Read-only verification: find the actual deploy manifests under the qwen3-235b recipe tree.
# Expected result: identify the real paths to agg/disagg manifests, then update the two markdown links.

fd -HI 'deploy*.ya?ml' recipes/qwen3-235b-a22b-fp8 -t f
rg -n --glob 'recipes/qwen3-235b-a22b-fp8/**' 'kind:\s*DynamoGraphDeployment|trtllm|agg|disagg' -S

Repository: ai-dynamo/dynamo

Length of output: 8324


Update the two Qwen3 recipe links to point to the correct directories.

The markdown links at lines 20–21 reference paths that don't exist:

  • recipes/qwen3-235b-a22b-fp8/trtllm/agg/deploy.yaml → no such file
  • recipes/qwen3-235b-a22b-fp8/trtllm/disagg/deploy.yaml → no such file

The actual deploy manifests are hardware-specific (in hopper/ and blackwell/ subdirectories). Link to the parent directories instead:

  • https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-235b-a22b-fp8/trtllm/agg/
  • https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-235b-a22b-fp8/trtllm/disagg/

Alternatively, if you want to reference specific variants, update the link text to clarify which hardware is being documented.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/backends/trtllm/multinode/trtllm-multinode-examples.md` around lines 20
- 21, The two markdown links labeled "Qwen3-235B-A22B-FP8 aggregated" and
"Qwen3-235B-A22B-FP8 disaggregated" point to non-existent deploy.yaml files;
update their hrefs to the parent agg/ and disagg/ directories (or to specific
hardware subdirs like hopper/ or blackwell/ with clarified link text) so the
links resolve to the actual deploy manifest locations.

- [Qwen3-32B-FP8 aggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-32b-fp8/trtllm/agg/deploy.yaml)
- [Qwen3-32B-FP8 disaggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-32b-fp8/trtllm/disagg/deploy.yaml)
- [GPT-OSS-120B aggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/gpt-oss-120b/trtllm/agg/deploy.yaml)
- [GPT-OSS-120B disaggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/gpt-oss-120b/trtllm/disagg/deploy.yaml)
- [Nemotron-3-Super-FP8 disaggregated](https://github.com/ai-dynamo/dynamo/blob/main/recipes/nemotron-3-super-fp8/trtllm/disagg/deploy.yaml)

For model-level setup, prerequisites, and hardware notes, use the recipe
README files:

- [DeepSeek-R1 recipes](../../../../recipes/deepseek-r1/README.md)
- [Qwen3-235B-A22B-FP8 recipes](../../../../recipes/qwen3-235b-a22b-fp8/README.md)
- [Qwen3-32B-FP8 recipes](../../../../recipes/qwen3-32b-fp8/README.md)
- [GPT-OSS-120B recipes](../../../../recipes/gpt-oss-120b/README.md)
- [Kimi-K2.5 recipes](../../../../recipes/kimi-k2.5/README.md)
- [DeepSeek-R1 recipes](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-r1/README.md)
- [Qwen3-235B-A22B-FP8 recipes](https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-235b-a22b-fp8/README.md)
- [Qwen3-32B-FP8 recipes](https://github.com/ai-dynamo/dynamo/blob/main/recipes/qwen3-32b-fp8/README.md)
- [GPT-OSS-120B recipes](https://github.com/ai-dynamo/dynamo/blob/main/recipes/gpt-oss-120b/README.md)
- [Kimi-K2.5 recipes](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k2.5/README.md)

## Quick Start

Expand Down Expand Up @@ -76,7 +76,7 @@ curl http://localhost:8000/v1/models
## Notes

- The TRT-LLM engine config files used by launch and deploy flows live under
[`examples/backends/trtllm/engine_configs/`](../../../../examples/backends/trtllm/engine_configs/README.md).
[`examples/backends/trtllm/engine_configs/`](https://github.com/ai-dynamo/dynamo/blob/main/examples/backends/trtllm/engine_configs/README.md).
- If you need to customize model parallelism, replica counts, or routing mode,
edit the recipe-local manifest rather than introducing a separate scheduler-specific guide.
- For the current catalog of supported recipes, see [recipes/README.md](../../../../recipes/README.md).
- For the current catalog of supported recipes, see [recipes/README.md](https://github.com/ai-dynamo/dynamo/blob/main/recipes/README.md).
2 changes: 1 addition & 1 deletion docs/backends/trtllm/trtllm-reference-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ TensorRT-LLM documents `n`/`best_of` behavior and validates this guard as greedy

## Multimodal Support

Dynamo with the TensorRT-LLM backend supports multimodal models, enabling you to process both text and images (or pre-computed embeddings) in a single request. For detailed setup instructions, example requests, and best practices, see the [TensorRT-LLM Multimodal Guide](../../features/multimodal/multimodal-trtllm.md).
Dynamo with the TensorRT-LLM backend supports multimodal models, enabling you to process both text and images (or pre-computed embeddings) in a single request. For detailed setup instructions, example requests, and best practices, see the [TensorRT-LLM Multimodal Guide](https://github.com/ai-dynamo/dynamo/blob/main/docs/features/multimodal/multimodal-trtllm.md).

## Diffusion Support (Experimental)

Expand Down
2 changes: 1 addition & 1 deletion docs/benchmarks/mocker-trace-replay.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ flowchart TD
MES --> P
```

See [`lib/mocker/src/replay/offline/README.md`](../../lib/mocker/src/replay/offline/README.md) for offline-harness internals (logical clock, event queue, worker model) and [`docs/mocker/mocker.md`](../mocker/mocker.md) for engine-core details (scheduler, KV block manager).
See the [Offline Replay README](https://github.com/ai-dynamo/dynamo/blob/main/lib/mocker/src/replay/offline/README.md) for offline-harness internals (logical clock, event queue, worker model) and the [Dynamo Mocker](../mocker/mocker.md) for engine-core details (scheduler, KV block manager).

## Quick Start

Expand Down
14 changes: 7 additions & 7 deletions docs/components/router/router-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@ Use the fixture-backed tests in `lib/bench/tests` when you want a realistic repl

Current coverage uses the checked-in 1000-line Mooncake trace fixture:

- [active_sequences_trace.rs](../../../lib/bench/tests/active_sequences_trace.rs)
- [mooncake_trace.rs](../../../lib/bench/tests/mooncake_trace.rs)
- [mooncake_trace_1000.jsonl](../../../lib/bench/testdata/mooncake_trace_1000.jsonl)
- [active_sequences_trace.rs](https://github.com/ai-dynamo/dynamo/blob/main/lib/bench/tests/active_sequences_trace.rs)
- [mooncake_trace.rs](https://github.com/ai-dynamo/dynamo/blob/main/lib/bench/tests/mooncake_trace.rs)
- [mooncake_trace_1000.jsonl](https://github.com/ai-dynamo/dynamo/blob/main/lib/bench/testdata/mooncake_trace_1000.jsonl)

These tests are useful for catching regressions such as:

Expand All @@ -66,10 +66,10 @@ Use the Python tests in `tests/router` when you need the full request plane and

Current entry points include:

- [test_router_e2e_with_mockers.py](../../../tests/router/test_router_e2e_with_mockers.py)
- [test_router_e2e_with_vllm.py](../../../tests/router/test_router_e2e_with_vllm.py)
- [test_router_e2e_with_trtllm.py](../../../tests/router/test_router_e2e_with_trtllm.py)
- [test_router_e2e_with_sglang.py](../../../tests/router/test_router_e2e_with_sglang.py)
- [test_router_e2e_with_mockers.py](https://github.com/ai-dynamo/dynamo/blob/main/tests/router/test_router_e2e_with_mockers.py)
- [test_router_e2e_with_vllm.py](https://github.com/ai-dynamo/dynamo/blob/main/tests/router/test_router_e2e_with_vllm.py)
- [test_router_e2e_with_trtllm.py](https://github.com/ai-dynamo/dynamo/blob/main/tests/router/test_router_e2e_with_trtllm.py)
- [test_router_e2e_with_sglang.py](https://github.com/ai-dynamo/dynamo/blob/main/tests/router/test_router_e2e_with_sglang.py)

Use this layer for changes involving:

Expand Down
4 changes: 2 additions & 2 deletions docs/components/router/standalone-indexer.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The standalone KV indexer (`python -m dynamo.indexer`) is a lightweight service
- It preserves P2P recovery and gap detection/replay for the standalone ZMQ path.
- It indexes device, host-pinned, and disk tier blocks and reports per-tier matches in `/query` responses.

This is distinct from the [Standalone Router](../../../components/src/dynamo/router/README.md), which is a full routing service. The standalone indexer provides only the indexing and query layer without routing logic.
This is distinct from the [Standalone Router](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/router/README.md), which is a full routing service. The standalone indexer provides only the indexing and query layer without routing logic.

For Dynamo-native remote indexing, use `--serve-indexer` on `dynamo.frontend` or `dynamo.router` and `--use-remote-indexer` on consumers instead. That request-plane service reuses the router's existing event ingestion and recovery machinery; it is not implemented by `dynamo.indexer`.

Expand Down Expand Up @@ -483,4 +483,4 @@ sequenceDiagram
- **[Mooncake KV Indexer RFC](https://github.com/kvcache-ai/Mooncake/issues/1403)**: Community API standardization for KV cache indexers
- **[Configuration and Tuning](router-configuration.md)**: Full KV router configuration and tuning
- **[Router Design](../../design-docs/router-design.md)**: Architecture and event transport modes
- **[Standalone Router](../../../components/src/dynamo/router/README.md)**: Full routing service (routes requests to workers)
- **[Standalone Router](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/router/README.md)**: Full routing service (routes requests to workers)
2 changes: 1 addition & 1 deletion docs/digest/agentic-inference/agentic-inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ The `agent_hints` fields:
- **`osl`** (output sequence length) is the harness's estimate of how many tokens this request will generate. The router uses this to gauge how long a worker will be occupied, which improves load balancing. A harness can learn this over time by tracking average output lengths per tool call type.
- **`speculative_prefill`** signals the orchestrator to begin caching this request's prefix on a likely worker before the full request is ready. This is useful when the harness knows a tool call is about to return and wants to warm the cache ahead of time.

The `cache_control` field will look familiar to anyone who has used Anthropic's prompt caching API. It tells the orchestrator to pin the computed prefix on the worker for the specified TTL, protecting it from eviction during tool call gaps. Currently `ephemeral` is the only supported type (to match Anthropic's API). We discuss how this works in the cache retention section below. You can find complete documentation on agent hints [here](../../components/frontend/nvext.md#cache-control).
The `cache_control` field will look familiar to anyone who has used Anthropic's prompt caching API. It tells the orchestrator to pin the computed prefix on the worker for the specified TTL, protecting it from eviction during tool call gaps. Currently `ephemeral` is the only supported type (to match Anthropic's API). We discuss how this works in the cache retention section below. You can find complete documentation on [Agent Hints](../../agents/agent-hints.md) and [NVIDIA Request Extensions](../../components/frontend/nvext.md).

## Layer 2: The Router

Expand Down
2 changes: 1 addition & 1 deletion docs/kubernetes/disagg-communication-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -820,5 +820,5 @@ resources:

- [Disaggregated Serving Architecture](../design-docs/disagg-serving.md)
- [AIConfigurator Deployment Guide](../features/disaggregated-serving/README.md)
- [NIXL Benchmark Deployment](../../deploy/pre-deployment/nixl/README.md)
- [NIXL Benchmark Deployment](https://github.com/ai-dynamo/dynamo/blob/main/deploy/pre-deployment/nixl/README.md)
- [KV Cache Transfer Methods](../backends/trtllm/trtllm-kv-cache-transfer.md)
2 changes: 1 addition & 1 deletion docs/kubernetes/snapshot.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ Failover restore is not yet available. The current Snapshot flow does not suppor

It is possible to checkpoint and restore pods without the Dynamo operator via the lower-level `snapshotctl` utility. However, the snapshot helm chart must be installed, with a running `snapshot-agent` DaemonSet in the namespace with the checkpoint PVC mounted.

`snapshotctl` is intended for lower-level debugging and validation workflows, not as the primary user-facing checkpoint interface. For command details and manifest requirements, see [deploy/snapshot/cmd/snapshotctl/README.md](../../deploy/snapshot/cmd/snapshotctl/README.md).
`snapshotctl` is intended for lower-level debugging and validation workflows, not as the primary user-facing checkpoint interface. For command details and manifest requirements, see the [snapshotctl README](https://github.com/ai-dynamo/dynamo/blob/main/deploy/snapshot/cmd/snapshotctl/README.md).

### Checkpoint from a worker pod manifest

Expand Down
Loading
Loading