Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
76da164
feat(triton): Deliver dynamo.triton Functionality
whoisj Aug 25, 2026
53773b4
adopt changes required by @dagil-nvidia
whoisj Aug 28, 2026
3488b4a
apply changes recommended by @coderabbitai
whoisj Aug 28, 2026
bd489bc
annotate CODEOWNERS file
whoisj Aug 28, 2026
9f24770
adopt changes recommended by @nv-rinig
whoisj Sep 2, 2026
f59b878
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj Sep 2, 2026
42a2e0b
changes requested by @dagil-nvidia
whoisj Sep 4, 2026
7eeaf15
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj Sep 4, 2026
39f8599
Update components/src/dynamo/triton/main.py
whoisj Sep 4, 2026
1ce3572
Update components/src/dynamo/triton/handlers.py
whoisj Sep 4, 2026
481f30e
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj Sep 9, 2026
9342635
Update components/src/dynamo/triton/handlers.py
whoisj Sep 9, 2026
2c71913
adopt changes suggested by @tanmayv25
whoisj Sep 9, 2026
611864b
fixup unit test issues
whoisj Sep 9, 2026
2595c5b
fix github edit corruption
whoisj Sep 9, 2026
6a78bce
Update components/src/dynamo/triton/tests/conftest.py
whoisj Sep 9, 2026
1ef6553
fix codeowners' areas
whoisj Sep 9, 2026
2a65977
feat(triton): Deliver Build, Test, and CI Functionality (#13943)
whoisj Sep 10, 2026
0409e7a
apply change suggested by @Vinya456
whoisj Sep 10, 2026
1a8f446
exclude triton from mypy check
whoisj Sep 10, 2026
56b2681
improve pytest skip logic
whoisj Sep 10, 2026
7ef340f
fix(triton): return None from pytest_ignore_collect for non-triton paths
nv-rinig Sep 11, 2026
c8ecb9a
fix(tests): return None from directory pytest_ignore_collect hooks
nv-rinig Sep 11, 2026
47c566c
fix(tests): seed the real triton package before pytest collection
nv-rinig Sep 11, 2026
10e4822
attempt skip tests when torch unavailable
whoisj Sep 11, 2026
ecd8f17
fix(tests): use try/except torch guard so E402 passes ruff and flake8
nv-rinig Sep 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 29 additions & 22 deletions .github/FILTERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,35 @@ The `filters.yaml` file controls which CI jobs run based on changed files.

When you open a PR, CI checks which files changed and runs only relevant jobs:

| Filter | Triggers |
|--------|----------|
| `core` | Main test suite (vLLM, SGLang, TRT-LLM containers) |
| `dev_images` | dev / local-dev image builds only (no runtime or GPU jobs) |
| `operator` | Kubernetes operator tests |
| `snapshot` | Checkpoint-placeholder image + all-framework standalone Snapshot deploy tests (github.com/ai-dynamo/snapshot is external; this covers Dynamo's integration surface) |
| `snapshot_vllm` / `snapshot_sglang` / `snapshot_trtllm` | That framework's checkpoint deploy suite |
| `deploy` | Deploy-specific tests |
| `vllm` / `sglang` / `trtllm` | Backend-specific tests |
| `sidecar` | Unified multi-architecture sidecar image build, publish, and compliance checks for changes under `lib/sidecar/**` (docs excluded), its shared workflow, and shared compliance inputs |
| `benchmarks` | Dynamo runtime pipeline (runs `tests/benchmarks/**` pytest suite) |
| `sample` | Sample-backend unified test (piggybacks on vllm image) |
| `efa` | EFA runtime image builds for vLLM, SGLang, TRT-LLM (`container/templates/aws.Dockerfile` change) |
| `docs` | Docs Lint, Fern Configuration, Docs Website Composition, and Fern Broken Links checks; Fern preview or publish workflow |
| `fern_components` | Parse custom MDX components (a step inside Fern Configuration Check) |
| `examples` | Recipe Kustomize generation and docs-artifact unit checks |
| `ignore` | Nothing (classification only) |
| `rust` | Rust pre merge checks |

> **Note:** `ignore` doesn't directly trigger CI jobs. It exists to satisfy coverage requirements - every file must match at least one filter. Sidecar source and proto files also match `rust`, so the existing workspace Rust checks cover sidecar tests before the image is built and published. `docs` gates the Docs Lint, Fern Configuration Check, Docs Website Composition Check, and Fern Broken Links Check jobs in `pre-merge.yml`. `examples` gates Recipe Check.

> **TODO:** The sidecar image also consumes root Cargo files, shared libraries, and composite actions. Expanding the filter to cover every remaining build input is deferred until the additional PR CI fan-out is evaluated and agreed.
| Filter | Triggers |
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `core` | Main test suite (vLLM, SGLang, TRT-LLM containers) |
| `dev_images` | dev / local-dev image builds only (no runtime or GPU jobs) |
| `operator` | Kubernetes operator tests |
| `snapshot` | Checkpoint-placeholder image + all-framework standalone Snapshot deploy tests (github.com/ai-dynamo/snapshot is external; this covers Dynamo's integration surface) |
| `snapshot_vllm` / `snapshot_sglang` / `snapshot_trtllm` | That framework's checkpoint deploy suite |
| `deploy` | Deploy-specific tests |
| `vllm` / `sglang` / `trtllm` / `triton` | Backend-specific tests |
| `sidecar` | Unified multi-architecture sidecar image build, publish, and compliance checks for changes under `lib/sidecar/**` (docs excluded), its shared workflow, and shared compliance inputs |
| `benchmarks` | Dynamo runtime pipeline (runs `tests/benchmarks/**` pytest suite) |
| `sample` | Sample-backend unified test (piggybacks on vllm image) |
| `efa` | EFA runtime image builds for vLLM, SGLang, TRT-LLM (`container/templates/aws.Dockerfile` change) |
| `docs` | Docs Lint, Fern Configuration, Docs Website Composition, and Fern Broken Links checks; Fern preview or publish workflow |
| `fern_components` | Parse custom MDX components (a step inside Fern Configuration Check) |
| `examples` | Recipe Kustomize generation and docs-artifact unit checks |
| `ignore` | Nothing (classification only) |
| `rust` | Rust pre merge checks |

> [!NOTE]
> `ignore` doesn't directly trigger CI jobs.
> It exists to satisfy coverage requirements - every file must match at least one filter.
> Sidecar source and proto files also match `rust`, so the existing workspace Rust checks cover sidecar tests before the image is built and published.
> `docs` gates the Docs Lint, Fern Configuration Check, Docs Website Composition Check, and Fern Broken Links Check jobs in `pre-merge.yml`.
> `examples` gates Recipe Check.

> [!TODO]
> The sidecar image also consumes root Cargo files, shared libraries, and composite actions.
> Expanding the filter to cover every remaining build input is deferred until the additional PR CI fan-out is evaluated and agreed.

## Fixing "Uncovered Files" Errors

Expand Down
6 changes: 6 additions & 0 deletions .github/codeowners/areas.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,12 @@ areas:
- examples/backends/sglang/
- lib/mocker/servers/sglang/
- lib/sidecar/sglang/
- label: backend-triton
github_team: '@ai-dynamo/dynamo-backend-triton-codeowners'
path_globs:
- components/src/dynamo/triton/**
- container/launch_message/triton.txt
- docs/backends/triton
- label: kv-memory
github_team: '@ai-dynamo/dynamo-kv-memory-codeowners'
path_globs:
Expand Down
7 changes: 7 additions & 0 deletions CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
# Area index (base owner per subsystem; the catch-all owns everything else):
# agents @ai-dynamo/dynamo-agents-codeowners
# backend-sglang @ai-dynamo/dynamo-backend-sglang-codeowners
# backend-triton @ai-dynamo/dynamo-backend-triton-codeowners
# backend-trtllm @ai-dynamo/dynamo-backend-trtllm-codeowners
# backend-vllm @ai-dynamo/dynamo-backend-vllm-codeowners
# diffusion @ai-dynamo/dynamo-diffusion-codeowners
Expand All @@ -34,6 +35,7 @@
# Teams referenced (each must exist in the org before this file validates):
# @ai-dynamo/dynamo-agents-codeowners
# @ai-dynamo/dynamo-backend-sglang-codeowners
# @ai-dynamo/dynamo-backend-triton-codeowners
# @ai-dynamo/dynamo-backend-trtllm-codeowners
# @ai-dynamo/dynamo-backend-vllm-codeowners
# @ai-dynamo/dynamo-diffusion-codeowners
Expand Down Expand Up @@ -107,6 +109,9 @@
/components/src/dynamo/sglang/tests/ @ai-dynamo/dynamo-backend-sglang-codeowners
/components/src/dynamo/sglang/request_handlers/ @ai-dynamo/dynamo-backend-sglang-codeowners

# === backend-triton (@ai-dynamo/dynamo-backend-triton-codeowners) ===
/components/src/dynamo/triton/** @ai-dynamo/dynamo-backend-triton-codeowners

# === kv-memory (@ai-dynamo/dynamo-kv-memory-codeowners) ===
/lib/memory/ @ai-dynamo/dynamo-kv-memory-codeowners
/lib/tokens/ @ai-dynamo/dynamo-kv-memory-codeowners
Expand Down Expand Up @@ -469,6 +474,7 @@ requirements.txt
/lib/llm/src/grpc.rs @ai-dynamo/dynamo-frontend-codeowners
/lib/llm/src/http.rs @ai-dynamo/dynamo-frontend-codeowners
/lib/llm/src/lora.rs @ai-dynamo/dynamo-frontend-codeowners
/docs/backends/triton @ai-dynamo/dynamo-backend-triton-codeowners
/tests/frontend/grpc/ @ai-dynamo/dynamo-frontend-codeowners
/tests/router/configs/ @ai-dynamo/dynamo-router-codeowners
/lib/llm/src/kv_router/ @ai-dynamo/dynamo-router-codeowners
Expand Down Expand Up @@ -535,6 +541,7 @@ requirements.txt
/lib/llm/src/block_manager/offload/ @ai-dynamo/dynamo-kv-memory-codeowners
/lib/llm/src/block_manager/state.rs @ai-dynamo/dynamo-kv-memory-codeowners
/lib/llm/src/block_manager/storage/ @ai-dynamo/dynamo-kv-memory-codeowners
/container/launch_message/triton.txt @ai-dynamo/dynamo-backend-triton-codeowners
/lib/backend-common/examples/mocker/ @ai-dynamo/dynamo-performance-codeowners
/lib/bindings/python/rust/parsers.rs @ai-dynamo/dynamo-frontend-codeowners
/lib/llm/src/block_manager/layout.rs @ai-dynamo/dynamo-kv-memory-codeowners
Expand Down
6 changes: 6 additions & 0 deletions components/src/dynamo/triton/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<!-- # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0 -->

# Triton

See [docs/backends/triton/](../../../../docs/backends/triton/README.md) for documentation.
2 changes: 2 additions & 0 deletions components/src/dynamo/triton/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
7 changes: 7 additions & 0 deletions components/src/dynamo/triton/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

from dynamo.triton.main import main

if __name__ == "__main__":
main()
77 changes: 77 additions & 0 deletions components/src/dynamo/triton/args.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

import argparse

from dynamo.common.configuration.arg_group import ArgGroup
from dynamo.common.configuration.config_base import ConfigBase
from dynamo.common.configuration.utils import add_argument

_DISCOVER_BACKEND_CHOICES = ["kubernetes", "etcd", "file", "mem"]
_REQUEST_PLANE_CHOICES = ["tcp", "nats"]


class DynamoArgGroup(ArgGroup):
"""Dynamo Runtime configuration options."""

@staticmethod
def add_arguments(parser: argparse.ArgumentParser) -> None:
if not isinstance(parser, argparse.ArgumentParser):
raise TypeError("parser must be an instance of argparse.ArgumentParser")

# -- Dynamo Runtime Options -------------------------------------------------
dynamo_group = parser.add_argument_group("Dynamo Runtime Options")
add_argument(
dynamo_group,
flag_name="--namespace",
env_var="DYN_NAMESPACE",
default="dynamo",
help="Dynamo namespace.",
)
add_argument(
dynamo_group,
flag_name="--discovery-backend",
env_var="DYN_DISCOVERY_BACKEND",
default="etcd",
choices=_DISCOVER_BACKEND_CHOICES,
help="Service discovery backend: kubernetes (K8s API), etcd (distributed KV), file (local filesystem), mem (in-memory).",
)
add_argument(
dynamo_group,
flag_name="--request-plane",
env_var="DYN_REQUEST_PLANE",
default="tcp",
choices=_REQUEST_PLANE_CHOICES,
help="How requests are distributed from routers to workers. "
"'tcp' is fastest.",
)


class Config(ConfigBase):
"""Configuration for Dynamo Runtime generic options."""

discovery_backend: str
request_plane: str
namespace: str

def validate(self) -> None:
if hasattr(super(), "validate"):
super().validate()

if (
not self.discovery_backend
or self.discovery_backend not in _DISCOVER_BACKEND_CHOICES
):
raise ValueError(
"--discovery-backend is required and must be one of "
f"'{''', '''.join(_DISCOVER_BACKEND_CHOICES)}' "
"(or set DYN_DISCOVERY_BACKEND)."
)
if not self.request_plane or self.request_plane not in _REQUEST_PLANE_CHOICES:
raise ValueError(
"--request-plane is required and must be one of "
f"'{''', '''.join(_REQUEST_PLANE_CHOICES)}' "
"(or set DYN_REQUEST_PLANE)."
)
if not self.namespace:
raise ValueError("--namespace is required (or set DYN_NAMESPACE).")
Loading
Loading