Repository navigation
feat(triton): Deliver dynamo.triton Functionality #13774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
76da164
feat(triton): Deliver dynamo.triton Functionality
whoisj 53773b4
adopt changes required by @dagil-nvidia
whoisj 3488b4a
apply changes recommended by @coderabbitai
whoisj bd489bc
annotate CODEOWNERS file
whoisj 9f24770
adopt changes recommended by @nv-rinig
whoisj f59b878
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj 42a2e0b
changes requested by @dagil-nvidia
whoisj 7eeaf15
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj 39f8599
Update components/src/dynamo/triton/main.py
whoisj 1ce3572
Update components/src/dynamo/triton/handlers.py
whoisj 481f30e
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj 9342635
Update components/src/dynamo/triton/handlers.py
whoisj 2c71913
adopt changes suggested by @tanmayv25
whoisj 611864b
fixup unit test issues
whoisj 2595c5b
fix github edit corruption
whoisj 6a78bce
Update components/src/dynamo/triton/tests/conftest.py
whoisj 1ef6553
fix codeowners' areas
whoisj 2a65977
feat(triton): Deliver Build, Test, and CI Functionality (#13943)
whoisj 0409e7a
apply change suggested by @Vinya456
whoisj 1a8f446
exclude triton from mypy check
whoisj 56b2681
improve pytest skip logic
whoisj 7ef340f
fix(triton): return None from pytest_ignore_collect for non-triton paths
nv-rinig c8ecb9a
fix(tests): return None from directory pytest_ignore_collect hooks
nv-rinig 47c566c
fix(tests): seed the real triton package before pytest collection
nv-rinig 10e4822
attempt skip tests when torch unavailable
whoisj ecd8f17
fix(tests): use try/except torch guard so E402 passes ruff and flake8
nv-rinig File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| <!-- # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 --> | ||
|
|
||
| # Triton | ||
|
|
||
| See [docs/backends/triton/](../../../../docs/backends/triton/README.md) for documentation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| from dynamo.triton.main import main | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import argparse | ||
|
|
||
| from dynamo.common.configuration.arg_group import ArgGroup | ||
| from dynamo.common.configuration.config_base import ConfigBase | ||
| from dynamo.common.configuration.utils import add_argument | ||
|
|
||
| _DISCOVER_BACKEND_CHOICES = ["kubernetes", "etcd", "file", "mem"] | ||
| _REQUEST_PLANE_CHOICES = ["tcp", "nats"] | ||
|
|
||
|
|
||
| class DynamoArgGroup(ArgGroup): | ||
| """Dynamo Runtime configuration options.""" | ||
|
|
||
| @staticmethod | ||
| def add_arguments(parser: argparse.ArgumentParser) -> None: | ||
| if not isinstance(parser, argparse.ArgumentParser): | ||
| raise TypeError("parser must be an instance of argparse.ArgumentParser") | ||
|
|
||
| # -- Dynamo Runtime Options ------------------------------------------------- | ||
| dynamo_group = parser.add_argument_group("Dynamo Runtime Options") | ||
| add_argument( | ||
| dynamo_group, | ||
| flag_name="--namespace", | ||
| env_var="DYN_NAMESPACE", | ||
| default="dynamo", | ||
| help="Dynamo namespace.", | ||
| ) | ||
| add_argument( | ||
| dynamo_group, | ||
| flag_name="--discovery-backend", | ||
| env_var="DYN_DISCOVERY_BACKEND", | ||
| default="etcd", | ||
| choices=_DISCOVER_BACKEND_CHOICES, | ||
| help="Service discovery backend: kubernetes (K8s API), etcd (distributed KV), file (local filesystem), mem (in-memory).", | ||
| ) | ||
| add_argument( | ||
| dynamo_group, | ||
| flag_name="--request-plane", | ||
| env_var="DYN_REQUEST_PLANE", | ||
| default="tcp", | ||
| choices=_REQUEST_PLANE_CHOICES, | ||
| help="How requests are distributed from routers to workers. " | ||
| "'tcp' is fastest.", | ||
| ) | ||
|
|
||
|
|
||
| class Config(ConfigBase): | ||
| """Configuration for Dynamo Runtime generic options.""" | ||
|
|
||
| discovery_backend: str | ||
| request_plane: str | ||
| namespace: str | ||
|
|
||
| def validate(self) -> None: | ||
| if hasattr(super(), "validate"): | ||
| super().validate() | ||
|
|
||
| if ( | ||
| not self.discovery_backend | ||
| or self.discovery_backend not in _DISCOVER_BACKEND_CHOICES | ||
| ): | ||
| raise ValueError( | ||
| "--discovery-backend is required and must be one of " | ||
| f"'{''', '''.join(_DISCOVER_BACKEND_CHOICES)}' " | ||
| "(or set DYN_DISCOVERY_BACKEND)." | ||
| ) | ||
| if not self.request_plane or self.request_plane not in _REQUEST_PLANE_CHOICES: | ||
| raise ValueError( | ||
| "--request-plane is required and must be one of " | ||
| f"'{''', '''.join(_REQUEST_PLANE_CHOICES)}' " | ||
| "(or set DYN_REQUEST_PLANE)." | ||
| ) | ||
| if not self.namespace: | ||
| raise ValueError("--namespace is required (or set DYN_NAMESPACE).") |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.