Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/auto_deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ For expert users, `build_and_run_ad.py` provides advanced configuration capabili

#### CLI Arguments with Dot Notation

The script supports flexible CLI argument parsing using dot notation to modify nested configurations dynamically. You can target any field in both the [`ExperimentConfig`](./build_and_run_ad.py) and nested [`AutoDeployConfig`](../../tensorrt_llm/_torch/auto_deploy/llm_args.py)/[`LlmArgs`](../../tensorrt_llm/_torch/auto_deploy/llm_args.) objects:
The script supports flexible CLI argument parsing using dot notation to modify nested configurations dynamically. You can target any field in both the [`ExperimentConfig`](./build_and_run_ad.py) and nested [`AutoDeployConfig`](../../tensorrt_llm/_torch/auto_deploy/llm_args.py)/[`LlmArgs`](../../tensorrt_llm/_torch/auto_deploy/llm_args.py) objects:

```bash
# Configure model parameters
Expand Down
2 changes: 1 addition & 1 deletion examples/models/core/qwen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,7 @@ For further details, please refer to [speculative-decoding.md](../../../../docs/
### Dynamo

NVIDIA Dynamo is a high-throughput low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments.
Dynamo supports TensorRT LLM as one of its inference engine. For details on how to use TensorRT LLM with Dynamo please refer to [LLM Deployment Examples using TensorRT-LLM](https://github.com/ai-dynamo/dynamo/blob/main/examples/tensorrt_llm/README.md)
Dynamo supports TensorRT LLM as one of its inference engine. For details on how to use TensorRT LLM with Dynamo please refer to [LLM Deployment Examples using TensorRT-LLM](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/trtllm/deploy)

## Qwen3-Next

Expand Down
4 changes: 2 additions & 2 deletions examples/wide_ep/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,8 @@ Refer to the [Troubleshooting and FAQ](https://github.com/NVIDIA/TensorRT-LLM/bl
## References

To understand more details on wide EP and the optimizations we've added, refer to the technical blog series: Scaling Expert Parallelism in TensorRT-LLM
- [Part 1: Design and Implementation of Large-scale EP](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog4_Scaling_Expert_Parallelism_in_TensorRT-LLM.md)
- [Part 2: Performance Status and Optimization](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog8_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md)
- [Part 1: Design and Implementation of Large-scale EP](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md)
- [Part 2: Performance Status and Optimization](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md)
- [Part 3: Pushing the Performance Boundary](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog14_Scaling_Expert_Parallelism_in_TensorRT-LLM_part3.md)

To review how wide EP helps with Blackwell's leading inference benchmarks, also read these recent blog posts:
Expand Down
2 changes: 1 addition & 1 deletion tensorrt_llm/scaffolding/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,6 @@ Future work includes the following aspects:

- Provide information for combined performance optimization with backends. Scaffolding can provide some information which is helpful for LLM inference engine. For a example, Controller may aware the prefix relation between generation requests, that would be helpful for kvcache reuse.

- Develop auxiliary components to support generic requirements. Now we have developed some interesting auxiliary components. For an [example](examples/scaffolding/token_budget_majority_vote.py), we developed `GenerationTokenCounter` as a task collection decorator so that Controller could get the output tokens count for itself and its Sub-Controller. There are still many such works waiting for us to do.
- Develop auxiliary components to support generic requirements. Now we have developed some interesting auxiliary components. For an [example](../../examples/scaffolding/token_budget_majority_vote.py), we developed `GenerationTokenCounter` as a task collection decorator so that Controller could get the output tokens count for itself and its Sub-Controller. There are still many such works waiting for us to do.

You can see more specific work in this [link](https://github.com/NVIDIA/TensorRT-LLM/issues/3706#issuecomment-2820015957).
4 changes: 2 additions & 2 deletions tests/integration/defs/perf/README_release_test.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,8 @@ if self._config.backend == "pytorch":

### 3.1 Full Test Cycles

1. **llm_perf_full.yml** - Release performance test
- [test_lists/qa/llm_perf_full.yml](../../test_lists/qa/llm_perf_full.yml)
1. **llm_perf_core.yml** - Release performance test

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/integration/test_lists/qa/README.md:56 still documents llm_perf_full.yml as "Main performance test configuration" — same rename, one-line fix, and it belongs in this PR rather than a follow-up.

- [test_lists/qa/llm_perf_core.yml](../../test_lists/qa/llm_perf_core.yml)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/integration/test_lists/qa/ currently contains only llm_perf_core.yml, llm_perf_disagg.yml, llm_perf_multinode.{yml,txt}, and llm_spark_perf.yml. So the three siblings of the link you fixed are still dead: llm_perf_cluster.yml (line 115), llm_perf_nim.yml (line 117), and llm_perf_sanity.yml (line 122). Fixing one entry in a list of four makes the rest look verified. Either update/remove them here or state in the PR description that they fall into the deferred stale-content group.

2. **llm_perf_cluster.yml** - Cluster performance test(for Blackwell)
- [test_lists/qa/llm_perf_cluster.yml](../../test_lists/qa/llm_perf_cluster.yml)
3. **llm_perf_nim.yml** - NIM performance test
Expand Down