diff --git a/examples/auto_deploy/README.md b/examples/auto_deploy/README.md index 343ad8c87f35..8bc71a01528d 100644 --- a/examples/auto_deploy/README.md +++ b/examples/auto_deploy/README.md @@ -176,7 +176,7 @@ For expert users, `build_and_run_ad.py` provides advanced configuration capabili #### CLI Arguments with Dot Notation -The script supports flexible CLI argument parsing using dot notation to modify nested configurations dynamically. You can target any field in both the [`ExperimentConfig`](./build_and_run_ad.py) and nested [`AutoDeployConfig`](../../tensorrt_llm/_torch/auto_deploy/llm_args.py)/[`LlmArgs`](../../tensorrt_llm/_torch/auto_deploy/llm_args.) objects: +The script supports flexible CLI argument parsing using dot notation to modify nested configurations dynamically. You can target any field in both the [`ExperimentConfig`](./build_and_run_ad.py) and nested [`AutoDeployConfig`](../../tensorrt_llm/_torch/auto_deploy/llm_args.py)/[`LlmArgs`](../../tensorrt_llm/_torch/auto_deploy/llm_args.py) objects: ```bash # Configure model parameters diff --git a/examples/models/core/qwen/README.md b/examples/models/core/qwen/README.md index 6c7654357a18..86fd8bb92351 100644 --- a/examples/models/core/qwen/README.md +++ b/examples/models/core/qwen/README.md @@ -289,7 +289,7 @@ For further details, please refer to [speculative-decoding.md](../../../../docs/ ### Dynamo NVIDIA Dynamo is a high-throughput low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments. -Dynamo supports TensorRT LLM as one of its inference engine. For details on how to use TensorRT LLM with Dynamo please refer to [LLM Deployment Examples using TensorRT-LLM](https://github.com/ai-dynamo/dynamo/blob/main/examples/tensorrt_llm/README.md) +Dynamo supports TensorRT LLM as one of its inference engine. For details on how to use TensorRT LLM with Dynamo please refer to [LLM Deployment Examples using TensorRT-LLM](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/trtllm/deploy) ## Qwen3-Next diff --git a/examples/wide_ep/README.md b/examples/wide_ep/README.md index 4f2382225afd..c066bcd10a9d 100644 --- a/examples/wide_ep/README.md +++ b/examples/wide_ep/README.md @@ -166,8 +166,8 @@ Refer to the [Troubleshooting and FAQ](https://github.com/NVIDIA/TensorRT-LLM/bl ## References To understand more details on wide EP and the optimizations we've added, refer to the technical blog series: Scaling Expert Parallelism in TensorRT-LLM - - [Part 1: Design and Implementation of Large-scale EP](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog4_Scaling_Expert_Parallelism_in_TensorRT-LLM.md) - - [Part 2: Performance Status and Optimization](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog8_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md) + - [Part 1: Design and Implementation of Large-scale EP](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog04_Scaling_Expert_Parallelism_in_TensorRT-LLM.md) + - [Part 2: Performance Status and Optimization](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog08_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.md) - [Part 3: Pushing the Performance Boundary](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog14_Scaling_Expert_Parallelism_in_TensorRT-LLM_part3.md) To review how wide EP helps with Blackwell's leading inference benchmarks, also read these recent blog posts: diff --git a/tensorrt_llm/scaffolding/README.md b/tensorrt_llm/scaffolding/README.md index ff6adc0672e7..7f887abd2efd 100644 --- a/tensorrt_llm/scaffolding/README.md +++ b/tensorrt_llm/scaffolding/README.md @@ -38,6 +38,6 @@ Future work includes the following aspects: - Provide information for combined performance optimization with backends. Scaffolding can provide some information which is helpful for LLM inference engine. For a example, Controller may aware the prefix relation between generation requests, that would be helpful for kvcache reuse. -- Develop auxiliary components to support generic requirements. Now we have developed some interesting auxiliary components. For an [example](examples/scaffolding/token_budget_majority_vote.py), we developed `GenerationTokenCounter` as a task collection decorator so that Controller could get the output tokens count for itself and its Sub-Controller. There are still many such works waiting for us to do. +- Develop auxiliary components to support generic requirements. Now we have developed some interesting auxiliary components. For an [example](../../examples/scaffolding/token_budget_majority_vote.py), we developed `GenerationTokenCounter` as a task collection decorator so that Controller could get the output tokens count for itself and its Sub-Controller. There are still many such works waiting for us to do. You can see more specific work in this [link](https://github.com/NVIDIA/TensorRT-LLM/issues/3706#issuecomment-2820015957). diff --git a/tests/integration/defs/perf/README_release_test.md b/tests/integration/defs/perf/README_release_test.md index 5555b0445ccc..e6b9cb1d18f9 100644 --- a/tests/integration/defs/perf/README_release_test.md +++ b/tests/integration/defs/perf/README_release_test.md @@ -109,8 +109,8 @@ if self._config.backend == "pytorch": ### 3.1 Full Test Cycles -1. **llm_perf_full.yml** - Release performance test - - [test_lists/qa/llm_perf_full.yml](../../test_lists/qa/llm_perf_full.yml) +1. **llm_perf_core.yml** - Release performance test + - [test_lists/qa/llm_perf_core.yml](../../test_lists/qa/llm_perf_core.yml) 2. **llm_perf_cluster.yml** - Cluster performance test(for Blackwell) - [test_lists/qa/llm_perf_cluster.yml](../../test_lists/qa/llm_perf_cluster.yml) 3. **llm_perf_nim.yml** - NIM performance test