-
Notifications
You must be signed in to change notification settings - Fork 691
docs: Reference Deepseek R1 configs in TRTLLM README #1414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
WalkthroughThe README file in the TensorRT LLM examples directory was updated to include instructions for deploying Deepseek R1 using specific configuration files. The update provides guidance on replacing default configs with Deepseek R1 configs and includes a reference link for GB200 hardware examples. Changes
Poem
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (1)
examples/tensorrt_llm/README.md (1)
129-134: Consider a dedicated subsection for Deepseek R1 configs.
Extract these lines into a standalone "#### Deepseek R1 Example Configs" subheading (placed above the head/worker node steps) to improve readability and clearly separate R1-specific guidance from the multi-node instructions.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
examples/tensorrt_llm/README.md(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Mirror Repository to GitLab
- GitHub Check: Build and Test - vllm
🔇 Additional comments (1)
examples/tensorrt_llm/README.md (1)
129-134: Verify Deepseek R1 config directory and files.
Ensure that theconfigs/deepseek_r1path exists relative to this README and contains the referencedagg.yamlanddisagg.yamlfiles. You may also consider using a leading./(e.g.,./configs/deepseek_r1/) to make it explicit that this is a directory link.
|
Gitlab failure is unrelated to this PR, introduced by some other changes in new benchmarks/ directory - reached out to Rudy to help fix those failures separately. |
Overview:
Add explicit reference to Deepseek R1 configs for discoverability. Note these are just example configs for a particular configuration on GB200. They can be customized as needed for different parallelism strategies and hardware configurations, such as 8xH200 nodes.
Currently, these configs assume that each worker can fit on a single node (4xGB200, 8xH200, etc.), but support and steps for serving a single worker instance on multiple nodes (ex: 16xH100) will come in the future.
Summary by CodeRabbit