[NVIDIA] Add DSR1 TensorRT Support and Enhanced Plotting - #7
Merged
Conversation
Collaborator
|
Thank you for the PR. Can you revert the |
Collaborator
Author
|
Done. |
cquil11
changed the title
Oseltamivir
added a commit
that referenced
this pull request
Jun 23, 2026
Add summarize.py (compact NCCL/DeepEP results table, printed at end of every job) and make it the result gate. Fix review findings: benchmark failures/skipped-deepep now fail the job instead of reporting green (#1); DeepEP nodes from SLURM_NNODES not world_size//8 (#3); apply Buffer.set_num_sms so num_comm_sms is real (#8); nccl-tests -c 1 with a missing check footer is now invalid (#7); use context managers for file reads (#4,#5); launchers export COLLECTIVEX_IMAGE/_DIGEST for provenance (#9); trim workflow_dispatch sku options to launcher-backed pools (#2). Artifact-path finding (#6) already fixed via cx_collect_results.
Oseltamivir
added a commit
that referenced
this pull request
Aug 7, 2026
queries.md #7. A class-level attribute naming another attribute, so seven shared getattr/setattr sites could reach a field one adapter spelled differently. It bought nothing: the only divergence was nccl-ep calling it combine_input_t, and nothing outside our own code ever saw the name. Verified before renaming, because the "_t" suffix looks like it might be load-bearing: - The nccl handle is adapter-owned. It is our namespace object holding adapter fields plus h.handle, which is the actual library group, so nccl.ep never observes this field's name. - The value genuinely is not a torch tensor -- it is an nccl.ep wrapper, which is what the suffix connoted -- but every shared site only passes it through opaquely. The rename keeps that property, and the comment now states it where the field is set rather than encoding it in the spelling. - No collision: h.combine_in_t is a DIFFERENT field (the dispatch output that HT slices), and it stays. The two names differing by one character is itself an argument for the rename. - The locally-untracked tests/test_ep_nccl_handle.py does not reference the field. So: nccl-ep's stage() sets handle.combine_input like every other adapter, the base declaration and flashinfer's redundant re-declaration of the default both go, and the six getattr/setattr sites become direct attribute access. The test that asserted the indirection was tautological -- it set combine_input_attr to "combine_input" and then checked the machinery honoured it, which is true by construction. It is now the behavioural claim that matters: a staged tensor reaches combine. 163 tests green.
xiaohuguo2023
added a commit
that referenced
this pull request
Aug 19, 2026
Move the K3 FP4 MI355X recipe onto vllm/vllm-openai-rocm:nightly-8efa13b700f1836657699cae2503dc2feab27fa0 (vLLM 0.27.2rc1.dev122+g8efa13b70, ROCm 7.2.3): 3 image: pins in configs/amd-master.yaml (:627/:1296/:1325) + doc/comment tag copies + docs/waiver/2508.md. Also carries the native) KV-offload arm in the base entry script. WIP — NOT yet validated on the new image: 0.27 rewrote rocm_aiter_mla.py, so the patch orchestrator dies at 5/9 (patch_fp8asm.py padded_q anchor). Patch reconciliation (drop upstreamed patches #1/#5/#6 + trio #2/#3/#4, keep #7 offload eagle veto, re-port dspark 4/7,4c,5/7) + conc-1 DSpark boot/accuracy + perf compare pending. Do not merge until green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Aug 27, 2026
This was referenced Aug 27, 2026
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds TensorRT-LLM FP4 support for DSR1 model and enhances the plotting system to better handle multiple model variants and precision types.
Changes Made
1. DSR1 Template Enhancements
✅ Added precision parameter to DSR1 template with fp8 default
✅ Added b200-trt job for DSR1 with TensorRT support
✅ Set b200-trt to use fp4 precision specifically
✅ Updated collect-results to include b200-trt job
2. TRT-LLM Configuration
✅ Created dsr1_b200_trt_slurm.sh benchmark script
✅ Uses TensorRT-LLM with trtllm-serve command
✅ Configured for DSR1 FP4 model (nvidia/DeepSeek-R1-0528-FP4)
✅ MTP support
3. Plotting System Improvements
✅ Enhanced model grouping - groups by model family (70b, dsr1) instead of full model names
✅ Added precision distinction - different markers for fp8 (circles) vs fp4 (squares)
✅ Improved legend labels - shows precision in labels (e.g., "B200-TRT (fp4)")
New Features