Skip to content

[https://nvbugs/6786555][fix] [https://nvbugs/6786567] Gate SM103 graph test on SM count - #19280

Merged
HuiGao-NV merged 1 commit into
NVIDIA:mainfrom
longcheng-nv:fix/pr19076-sm103-148sm-test
Sep 17, 2026
Merged

HuiGao-NV merged 1 commit into
NVIDIA:mainfrom
longcheng-nv:fix/pr19076-sm103-148sm-test

Conversation

@longcheng-nv

@longcheng-nv longcheng-nv commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Dev Engineer Review

The CUDA Graph test now requires SM103 and exactly 148 SMs before it runs the register route. This prevents unsupported 160-SM configurations from exercising the route. The change affects test gating only.

QA Engineer Review

The test covers the 148-SM SM103 register plan, warmup, and CUDA Graph replay for b=512, k=2048. Unsupported configurations skip. Validation passed for Python compilation and pre-commit; targeted pytest collection needs follow-up because local prebuilt bindings were incompatible. No integration test-list entry was found. Coverage verdict: needs follow-up.

Per-File QA Perspective

  • tests/unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py: Verifies register-route selection and replay on the supported 148-SM SM103 topology. No matching CI or manual-QA list entry was found.

Description

The SM103 b=512, k=2048 register route added by #19076 is intentionally limited to devices with 148 SMs. The corresponding CUDA Graph test only checked the SM architecture, so a 160-SM GB300 runner selected the baseline main plan and failed the test's reg assertion.

Skip this hardware-specific CUDA Graph test unless the SM103 device has 148 SMs. The static route test continues to verify that the 160-SM topology retains the baseline route.

Test Coverage

  • python3 -m py_compile tests/unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py
  • python3 -m pre_commit run --files tests/unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py
  • Targeted pytest collection was attempted, but the local prebuilt bindings are incompatible with the current source (global_steady_clock_now import mismatch).

PR Checklist

  • Reviewed the checklist in the PR template; this test-only change has no API, dependency, documentation, CODEOWNERS, or architecture-diagram impact.

Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>
@longcheng-nv
longcheng-nv requested a review from a team as a code owner September 16, 2026 12:35
@longcheng-nv longcheng-nv changed the title [None][test] Gate SM103 graph test on SM count [None][fix] Gate SM103 graph test on SM count Sep 16, 2026
@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 150881c0-1f13-4a56-88a8-b5e85f3a8895

📥 Commits

Reviewing files that changed from the base of the PR and between 7b1bedb and 357a303.

📒 Files selected for processing (1)
  • tests/unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The SM103 CUDA-graph test now targets the 148-SM register plan. It skips SM103 devices with a different multiprocessor count before validating register planning and replay results.

Changes

SM103 CUDA-graph test

Layer / File(s) Summary
Register plan device guard
tests/unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py
The test description now specifies the 148-SM SM103 configuration. A device-property guard skips SM103 devices whose multiprocessor count is not 148.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Suggested reviewers: bowenfu

Merge Risk: ⚪ Minimal · up to 357a3

The test is restricted to the intended 148-SM SM103 configuration, with other configurations skipped. No concrete merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the fix and states the main change: gating the SM103 graph test on SM count. It is concise and directly related to the changes.
Description check ✅ Passed The description explains the issue, the hardware-specific solution, test coverage, and checklist impact. It also documents why targeted pytest execution could not run.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #73844 [ run ] triggered by Bot. Commit: 357a303 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #73844 [ run ] completed with state SUCCESS. Commit: 357a303
/LLM/main/L0_MergeRequest_PR pipeline #60703 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, or
  • Wait for the PR to be fully approved — the label is added automatically once approval is complete.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@github-actions

Copy link
Copy Markdown

Removed the "ci: full pre-merge approved" label because @longcheng-nv could not be verified as an active member of NVIDIA/trt-llm-ci-approvers. Ask a member of that team to apply it.

@github-actions

Copy link
Copy Markdown

Removed the "ci: full pre-merge approved" label because @longcheng-nv could not be verified as an active member of NVIDIA/trt-llm-ci-approvers. Ask a member of that team to apply it.

@longcheng-nv longcheng-nv changed the title [None][fix] Gate SM103 graph test on SM count [https://nvbugs/6786555][fix] [https://nvbugs/6786567] Gate SM103 graph test on SM count Sep 17, 2026
@longcheng-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --reuse-test 60703

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #73941 [ run ] triggered by Bot. Commit: 357a303 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #73941 [ run ] completed with state SUCCESS. Commit: 357a303
/LLM/main/L0_MergeRequest_PR pipeline #60798 completed with status: 'SUCCESS'

CI Report

Link to invocation

@HuiGao-NV HuiGao-NV left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HuiGao-NV

Copy link
Copy Markdown
Collaborator

Please unwaive test case unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py::test_selfsampling_sm103_b512_k2048_cuda_graph.

@HuiGao-NV
HuiGao-NV merged commit 9a08c65 into NVIDIA:main Sep 17, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants