[TRTLLM-12288][feat] revert all the changes of the PR#14775 by JadoTu · Pull Request #15772 · NVIDIA/TensorRT-LLM

JadoTu · 2026-06-30T08:49:08Z

Summary by CodeRabbit

Bug Fixes
- Simplified NVFP4 handling across model, linear, and MoE paths to use the standard execution flow.
- Removed platform-specific special cases that could affect behavior on newer hardware.
Tests
- Updated integration coverage to remove the deprecated Hopper-only NVFP4 test case.

Description

This PR is to revert the changes of #14775 because this feature has the better perf with #13476.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

coderabbitai · 2026-06-30T09:01:09Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e69b57bc-9557-409e-aa93-9b98d43b2b51

📥 Commits

Reviewing files that changed from the base of the PR and between 3689eb2 and 6c3a149.

📒 Files selected for processing (7)

tensorrt_llm/_torch/models/modeling_nemotron_h.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/quantization.py
tensorrt_llm/_torch/modules/fused_moe/triton_dequant_nvfp4.py
tensorrt_llm/_torch/modules/linear.py
tests/integration/defs/accuracy/test_llm_api_pytorch.py
tests/integration/test_lists/test-db/l0_dgx_h100.yml

💤 Files with no reviewable changes (6)

tensorrt_llm/_torch/modules/fused_moe/triton_dequant_nvfp4.py
tests/integration/test_lists/test-db/l0_dgx_h100.yml
tensorrt_llm/_torch/modules/linear.py
tests/integration/defs/accuracy/test_llm_api_pytorch.py
tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py
tensorrt_llm/_torch/modules/fused_moe/quantization.py

📝 Walkthrough

Walkthrough

Removes the SM<100 (Hopper) NVFP4→W4A16 fallback path: deletes W4A16NVFP4LinearMethod, W4A16NVFP4CutlassFusedMoEMethod, the triton_dequant_nvfp4 file, CutlassFusedMoE._run_moe_w4a16_nvfp4, and Nemotron-H Hopper patching helpers, along with the corresponding integration test.

Changes

W4A16 NVFP4 Hopper Fallback Removal

Layer / File(s)	Summary
W4A16NVFP4 quantization method classes removed `tensorrt_llm/_torch/modules/linear.py`, `tensorrt_llm/_torch/modules/fused_moe/quantization.py`	`W4A16NVFP4LinearMethod` (with `transform_weights`/`apply`) and `W4A16NVFP4CutlassFusedMoEMethod` (with FP8 scale unswizzling and Triton dequant) are deleted; the Triton dequant file `triton_dequant_nvfp4.py` they depended on is removed entirely.
CutlassFusedMoE W4A16 routing and fallback removed `tensorrt_llm/_torch/modules/fused_moe/fused_moe_cutlass.py`	`quantize_input` early-return for `W4A16NVFP4CutlassFusedMoEMethod`, `run_moe` SM<100 dispatch branch, and the entire `_run_moe_w4a16_nvfp4` bf16-workspace fallback method are deleted.
Nemotron-H Hopper patching and SM-version gating removed `tensorrt_llm/_torch/models/modeling_nemotron_h.py`	Module-level helpers `_force_moe_backend_for_w4a16_on_hopper` and `_use_w4a16_for_nvfp4_on_hopper` deleted; `NemotronHForCausalLM.__init__` calls `super().__init__` directly; `__post_init__` override removed; `NemotronHLayer.__init__` NVFP4 detection drops SM<100 gating; unused imports removed.
Hopper W4A16 test removed `tests/integration/defs/accuracy/test_llm_api_pytorch.py`, `tests/integration/test_lists/test-db/l0_dgx_h100.yml`	`test_nvfp4_4gpus_hopper_w4a16` method removed from `TestNemotronV3Super` and its entry removed from the H100 4-GPU pre-merge test list.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description explains the revert, but it omits the required Test Coverage section and the template's summary block.	Add a Test Coverage section and include the required summary/template fields so the PR description is complete.
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title matches the required ticket/type format and clearly states that this PR reverts PR `#14775`.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

…VIDIA#14775)" Signed-off-by: jiant <107457950+JadoTu@users.noreply.github.com>

JadoTu · 2026-06-30T10:01:53Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-30T10:07:29Z

PR_Github #56593 [ run ] triggered by Bot. Commit: 1576a3f Link to invocation

tensorrt-cicd · 2026-06-30T21:44:34Z

PR_Github #56593 [ run ] completed with state SUCCESS. Commit: 1576a3f
/LLM/main/L0_MergeRequest_PR pipeline #45420 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-01T09:03:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-01T09:10:24Z

PR_Github #56893 [ run ] triggered by Bot. Commit: b093bbc Link to invocation

tensorrt-cicd · 2026-07-02T00:17:43Z

PR_Github #56893 [ run ] completed with state FAILURE. Commit: b093bbc
/LLM/main/L0_MergeRequest_PR pipeline #45704 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-03T02:35:15Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-03T02:42:53Z

PR_Github #57319 [ run ] triggered by Bot. Commit: 40e0fbd Link to invocation

tensorrt-cicd · 2026-07-03T03:42:46Z

PR_Github #57319 [ run ] completed with state FAILURE. Commit: 40e0fbd
/LLM/main/L0_MergeRequest_PR pipeline #46080 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-07T02:02:55Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-07T02:08:44Z

PR_Github #57887 [ run ] triggered by Bot. Commit: 354577d Link to invocation

tensorrt-cicd · 2026-07-07T11:46:41Z

PR_Github #57887 [ run ] completed with state SUCCESS. Commit: 354577d
/LLM/main/L0_MergeRequest_PR pipeline #46584 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Jian Tu <107457950+JadoTu@users.noreply.github.com>

JadoTu · 2026-07-17T04:14:23Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-17T04:21:11Z

PR_Github #59869 [ run ] triggered by Bot. Commit: 9d33815 Link to invocation

tensorrt-cicd · 2026-07-17T08:00:07Z

PR_Github #59869 [ run ] completed with state SUCCESS. Commit: 9d33815
/LLM/main/L0_MergeRequest_PR pipeline #48275 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-18T13:14:14Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-18T13:20:48Z

PR_Github #60103 [ run ] triggered by Bot. Commit: 2f93175 Link to invocation

tensorrt-cicd · 2026-07-18T17:19:39Z

PR_Github #60103 [ run ] completed with state FAILURE. Commit: 2f93175
/LLM/main/L0_MergeRequest_PR pipeline #48490 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-18T17:23:49Z

/bot run

tensorrt-cicd · 2026-07-18T17:29:16Z

PR_Github #60117 [ run ] triggered by Bot. Commit: 2f93175 Link to invocation

tensorrt-cicd · 2026-07-18T21:22:34Z

PR_Github #60117 [ run ] completed with state FAILURE. Commit: 2f93175
/LLM/main/L0_MergeRequest_PR pipeline #48501 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-19T06:51:54Z

/bot run

tensorrt-cicd · 2026-07-19T06:57:27Z

PR_Github #60149 [ run ] triggered by Bot. Commit: 2f93175 Link to invocation

tensorrt-cicd · 2026-07-19T09:52:16Z

PR_Github #60149 [ run ] completed with state FAILURE. Commit: 2f93175
/LLM/main/L0_MergeRequest_PR pipeline #48527 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-19T18:30:58Z

/bot run

tensorrt-cicd · 2026-07-19T18:37:12Z

PR_Github #60190 [ run ] triggered by Bot. Commit: 2f93175 Link to invocation

tensorrt-cicd · 2026-07-19T21:33:58Z

PR_Github #60190 [ run ] completed with state FAILURE. Commit: 2f93175
/LLM/main/L0_MergeRequest_PR pipeline #48561 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JadoTu · 2026-07-20T03:17:47Z

/bot run

tensorrt-cicd · 2026-07-20T03:24:38Z

PR_Github #60245 [ run ] triggered by Bot. Commit: 2f93175 Link to invocation

tensorrt-cicd · 2026-07-20T04:15:32Z

PR_Github #60245 [ run ] completed with state SUCCESS. Commit: 2f93175
/LLM/main/L0_MergeRequest_PR pipeline #48610 completed with status: 'SUCCESS'

CI Report

Link to invocation

fredricz-20070104

Approved

BowenFu

LGTM. Clean, symmetric revert of #14775: every W4A16/Hopper NVFP4-dequant helper import, definition, and call-site is removed (incl. the whole triton_dequant_nvfp4.py and the Hopper SM-gating in modeling_nemotron_h.py), with no orphaned references. Native NVFP4 dispatch (NVFP4LinearMethod/NVFP4CutlassFusedMoEMethod) is unchanged — restores the pre-#14775 known-good path (superseded by #13476).

ZhanruiSunCh

LGTM for infra part.

brnguyen2

Approving — the comments below are optional touch-ups, not blockers.

Revert is complete — I grepped the tree for every symbol #14775 introduced (W4A16NVFP4LinearMethod, W4A16NVFP4CutlassFusedMoEMethod, triton_dequant_nvfp4, _use_w4a16_for_nvfp4_on_hopper, _force_moe_backend_for_w4a16_on_hopper) and nothing dangles in source. One stale leftover: tests/integration/defs/.test_durations:436 still lists test_nvfp4_4gpus_hopper_w4a16; auto-generated, so drop it whenever that file is next regenerated.

The one thing the description should say out loud: #13476's Marlin path is not a drop-in replacement from the user's point of view. is_nvfp4_marlin_enabled() (tensorrt_llm/_torch/utils.py:108) requires marlin to be listed explicitly in nvfp4_gemm_config.allowed_backends, and the default is ['cutlass', 'cublaslt', 'cuda_core']; the MoE side additionally needs moe_config.backend="MARLIN" (create_moe.py:66). The code being reverted here engaged automatically on SM90. So a user who runs an NVFP4 Nemotron-H checkpoint on H100 with a default config gets a load failure after this merges, where it worked before. Two asks:

Note the required config in the PR description so the behavior change is on record.
Since Hopper NVFP4 Nemotron-H is now a documented-config-only path, add it to the docs — either the Nemotron deployment guide or docs/source/models/supported-models.md — with the two settings spelled out. Alternatively, if Marlin is meant to be the Hopper default now, add marlin to Nvfp4GemmConfig.allowed_backends defaults (it is already gated on SM90 + kernel availability, so it's inert elsewhere) and pick MARLIN for MoE on SM90+NVFP4. That would make this a true no-user-impact revert.

brnguyen2 · 2026-08-11T23:58:04Z

-            task.evaluate(llm,
-                          extra_evaluator_kwargs=self.EXTRA_EVALUATOR_KWARGS)
-
    @skip_pre_hopper


The removed test_nvfp4_4gpus_hopper_w4a16 was the only Hopper NVFP4 Nemotron-H case exercising MTP (max_draft_len=4) together with CudaGraphConfig(enable_padding=True). test_nvfp4_marlin_multi_gpus (line 6907) is a plain MMLU run with no speculative config and no CUDA-graph config, so that combination now has no SM90 coverage at all. Add a spec-decode + cuda-graph parametrization to the Marlin test rather than letting the dimension disappear with the reverted feature.

JadoTu requested review from a team as code owners June 30, 2026 08:49

JadoTu requested review from leslie-fang25, tomeras91 and yechank-nvidia June 30, 2026 08:49

github-actions Bot assigned JadoTu Jun 30, 2026

JadoTu force-pushed the revert_pr_14775 branch 2 times, most recently from e20d093 to d3f98f5 Compare June 30, 2026 09:21

Revert "[TRTLLM-12288][feat] Support Nemotron-H nvfp4 ckpt on Hopper (N…

1576a3f

…VIDIA#14775)" Signed-off-by: jiant <107457950+JadoTu@users.noreply.github.com>

JadoTu force-pushed the revert_pr_14775 branch from d3f98f5 to 1576a3f Compare June 30, 2026 09:59

Merge branch 'main' into revert_pr_14775

b093bbc

xinhe-nv approved these changes Jul 3, 2026

View reviewed changes

Merge branch 'main' into revert_pr_14775

40e0fbd

Merge branch 'main' into revert_pr_14775

354577d

Merge branch 'main' into revert_pr_14775

0965330

Signed-off-by: Jian Tu <107457950+JadoTu@users.noreply.github.com>

JadoTu requested review from ZhanruiSunCh, tburt-nv and yufeiwu-nv July 17, 2026 04:14

yufeiwu-nv removed their request for review July 17, 2026 23:31

Merge branch 'main' into revert_pr_14775

2f93175

JadoTu disabled auto-merge July 20, 2026 05:19

mikeiovine removed the request for review from pcastonguay July 20, 2026 15:48

fredricz-20070104 approved these changes Jul 21, 2026

View reviewed changes

BowenFu approved these changes Jul 21, 2026

View reviewed changes

ZhanruiSunCh approved these changes Jul 28, 2026

View reviewed changes

brnguyen2 approved these changes Aug 11, 2026

View reviewed changes

mikeiovine approved these changes Aug 26, 2026

View reviewed changes

Uh oh!

Conversation

JadoTu commented Jun 30, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

PR Checklist

GitHub Bot Help

Uh oh!

coderabbitai Bot commented Jun 30, 2026

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (2 warnings)

Uh oh!

JadoTu commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

tensorrt-cicd commented Jun 30, 2026

Uh oh!

JadoTu commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 2, 2026

Uh oh!

JadoTu commented Jul 3, 2026

Uh oh!

tensorrt-cicd commented Jul 3, 2026

Uh oh!

tensorrt-cicd commented Jul 3, 2026

Uh oh!

JadoTu commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

JadoTu commented Jul 17, 2026

Uh oh!

tensorrt-cicd commented Jul 17, 2026

Uh oh!

tensorrt-cicd commented Jul 17, 2026

Uh oh!

JadoTu commented Jul 18, 2026

Uh oh!

tensorrt-cicd commented Jul 18, 2026

Uh oh!

tensorrt-cicd commented Jul 18, 2026

Uh oh!

JadoTu commented Jul 18, 2026

Uh oh!

tensorrt-cicd commented Jul 18, 2026

Uh oh!

tensorrt-cicd commented Jul 18, 2026

Uh oh!

JadoTu commented Jul 19, 2026

Uh oh!

tensorrt-cicd commented Jul 19, 2026

Uh oh!

tensorrt-cicd commented Jul 19, 2026

Uh oh!

JadoTu commented Jul 19, 2026

Uh oh!

tensorrt-cicd commented Jul 19, 2026

Uh oh!

tensorrt-cicd commented Jul 19, 2026

Uh oh!

JadoTu commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

fredricz-20070104 left a comment

Choose a reason for hiding this comment

Uh oh!

BowenFu left a comment

Choose a reason for hiding this comment

JadoTu commented Jun 30, 2026 •

edited by coderabbitai Bot

Loading