[https://nvbugs/6562942][fix] Cherry-pick #19815: Load Qwen3.5 packed INT4 checkpoints by pranav-nvidia · Pull Request #20171 · NVIDIA/TensorRT-LLM

pranav-nvidia · 2026-10-12T20:56:06Z

Dev Engineer Review

The working-tree diff contains only formatting changes in modeling_utils.py and linear.py. It changes import wrapping and line breaks, with no observable behavior or API changes. The diff does not include the Qwen3.5 INT4 mapper, shared concatenation helper, or related tests described in the supplied PR objectives.

QA Engineer Review

No test changes.

Per-File QA Perspective

tensorrt_llm/_torch/models/modeling_utils.py: Import formatting changed. No behavior change requires QA verification.
tensorrt_llm/_torch/modules/linear.py: Import formatting changed. No behavior change requires QA verification.

Description

Forward-port of #19815 (merged to release/1.3) to main, cherry-picked from 4450bbc. Qwen3.5 INT4 checkpoints pack two output channels per byte, so projection fusion compared the physical rows against the unpacked dimensions and failed with "expected 10240, got 5120". The mapper now fuses packed INT4 projections in tensor-parallel rank order, the Qwen4 rank-concatenation helper moves to modeling_utils and is shared by both mappers, and the W4A16 AWQ loader accepts checkpoints that carry no pre_quant_scale. Retained pre-scales are supported only when every component of a fused projection provides the same vector.

The patch applies unchanged; only tests/integration/test_lists/test-db/l0_b200.yml needed a manual merge to keep the entries added on both sides.

Test Coverage

The same tests as the release PR: tests/unittest/_torch/models/checkpoints/hf/test_qwen3_5_weight_mapper.py, tests/unittest/_torch/modeling/test_qwen4_exp_support.py, tests/unittest/_torch/modules/test_linear_uneven_tp.py::TestW4A16AWQMLP::test_optional_pre_quant_scale, and tests/unittest/_torch/models/test_qwen3_5_int4_loading.py, the last two on the B200 pre-merge list. All pass on release/1.3 CI since the merge. End-to-end validation was a GB200 TP1 load and generation run of a ModelOpt Qwen3.5-27B int4_blockwise_weight_only checkpoint on the release code; it was not repeated on main.

PR Checklist

Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Forward-port of NVIDIA#19815 from release/1.3. (cherry picked from commit 4450bbc) Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>

coderabbitai · 2026-10-12T22:04:31Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA/TensorRT-LLM/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d31d0745-9578-436e-b5ed-4788ec24b645

📥 Commits

Reviewing files that changed from the base of the PR and between 8fb3037 and ad903d8.

📒 Files selected for processing (9)

tensorrt_llm/_torch/models/checkpoints/hf/qwen3_5_weight_mapper.py
tensorrt_llm/_torch/models/checkpoints/hf/qwen4_exp_weight_mapper.py
tensorrt_llm/_torch/models/modeling_utils.py
tensorrt_llm/_torch/modules/linear.py
tests/integration/test_lists/test-db/l0_b200.yml
tests/unittest/_torch/modeling/test_qwen4_exp_support.py
tests/unittest/_torch/models/checkpoints/hf/test_qwen3_5_weight_mapper.py
tests/unittest/_torch/models/test_qwen3_5_int4_loading.py
tests/unittest/_torch/modules/test_linear_uneven_tp.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Walkthrough

The changes add shared tensor-parallel rank-wise projection packing, apply it to Qwen4-Exp and Qwen3.5 mapping, and support dense-layout packed-INT4 AWQ weights for Qwen3.5. AWQ loading now accepts an absent pre-quantization scale.

Changes

Rank-wise projection mapping and AWQ loading

Layer / File(s)	Summary
Shared rank-wise concatenation `tensorrt_llm/_torch/models/modeling_utils.py`, `tensorrt_llm/_torch/models/checkpoints/hf/qwen4_exp_weight_mapper.py`, `tests/unittest/_torch/modeling/test_qwen4_exp_support.py`	Adds a validated helper that concatenates projection components in tensor-parallel rank order. The Qwen4-Exp mapper and its test use the shared helper instead of the removed local implementation.
Qwen3.5 dense-layout INT4 mapping `tensorrt_llm/_torch/models/checkpoints/hf/qwen3_5_weight_mapper.py`, `tests/unittest/_torch/models/checkpoints/hf/test_qwen3_5_weight_mapper.py`	Adds packed-INT4 storage and dimension checks, dense rank-wise packing for fused projections, and fused-scale validation. Tests cover combined and split inputs, tensor-parallel layouts, data parallelism, and invalid inputs.
Optional AWQ activation scales `tensorrt_llm/_torch/modules/linear.py`, `tests/unittest/_torch/models/test_qwen3_5_int4_loading.py`, `tests/unittest/_torch/modules/test_linear_uneven_tp.py`, `tests/integration/test_lists/test-db/l0_b200.yml`	The AWQ linear method loads a sharded `pre_quant_scale` when provided and sets it to `None` when absent. Tests check scale handling and Qwen3.5 INT4 projection outputs; the B200 test list includes both test cases.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant HFCheckpoint
  participant Qwen3_5MoeHfWeightMapper
  participant concatenate_weights_by_tp_rank
  participant W4A16_AWQ_LinearMethod
  participant GDNProjections
  HFCheckpoint->>Qwen3_5MoeHfWeightMapper: provide projection weights
  Qwen3_5MoeHfWeightMapper->>concatenate_weights_by_tp_rank: pack dense-layout components by rank
  concatenate_weights_by_tp_rank-->>Qwen3_5MoeHfWeightMapper: return rank-ordered components
  Qwen3_5MoeHfWeightMapper->>W4A16_AWQ_LinearMethod: provide fused weights and optional pre_quant_scale
  W4A16_AWQ_LinearMethod->>GDNProjections: load projection weights

Suggested reviewers: juney-nvidia

Merge Risk: ⚪ Minimal · up to ad903

No identified issue blocks merging; the attention-DP projection layout matches the loader.

Pre-merge checks · 4 · 1

Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 8 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Title check		The title clearly identifies the fix and the main change: loading Qwen3.5 packed INT4 checkpoints. The cherry-pick context is relevant.
Description check		The description includes the required Description, Test Coverage, and PR Checklist sections. It explains the problem, solution, test coverage, and the limitation that end-to-end validation was not rep…

Full details: Docstring Coverage

Explanation

Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 8 files. (1 skipped: 1 unsupported.)

Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

pranav-nvidia · 2026-10-12T22:38:35Z

/bot run

tensorrt-cicd · 2026-10-12T22:45:44Z

PR_Github #77350 [ run ] triggered by Bot. Commit: ad903d8 Link to invocation

tensorrt-cicd · 2026-10-13T01:27:42Z

PR_Github #77350 [ run ] completed with state SUCCESS. Commit: ad903d8
/LLM/main/L0_MergeRequest_PR pipeline #63766 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

mikeiovine

Stamp on behalf of runtime devs, delegating proper review to @NVIDIA/trt-llm-models-devs

pranav-nvidia · 2026-10-13T16:35:15Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-13T16:42:09Z

PR_Github #77559 [ run ] triggered by Bot. Commit: ad903d8 Link to invocation

tensorrt-cicd · 2026-10-13T19:39:44Z

PR_Github #77559 [ run ] completed with state SUCCESS. Commit: ad903d8
/LLM/main/L0_MergeRequest_PR pipeline #63940 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[https://nvbugs/6562942][fix] Load Qwen3.5 packed INT4 checkpoints

ad903d8

Forward-port of NVIDIA#19815 from release/1.3. (cherry picked from commit 4450bbc) Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>

github-actions Bot assigned pranav-nvidia Oct 12, 2026

pranav-nvidia marked this pull request as ready for review October 12, 2026 21:59

pranav-nvidia requested review from a team as code owners October 12, 2026 21:59

pranav-nvidia requested review from BowenFu, QiJune, asfiyab-nvidia, hanjingtian, moraxu, tongyuantongyu, weiminwang-nv and yechank-nvidia October 12, 2026 21:59

pranav-nvidia changed the title ~~[https://nvbugs/6562942][fix] Load Qwen3.5 packed INT4 checkpoints~~ [https://nvbugs/6562942][fix] Cherry-pick #19815: Load Qwen3.5 packed INT4 checkpoints Oct 12, 2026

mikeiovine approved these changes Oct 13, 2026

View reviewed changes

pranav-nvidia closed this Oct 13, 2026

SimengLiu-nv mentioned this pull request Oct 22, 2026

[None][chore] Mass merge release/1.3 branch back to main (batch 2) #20843

Merged

1 task done

Navigation Menu

Repository navigation

[https://nvbugs/6562942][fix] Cherry-pick #19815: Load Qwen3.5 packed INT4 checkpoints - #20171

[https://nvbugs/6562942][fix] Cherry-pick #19815: Load Qwen3.5 packed INT4 checkpoints#20171
pranav-nvidia wants to merge 1 commit into
NVIDIA:mainfrom
pranav-nvidia:fix/nvbug6562942-main-forward-port

pranav-nvidia commented Oct 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

pranav-nvidia commented Oct 12, 2026

Uh oh!

tensorrt-cicd commented Oct 12, 2026

Uh oh!

tensorrt-cicd commented Oct 13, 2026

Uh oh!

mikeiovine left a comment

Uh oh!

pranav-nvidia commented Oct 13, 2026

Uh oh!

tensorrt-cicd commented Oct 13, 2026

Uh oh!

tensorrt-cicd commented Oct 13, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

pranav-nvidia commented Oct 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

coderabbitai Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

pranav-nvidia commented Oct 12, 2026

Uh oh!

tensorrt-cicd commented Oct 12, 2026

Uh oh!

tensorrt-cicd commented Oct 13, 2026

Uh oh!

mikeiovine left a comment

Choose a reason for hiding this comment

Uh oh!

pranav-nvidia commented Oct 13, 2026

Uh oh!

tensorrt-cicd commented Oct 13, 2026

Uh oh!

tensorrt-cicd commented Oct 13, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pranav-nvidia commented Oct 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 12, 2026 •

edited

Loading