[https://nvbugs/6181383][fix] Build inner text/vision/audio sub-configs as empty PretrainedConfig() then setat by tensorrt-cicd · Pull Request #14399 · NVIDIA/TensorRT-LLM

tensorrt-cicd · 2026-05-21T08:22:36Z

Summary

Root cause: transformers 5.5.x's strict-dataclass PreTrainedConfig drops unknown kwargs (max_position_embeddings) until after post_init, but rope standardization inside post_init dereferences them first and raises AttributeError.
Fix: Build inner text/vision/audio sub-configs as empty PretrainedConfig() then setattr each dict entry, sidestepping the rope path; also relax DeepseekV3 detect_shared_mtp_weights to getattr(...) since num_nextn_predict_layers is absent on mistral_large_3 configs and used to read as None.
Automated fix generated by repair-bot

Test plan

Verify fix on the same GPU type as the original failure
Check for regressions in related tests

Links

Bug: https://nvbugs/6181383

Summary by CodeRabbit

Bug Fixes
- Improved checkpoint loading robustness for Mistral model configurations, particularly for vision and audio variants, enhancing compatibility with various configuration formats.
- Enhanced DeepseekV3 model configuration handling to gracefully manage missing optional configuration fields during checkpoint loading, improving overall stability.

coderabbitai · 2026-05-21T08:27:36Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 02d3bfa4-843b-4fc3-bf19-206aa1dbf8f6

📥 Commits

Reviewing files that changed from the base of the PR and between b7c2303 and 348f636.

📒 Files selected for processing (2)

tensorrt_llm/_torch/models/checkpoints/mistral/config_loader.py
tensorrt_llm/_torch/models/modeling_deepseekv3.py

📝 Walkthrough

Walkthrough

The PR adds a helper function in Mistral config loading to construct PretrainedConfig instances via setattr instead of from_dict, accommodating Transformers' strict dataclass validation. This helper is applied to both vision and audio checkpoint remapping. Additionally, DeepseekV3 weight loader now safely handles missing num_nextn_predict_layers attributes with a getattr fallback.

Changes

Mistral PretrainedConfig Construction Helper

Layer / File(s)	Summary
Helper function and vision config remapping `tensorrt_llm/_torch/models/checkpoints/mistral/config_loader.py`	Introduces `_dict_to_base_pretrained_config(d)` that creates a `PretrainedConfig` by instantiating empty and assigning keys via `setattr`. Updates `_remap_mistral_vision_args` to use this helper for both `text_config` and `vision_config` instead of `from_dict`.
Audio config remapping `tensorrt_llm/_torch/models/checkpoints/mistral/config_loader.py`	Updates `_remap_mistral_audio_args` to construct `text_config` using the new helper function, preserving any attached `quantization_config`.

DeepseekV3 Safe Config Attribute Access

Layer / File(s)	Summary
Safe num_nextn_predict_layers handling `tensorrt_llm/_torch/models/modeling_deepseekv3.py`	Changes `ckpt_nextn` initialization in `DeepseekV3WeightLoader.detect_shared_mtp_weights` to use `getattr(self.config, 'num_nextn_predict_layers', None) or 0`, defaulting to 0 instead of raising an AttributeError when the field is missing.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

xinhe-nv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title correctly identifies the main change: building inner text/vision/audio sub-configs as empty PretrainedConfig() then setattr, addressing the root cause of the transformers 5.5.x strict-dataclass issue.
Description check	✅ Passed	The PR description includes the root cause, fix explanation, test plan with checkmarks, and relevant bug link, but is missing explicit sections for 'Description' and 'Test Coverage' headings as specified in the template structure.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

2ez4bz · 2026-06-04T22:11:27Z

/bot run

2ez4bz · 2026-06-04T22:16:09Z

/bot run

tensorrt-cicd · 2026-06-04T22:21:40Z

PR_Github #52170 [ run ] triggered by Bot. Commit: ee05437 Link to invocation

tensorrt-cicd · 2026-06-05T00:02:00Z

PR_Github #52170 [ run ] completed with state ABORTED. Commit: ee05437
/LLM/main/L0_MergeRequest_PR pipeline #41492 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-05T04:37:51Z

/bot run

tensorrt-cicd · 2026-06-05T04:43:10Z

PR_Github #52242 [ run ] triggered by Bot. Commit: ee05437 Link to invocation

tensorrt-cicd · 2026-06-05T05:37:21Z

PR_Github #52242 [ run ] completed with state SUCCESS. Commit: ee05437
/LLM/main/L0_MergeRequest_PR pipeline #41558 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-05T05:52:49Z

/bot run

tensorrt-cicd · 2026-06-05T05:58:36Z

PR_Github #52268 [ run ] triggered by Bot. Commit: ee05437 Link to invocation

tensorrt-cicd · 2026-06-05T07:05:46Z

PR_Github #52268 [ run ] completed with state SUCCESS. Commit: ee05437
/LLM/main/L0_MergeRequest_PR pipeline #41579 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

evezhier · 2026-06-05T08:02:13Z

/bot run --extra-stage "GB200-4_GPUs-PyTorch-Post-Merge-1"

tensorrt-cicd · 2026-06-05T08:08:58Z

PR_Github #52310 [ run ] triggered by Bot. Commit: 3ae5580 Link to invocation

tensorrt-cicd · 2026-06-05T11:46:05Z

PR_Github #52310 [ run ] completed with state SUCCESS. Commit: 3ae5580
/LLM/main/L0_MergeRequest_PR pipeline #41616 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-05T15:00:53Z

/bot run

tensorrt-cicd · 2026-06-05T15:06:42Z

PR_Github #52355 [ run ] triggered by Bot. Commit: 3ae5580 Link to invocation

tensorrt-cicd · 2026-06-05T16:31:04Z

PR_Github #52355 [ run ] completed with state SUCCESS. Commit: 3ae5580
/LLM/main/L0_MergeRequest_PR pipeline #41654 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-05T17:53:45Z

/bot run

tensorrt-cicd · 2026-06-05T18:00:08Z

PR_Github #52402 [ run ] triggered by Bot. Commit: 413e619 Link to invocation

tensorrt-cicd · 2026-06-05T19:19:48Z

PR_Github #52402 [ run ] completed with state SUCCESS. Commit: 413e619
/LLM/main/L0_MergeRequest_PR pipeline #41697 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-05T19:41:47Z

/bot run

tensorrt-cicd · 2026-06-05T19:47:54Z

PR_Github #52415 [ run ] triggered by Bot. Commit: 413e619 Link to invocation

tensorrt-cicd · 2026-06-05T22:23:12Z

PR_Github #52415 [ run ] completed with state FAILURE. Commit: 413e619
/LLM/main/L0_MergeRequest_PR pipeline #41710 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-06T03:53:00Z

/bot run

tensorrt-cicd · 2026-06-06T03:59:56Z

PR_Github #52472 [ run ] triggered by Bot. Commit: 413e619 Link to invocation

tensorrt-cicd · 2026-06-06T08:03:44Z

PR_Github #52472 [ run ] completed with state FAILURE. Commit: 413e619
/LLM/main/L0_MergeRequest_PR pipeline #41764 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

…5.5.x In transformers 5.5.x, the base PreTrainedConfig is a strict dataclass and unknown kwargs are no longer set as attributes during __init__. They are only assigned at the end of __post_init__, but the rope-standardization path inside __post_init__ runs before that and dereferences self.max_position_embeddings, which is not a declared dataclass field on the base class. The MistralLarge3 vision path passes both rope_scaling and rope_theta into the inner text PretrainedConfig, triggering the path and raising AttributeError. Build the inner text/vision/audio sub-configs by constructing an empty PretrainedConfig (no kwargs -> no rope path) and setattr-ing each entry afterwards, matching the previous behavior where unknown kwargs landed on the instance. Also harden DeepseekV3WeightLoader.detect_shared_mtp_weights to use getattr with a default of None for num_nextn_predict_layers; the new strict dataclass raises AttributeError when the attribute is absent on mistral_large_3 configs, instead of returning None as the prior PreTrainedConfig did. The 'or 0' fallback is preserved. Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>

Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>

2ez4bz · 2026-06-08T18:10:16Z

/bot run

tensorrt-cicd · 2026-06-08T18:20:33Z

PR_Github #52840 [ run ] triggered by Bot. Commit: f6056ab Link to invocation

tensorrt-cicd · 2026-06-08T21:47:32Z

PR_Github #52840 [ run ] completed with state SUCCESS. Commit: f6056ab
/LLM/main/L0_MergeRequest_PR pipeline #42092 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-09T00:49:35Z

/bot run

tensorrt-cicd · 2026-06-09T00:55:58Z

PR_Github #52892 [ run ] triggered by Bot. Commit: f6056ab Link to invocation

tensorrt-cicd · 2026-06-09T03:09:48Z

PR_Github #52892 [ run ] completed with state SUCCESS. Commit: f6056ab
/LLM/main/L0_MergeRequest_PR pipeline #42142 completed with status: 'SUCCESS'

CI Report

Link to invocation

…gs as empty PretrainedConfig() then setat (NVIDIA#14399) Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>

tensorrt-cicd requested review from a team as code owners May 21, 2026 08:22

tensorrt-cicd assigned evezhier May 21, 2026

tensorrt-cicd requested review from hlu1 and symphonylyh May 21, 2026 08:22

github-actions Bot assigned tensorrt-cicd May 21, 2026

tensorrt-cicd force-pushed the repair-bot-bug6181383 branch 3 times, most recently from cf2d1a9 to eda0f73 Compare May 28, 2026 08:27

tensorrt-cicd force-pushed the repair-bot-bug6181383 branch 4 times, most recently from 803cbe0 to b43e4db Compare June 2, 2026 23:22

2ez4bz enabled auto-merge (squash) June 4, 2026 22:11

2ez4bz approved these changes Jun 5, 2026

View reviewed changes

tensorrt-cicd force-pushed the repair-bot-bug6181383 branch from 413e619 to d154fed Compare June 7, 2026 02:39

tensorrt-cicd added 2 commits June 8, 2026 00:08

[nvbugs/6181383][chore] Remove stale waiver after fix

f6056ab

Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>

tensorrt-cicd force-pushed the repair-bot-bug6181383 branch from d154fed to f6056ab Compare June 8, 2026 07:13

2ez4bz merged commit a33dec7 into NVIDIA:main Jun 9, 2026
7 checks passed

Uh oh!

Conversation

tensorrt-cicd commented May 21, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Links

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 21, 2026

Walkthrough

Changes

Estimated code review effort

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

2ez4bz commented Jun 4, 2026

Uh oh!

2ez4bz commented Jun 4, 2026

Uh oh!

tensorrt-cicd commented Jun 4, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

2ez4bz commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

2ez4bz commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

evezhier commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

2ez4bz commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

2ez4bz commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

2ez4bz commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

2ez4bz commented Jun 6, 2026

Uh oh!

tensorrt-cicd commented Jun 6, 2026

Uh oh!

tensorrt-cicd commented Jun 6, 2026

Uh oh!

2ez4bz commented Jun 8, 2026

Uh oh!

tensorrt-cicd commented Jun 8, 2026

Uh oh!

tensorrt-cicd commented Jun 8, 2026

Uh oh!

2ez4bz commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

tensorrt-cicd commented May 21, 2026 •

edited by coderabbitai Bot

Loading