Skip to content

[https://nvbugs/6665906][fix] Restore Mistral Large 3 multimodal inference - #19788

Merged
pranav-nvidia merged 8 commits into
NVIDIA:release/1.3from
pranav-nvidia:fix/nvbug6665906-release13
Oct 9, 2026
Merged

pranav-nvidia merged 8 commits into
NVIDIA:release/1.3from
pranav-nvidia:fix/nvbug6665906-release13

Conversation

@pranav-nvidia

@pranav-nvidia pranav-nvidia commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Description

Restore Mistral Large 3 multimodal inference on release/1.3. Share the Pixtral encoder scheduling contract between native and Hugging Face input processors, preserve each native image's processed geometry when batching unequal sizes, and retain neutral behavior for text-only checkpoints.

Provide head-size-104 packed-QKV padding-mask FMHA kernels on SM90, SM100, and SM120. The SM100 dispatch exception is limited to matching FP16/BF16 inputs, Q/K/V head size 104, and FULL/CAUSAL attention. SM90 and SM120 enable FP16, BF16, and forced-FP32 FP16 padding variants selected by their existing dispatcher: warp-specialized on SM90 and non-tiled on SM120. FP8 and other SM100 mask/layout combinations retain their existing routing.

This replaces closed, unmerged #19507 and removes the original four-GPU MMMU waiver. The main counterpart remains outstanding. GB200-4_GPUs-PyTorch-1 must pass before merge.

Test Coverage

  • Rebuilt the merged release source for 90-real;100-real;120-real. On H100, B200, and RTX PRO 6000 Blackwell, fmhaHead104Test passes all six cases, covering FP16/BF16 FULL/CAUSAL attention and fixed/runtime forced-FP32 flags with short and long ragged sequences. The tests call the fused dispatcher directly and compare against an independent FP32 reference. H100 uses the existing kernel suite; B200, GB202, and RTX PRO 6000 have explicit native-test entries.
  • pytest tests/unittest/_torch/attention/test_pixtral_head104_attention.py: two cases pass on each GPU, including the workspace bound below 64 MiB. Additional native probes pass 24 cases on H100 and 30 each on B200 and RTX PRO 6000, with ragged lengths [31, 7], [257, 65], and [2048, 32]. Baseline H100/SM120 forced-FP32 FULL requests reproduce missing kernels; the final build passes.
  • pytest tests/unittest/_torch/modeling/test_modeling_mistral.py tests/unittest/_torch/attention/test_fmha_v2_head_size_coverage.py -m cpu_only: 16 passed with CUDA unavailable. On RTX PRO 6000, the remaining 31 Mistral cases and test_modeling_pixtral.py -k test_pixtral_vision_model_vs_hf also pass. All selected final tests pass without skips; local style/DCO hooks pass.
  • Retained four-B200 validation: accuracy/test_llm_api_pytorch_multimodal.py::TestMistralLarge3_675B::test_nvfp4_4gpus[latency_moe_trtllm] passes 900 MMMU samples at 53.111%, above the unchanged 43.123% threshold, with the original 16,384-token budget. This result does not replace the required GB200 CI gate.

Validation uses Python 3.12, PyTorch 2.14.0a0 (26.08), CUDA 13.4, and mistral-common==1.12.0 with OpenCV.

PR Checklist

  • Please check this after reviewing the above items as appropriate for this PR.

Dev Engineer Review

  • Restores shared Pixtral encoder scheduling between native and Hugging Face Mistral processors.
  • Preserves per-image geometry when batching unequal image sizes.
  • Keeps text-only checkpoints neutral and preserves the native text-only tokenization path.
  • Adds head-size-104 packed-QKV padding-mask FMHA selection for supported SM90, SM100, and SM120 configurations.
  • Adds dispatcher guards for matching FP16/BF16 types, padding or causal masks, and head/value size 104.
  • Removes the MMMU waiver. This expands required integration coverage.
  • Current CI evidence includes aborted and failed runs. Full multi-GPU validation requires follow-up.
  • Repository inspection shows only formatting changes in cpp/kernels/fmha_v2/setup.py; verify that the intended kernel-generation changes are present in the reviewed commit.

QA Engineer Review

  • Adds native Mistral image-size, scheduling, text-only, and dummy-input tests.
  • Adds CPU-only FMHA v2 generation coverage and CUDA Pixtral head-104 numerical and workspace tests.
  • Adds the fmhaHead104Test C++ gtest and its build and execution fixtures.
  • Removes the four-GPU MMMU waiver.
  • Adds FMHA head-104 coverage to l0_gb202.yml, l0_rtx_pro_6000.yml, and l0_b200.yml.
  • The reported CPU tests passed. Reported H100, RTX PRO 6000, and B200 tests passed, but later CI runs failed and the required four-GPU validation was not consistently executed.
  • Coverage verdict: needs follow-up.

Per-File QA Perspective

  • cpp/kernels/fmha_v2/setup.py: Changes FMHA kernel-generation eligibility and metadata formatting. Verify supported architecture, dtype, masking, warp-specialization, and tiling combinations.
  • cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp: Adds head-104 FMHA v2 dispatch conditions. Verify fallback behavior for unsupported dtypes, masks, architectures, and head sizes.
  • tensorrt_llm/_torch/models/modeling_mistral.py: Changes image batching, encoder scheduling, dummy-data behavior, and text-only processing. Verify native/Hugging Face parity and text-only regressions.
  • tests/unittest/_torch/modeling/test_modeling_mistral.py: Covers native geometry preservation, scheduling, and text-only behavior. It is a unit test and is not directly listed in test-db lists.
  • tests/unittest/_torch/attention/test_fmha_v2_head_size_coverage.py: Covers supported and excluded generated head-104 kernel variants. It is a unit test and is listed in l0_gb202.yml and l0_rtx_pro_6000.yml.
  • tests/unittest/_torch/attention/test_pixtral_head104_attention.py: Covers ragged FP16/BF16 numerical parity and workspace limits. It is listed in l0_gb202.yml and l0_rtx_pro_6000.yml.
  • cpp/tests/unit_tests/kernels/fmhaHead104Test.cpp: Covers packed-QKV head-104 attention with ragged lengths, masks, dtypes, and FP32 accumulation. It is built by the CMake target and executed through the C++ integration test.
  • tests/integration/defs/cpp/conftest.py: Builds the head-104 gtest for the requested CUDA architecture.
  • tests/integration/defs/cpp/test_unit_tests.py: Executes the head-104 gtest for SM100 and SM120. The test is listed through l0_b200.yml.
  • tests/integration/test_lists/waives.txt: Removes the Mistral Large 3 four-GPU waiver. Verify that the unwaived test has suitable CI capacity.
  • tests/integration/test_lists/test-db/l0_gb202.yml: Adds FMHA head-104 unit and Pixtral attention coverage.
  • tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml: Adds the same FMHA and Pixtral coverage for RTX PRO 6000.
  • tests/integration/test_lists/test-db/l0_b200.yml: Adds the SM100 C++ head-104 unit test for B200 coverage.

…rence on Blackwell

Share Pixtral encoder scheduling across native and HF processors.
Preserve per-image geometry and use FMHA v2 for SM100 packed head-104 QKV.
Cover processor behavior, kernel generation, and bounded attention workspace.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/TensorRT-LLM/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 7e381143-a83d-467f-9e56-ae88c03963f1
📥 Commits

Reviewing files that changed from the base of the PR and between 3ba9245 and 531a412.

📒 Files selected for processing (1)
  • tests/unittest/_torch/executor/test_pytorch_model_engine_warmup.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

FMHA kernel generation and dispatch now support selected packed-QKV head-size-104 cases. Mistral input processors share Pixtral encoder behavior, and native image batching retains individual image sizes. Tests cover these changes. The default integration test list no longer skips the specified Mistral Large 3 latency test.

Changes

SM100 FMHA head-size-104 support

Layer / File(s) Summary
Kernel generation and runtime selection
cpp/kernels/fmha_v2/setup.py, cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp
Kernel enumeration accepts head size 104 for specified SM100 cases. Padding-mask selection also accepts specified SM90 and SM120 cases. The dispatcher excludes matching-dtype packed-QKV head-size-104 cases from TRTLLM-GEN selection.
Kernel and attention coverage
tests/unittest/_torch/attention/test_fmha_v2_head_size_coverage.py, tests/unittest/_torch/attention/test_pixtral_head104_attention.py, tests/integration/test_lists/test-db/l0_gb202.yml, tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml
CPU-only tests check generated kernel specifications and padding-mask selection. A CUDA test checks attention output for FP16 and BF16. Two integration test lists include the CUDA test.

Mistral input processing

Layer / File(s) Summary
Image processing and encoder contract
tensorrt_llm/_torch/models/modeling_mistral.py
The common image processor returns tensors and retains each image’s size. A shared contract provides Pixtral metadata, token sizing, capacity, and dummy-data behavior. It gates encoder behavior on the presence of vision_config.
HF and native processor integration
tensorrt_llm/_torch/models/modeling_mistral.py, tests/unittest/_torch/modeling/test_modeling_mistral.py
Both input processors use the shared contract. Tests check image batching, encoder scheduling, and text-only native checkpoints.

Test-list and warmup fixture updates

Layer / File(s) Summary
Default integration test list
tests/integration/test_lists/waives.txt
The default-list skip for the specified Mistral Large 3 latency test was removed.
Warmup test fixture
tests/unittest/_torch/executor/test_pytorch_model_engine_warmup.py
The mock engine in the compiled MXFP8 warmup test now has a rank-0 _WarmupTimer.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 531a4

This incremental change supplies the mock timer needed by the compiled warmup path and establishes no new user-facing behavior or merge-blocking risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 47.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 51 functions across 11 files. (4 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the NVBugs ticket, fix type, and primary change: restoring Mistral Large 3 multimodal inference.
Description check ✅ Passed The description provides a clear problem statement, implementation details, extensive test coverage, environment information, and checklist confirmation. It is sufficiently complete for review.
Full details: Docstring Coverage

Explanation

Docstring coverage is 47.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 51 functions across 11 files. (4 skipped: 4 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

…erage

Cover representative frontend geometry and image batching cases.
Assert exact scheduling budgets and successful dummy image metadata.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76095 [ run ] triggered by Bot. Commit: ccdf467 Link to invocation

@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76116 [ run ] triggered by Bot. Commit: 0663cec Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76095 [ run ] completed with state ABORTED. Commit: ccdf467

Link to invocation

@pranav-nvidia
pranav-nvidia marked this pull request as ready for review October 2, 2026 21:55
@pranav-nvidia
pranav-nvidia requested review from a team as code owners October 2, 2026 21:55
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76116 [ run ] completed with state FAILURE. Commit: 0663cec
/LLM/release-1.3/L0_MergeRequest_PR pipeline #37 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
  • If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@brnguyen2 brnguyen2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — the comments below are optional touch-ups, not blockers.

Both halves of the fix check out against the code paths they depend on. The vision sub-model is constructed with quant_config=None and has no KV cache, so the new dataType == dataTypeKv clause in the dispatcher is always true for Pixtral and the head-104 packed-QKV case reliably lands on fmha v2. The _has_vision_encoder gate is also sound: config_loader.py only synthesizes vision_config when vision_encoder_args is present, so audio-only and text-only native checkpoints never reach the geometry helpers. The native processor change is a real correctness fix beyond the Blackwell issue: with return_tensors="pt", transformers' MistralCommonTokenizer np.stacks the images, which raised on unequal sizes and produced a single image_sizes row for N images.

Two non-blocking items:

  • Release-branch targeting. This is a fix for a bug filed against main, landing on release/1.3. Please state the status of the main counterpart (#19507 or its successor) in the description so the branches do not diverge on the processor refactor.
  • SM90/SM120 scope. See the inline note on setup.py: the padding-mask variant for head 104 is still pruned on Hopper and SM120, where the dispatcher already uses fmha v2. If that is a deliberate scoping decision, say so in the description; otherwise a follow-up would let Pixtral-Large avoid the unfused path there too.

Merge gate as stated in the description: the GB200-4_GPUs-PyTorch-1 stage must be green on this PR, since the waiver removal is only justified by that run.

Comment thread cpp/kernels/fmha_v2/setup.py Resolved
…r output

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Generate the FP16/BF16 padding variants selected by each architecture.
Exercise attention parity and workspace bounds on Hopper and RTX Blackwell.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
@pranav-nvidia
pranav-nvidia requested review from a team as code owners October 5, 2026 19:25
@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76777 [ run ] triggered by Bot. Commit: 531a412 Link to invocation

Limit the SM100 packed-QKV exception to FP16/BF16 FULL and CAUSAL attention.
Enable SM90/SM120 FP16 FP32-accumulation padding kernels.
Cover ragged numerical parity and both force flags in native CI tests.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
@pranav-nvidia
pranav-nvidia requested review from a team as code owners October 8, 2026 19:59
@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76784 [ run ] triggered by Bot. Commit: 8f22286 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76777 [ run ] completed with state ABORTED. Commit: 531a412

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76784 [ run ] completed with state ABORTED. Commit: 8f22286
/LLM/release-1.3/L0_MergeRequest_PR pipeline #105 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76830 [ run ] triggered by Bot. Commit: 8f22286 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76830 [ run ] completed with state FAILURE. Commit: 8f22286
/LLM/release-1.3/L0_MergeRequest_PR pipeline #110 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@mikeiovine mikeiovine left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stamp on behalf of runtime devs, delegating proper review to @NVIDIA/trt-llm-devs; please ping me if you think this is not accurate

@pranav-nvidia

Copy link
Copy Markdown
Contributor Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76997 [ run ] triggered by Bot. Commit: 8f22286 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76997 [ run ] completed with state SUCCESS. Commit: 8f22286
/LLM/release-1.3/L0_MergeRequest_PR pipeline #136 completed with status: 'SUCCESS'

CI Report

Link to invocation

@pranav-nvidia
pranav-nvidia merged commit 48c796c into NVIDIA:release/1.3 Oct 9, 2026
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Oct 26, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
NVIDIA#20391. The shared Pixtral encoder contract builds item metadata in
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 2, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 2, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 3, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 3, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
SimengLiu-nv pushed a commit to SimengLiu-nv/TensorRT-LLM that referenced this pull request Nov 3, 2026
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants