Enable Qwen3.5 TRT-RTX EP path with CUDA graph

[Yen-Shi Wang (yen-shi)]

This PR enables Qwen3.5 text-only INT4 QDQ export and TRT-RTX EP inference with CUDA graph/shared past-present buffers.

Structure

The branch is rebased on latest main and intentionally split into two commits:

1. Add Qwen3.5 text-only export support
   • Mirrors the overlapping Qwen3.5 text-only builder/model-type work from PR Add text-only mode support for Qwen 3.5 model builder #2157.
   • Keeps Qwen3.5 text-only ONNX input position_ids as [B, S] and expands inside the graph to [3, B, S] for mRoPE.
2. Enable Qwen3.5 TRT-RTX shared-buffer inference
   • Shares Qwen3.5 recurrent/conv state buffers when past_present_share_buffer is enabled, preserving stable input/output addresses for TRT-RTX graph replay.
   • Fixes QDQ SkipLayerNorm output_3 producer wiring.
   • Keeps mixed-precision quantization logic in the shared base k_quant_linear path rather than Qwen-local code.
   • Adds the canonical NvTensorRtRtx name to the example EP choices.

PR #2157 compatibility

This branch was compared against #2157 using git merge-tree. The same Qwen files are touched, but Git auto-merges them cleanly and the simulation produced no conflict markers.

If #2157 merges first, the first commit in this branch is the overlap and can be dropped/rebased away; the second commit contains the TRT-RTX-specific delta.

Validation

• Rebased onto latest upstream main (bf6cf3fe).
• Built CUDA Release wheel with CUDA 13.2:
  python build.py --use_cuda --cuda_home="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.2" --config Release --update --build --parallel --skip_tests --skip_examples
• Installed the rebuilt wheel in the minimal TRT-RTX package environment.
• Exported and ran Qwen3.5 0.8B and 9B text-only INT4 QDQ models with TRT-RTX EP + CUDA graph enabled.
  • 0.8B: TTFT 1.18s, decode 64.44 tok/s, answer starts: "The history of artificial intelligence..."
  • 9B: TTFT 1.47s, decode 48.00 tok/s, answer is a coherent reasoning-style response.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Enables more reliable CUDA-graph-style replay/reuse for Qwen3.5 on the TRT-RTX execution provider by stabilizing input/output buffer addresses and fixing a couple of TRT-RTX QDQ export edge cases.

Changes:

• Add shared past/present recurrent-state buffer mode to keep bindings stable for TRT-RTX graph replay.
• Keep Qwen2VL-style attention_mask and 3D position_ids at decode-stable shapes for graph capture/shared-buffer runs, updating contents in place.
• Adjust TRT-RTX QDQ export behavior for SkipLayerNorm output naming and avoid mixed INT8 weight-only overrides on explicit QDQ paths.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/python/py/models/builders/qwen.py	Avoid applying mixed INT8 (weight-only/QOperator) overrides when exporting explicit QDQ.
src/python/py/models/builders/base.py	Pass redirected SkipLayerNorm output_3 name to primitives to avoid duplicate producers in QDQ export.
src/models/recurrent_state.h	Track whether past/present buffers are shared.
src/models/recurrent_state.cpp	Implement shared-buffer recurrent-state allocation/binding and disable swapping/rewind rebinding in that mode.
src/models/position_inputs.h	Add static-shape handling APIs for attention mask and 3D position IDs.
src/models/position_inputs.cpp	Implement static mask + stable decode position_ids tensors and in-place updates for TRT-RTX graph replay.
examples/python/model-qa.py	Add NvTensorRtRtx EP option to the example CLI.

[Yen-Shi Wang (yen-shi)]

@microsoft-github-policy-service agree company="NVIDIA Corporation"

[anshuman (anskumar01)]

kunal-vaishnavi , Baiju Meswani (@baijumeswani) , can you please help review.
Cc anujj

[Yen-Shi Wang (yen-shi)]

Thanks for the reviews. I rewrote/rebased the branch and addressed the outstanding threads in a smaller two-commit form:

• The first commit is the Qwen3.5 text-only export overlap with Add text-only mode support for Qwen 3.5 model builder #2157.
• The second commit is only the TRT-RTX-specific delta needed for Qwen3.5 shared-buffer/CUDA-graph inference.
• The previous Qwen-specific runtime edits in position_inputs.cpp were removed; the current branch no longer touches that file.
• The previous Qwen-local mixed-precision quantization block was removed; the branch now uses the shared base k_quant_linear path from latest main.
• I checked compatibility with Add text-only mode support for Qwen 3.5 model builder #2157 via git merge-tree; the same Qwen files are touched, but Git auto-merges them with no conflict markers.
• Rebuilt and verified Qwen3.5 0.8B and 9B export + TRT-RTX inference with the rebuilt wheel.

[Yen-Shi Wang (yen-shi)]

Hi Akshay Sonawane (@apsonawane) and kunal-vaishnavi,
I have updated my changes based on PR #2100 , PR #2148, and PR #2157. Trim down changes to only 2 commits, one is changes borrowed from #2157, and the other commit enables the shared past/present buffers to TRT-RTX EP to do optimized inference with CUDA graph enabled.

Can you review again please? Thanks!

[Akshay Sonawane (apsonawane)]

Yen-Shi Wang (@yen-shi) can you integrate this change as well: #2127
I feel this will break cuda graph capture, because LpNormalization op does have a cuda kernel so it fallbacks to CPU. Please confirm your change works after getting PR in your branch

[Yen-Shi Wang (yen-shi)]

Yen-Shi Wang (@yen-shi) can you integrate this change as well: #2127 I feel this will break cuda graph capture, because LpNormalization op does have a cuda kernel so it fallbacks to CPU. Please confirm your change works after getting PR in your branch

My changes have already been applied on top of #2127, and I've tested TRT-RTX EP on Qwen3.5 models without issues.

[Baiju Meswani (baijumeswani)]

Yen-Shi Wang (@yen-shi) could you please resolve the merge-conflicts as well?

[Yen-Shi Wang (yen-shi)]

Baiju Meswani (@baijumeswani) I've done final clean-up, can you review again please? Thanks!

[Baiju Meswani (baijumeswani)]

Yen-Shi Wang (@yen-shi) I hope you don't mind if I push on top of your branch. There seems to be a test failing. And I am also addressing kunal-vaishnavi comment.

I want to include this PR for the ort-genai release 0.14.0