Qwen3.5: drop fp32 cast around RMSNorm in builder

[xiaofeihan1]

Removes the use_fp32 cast flags forcing LayerNorm IO to fp32 for Qwen3_5TextModel. Keeping RMSNorm in the model's native fp16 eliminates ~216 Cast nodes (108 to-fp32 + 108 to-fp16) across input_layernorm, post_attention_layernorm, and attention paths in a 24-layer build.

Measured on Qwen3.5-0.8B int4 WebGPU (3-run avg, prefill-1000 / 100 tok):

NVIDIA RTX 5080
gen tps: 76.6 -> 86.8 (+13.2%)
prompt tps: 6937 -> 7153 (+3.1%)

Intel iGPU(Xe-LPG, Intel Core Ultra 9 285K)
gen tps: 33.1 -> 36.1 (+9.1%)
prompt tps: 785 -> 789 (+0.5%)

Apple M3 Max
gen tps: 115.7 -> 122.3 (+5.7%)
prompt tps: 3149.3 -> 3199.8 (+1.6%)

[Copilot]

Pull request overview

This PR updates the Qwen3.5 Python model builder to avoid forcing RMSNorm/LayerNorm I/O through fp32, reducing the number of Cast nodes and improving runtime performance (notably on WebGPU) by keeping normalization in the model’s native dtype.

Changes:

• Remove the Qwen3.5-specific layernorm_attrs["cast"]["use_fp32"] override and associated per-IO cast flags.
• Keep OffsetRMSNorm’s add_offset = 1 behavior unchanged while allowing LayerNorm ops to remain in io_dtype.

[kunal-vaishnavi]

We have typically seen accuracy/quality issues emerge when not upcasting norms from FP16 or BF16 to FP32. Can you share some evaluation data that shows there isn't significant output quality degradation?

[apsonawane]

Can we test this change on a 36+ layer larger variant like Qwen3.5-4B?

[xiaofeihan1]

Can we test this change on a 36+ layer larger variant like Qwen3.5-4B?

Thanks for the tips. I have tested it, and it functions as expected.

[xiaofeihan1]

We have typically seen accuracy/quality issues emerge when not upcasting norms from FP16 or BF16 to FP32. Can you share some evaluation data that shows there isn't significant output quality degradation?

Thanks for your information! No regression in output quality. I tested Qwen 4b(q4f16) for WebGPU EP on NV5080.

After this PR, prompt: 1777.8 tps, gen: 52.2 tps. (click to see the output)

Thinking Process:

1. Analyze the Request:

   • Input: A text explaining the current state of NVIDIA Nsight Graphics support for WebGPU and alternative methods for profiling WebGPU shaders.
   • Task: Summarize the provided text.
   • Goal: Create a concise overview of the key points without losing essential information.

2. Analyze the Input Text:

   • Main Point: NVIDIA Nsight Graphics does not natively support WebGPU.
   • Reason: WebGPU is a new, browser-based API, while Nsight focuses on low-level native APIs (Vulkan, D3D, OpenGL).
   • Workarounds/Alternatives:
     1. Browser DevTools: Use Chrome DevTools Performance tab for basic GPU workload insights (less detailed than Nsight).
     2. Emulate WebGPU with Vulkan: Use WGPU (Rust-based) to compile WebGPU to Vulkan, then profile the Vulkan backend with Nsight Graphics.
     3. SPIR-V Translation: Translate WebGPU (WGSL) shaders to SPIR-V (via SPIRV-Cross/glslang) and profile the resulting SPIR-V shaders in Nsight (assuming Vulkan backend).
     4. Chrome Internals: Use Chrome Task Manager and flags for rough GPU memory/execution insights.
     5. Future: Wait for WebGPU toolchain maturity (Chrome DevTools updates, etc.).
   • Conclusion: No direct support, but browser tools and Vulkan translation (WGPU) offer viable paths for profiling.

3. Drafting the Summary (Iterative Process):

   • Draft 1 (Mental): Nsight Graphics doesn't support WebGPU yet because WebGPU is new and browser-based while Nsight is for native APIs. You can profile it using Chrome DevTools Performance tab. Another way is to use WGPU to turn WebGPU into Vulkan so you can use Nsight. You can also use SPIR-V tools to translate shaders. Chrome Task Manager is another option. Eventually, WebGPU tools will get better.

   • Draft 2 (More formal): Currently, NVIDIA Nsight Graphics lacks native WebGPU support due to the API differences. However, there are workarounds. First, use Chrome DevTools for basic profiling. Second, use WGPU to translate WebGPU to Vulkan, allowing Nsight Graphics to profile the Vulkan backend. Third, translate WGSL shaders to SPIR-V for analysis. Fourth, use Chrome's Task Manager for high-level metrics. Finally, wait for WebGPU toolchain maturity.

   • Draft 3 (Refining for clarity and structure):
     Current Status: NVIDIA Nsight Graphics does not natively support WebGPU, as it targets low-level native APIs (Vulkan, D3D, OpenGL) rather than browser-based WebGPU.
     Workarounds for Profiling:

     1. Browser DevTools: Use Chrome's Performance tab for basic GPU workload insights.
     2. Vulkan Translation: Use WGPU to compile WebGPU code to Vulkan, enabling Nsight Graphics to profile the backend.
     3. SPIR-V Profiling: Translate WebGPU shaders to SPIR-V to leverage Nsight's shader analysis capabilities.
     4. Chrome Internals: Monitor GPU usage via Chrome Task Manager and benchmarking flags.
        Future Outlook: Native WebGPU profiling tools are expected to mature as the ecosystem grows.

4. Final Polish: Check against the original text to ensure accuracy.

   • Does it mention WGPU? Yes.
   • Does it mention SPIR-V? Yes.
   • Does it mention Chrome DevTools? Yes.
   • Does it state the lack of native support clearly? Yes.

5. Final Output Generation: (Matches Draft 3 structure but concise).

   • Headline: NVIDIA Nsight Graphics and WebGPU Compatibility.
   • Key Constraint: No native support.
   • Solutions: Browser tools, WGPU/Vulkan bridge, SPIR-V translation, Chrome internals.
   • Future: Maturity expected.

   Let's condense this into a bulleted summary for readability.

   • No Native Support: NVIDIA Nsight Graphics does not natively support WebGPU, as it is designed for low-level native APIs (Vulkan, D3D, OpenGL) rather than browser-based WebGPU.
   • Profiling Workarounds:
     • Browser DevTools: Use Chrome's Performance tab for basic GPU workload analysis.
     • Vulkan Backend: Use

Before this PR, prompt: 1694.7 tps, gen: 46.8 tps. (click to see the output)

Thinking Process:

1. Analyze the Request:

   • Input: A text explaining the current state of NVIDIA Nsight Graphics support for WebGPU and alternative methods for profiling WebGPU shaders.
   • Task: Summarize the text.
   • Goal: Create a concise overview of the key points without losing essential information.

2. Analyze the Input Text:

   • Main Point: NVIDIA Nsight Graphics does not natively support WebGPU.
   • Reason: WebGPU is a new, browser-based API, while Nsight focuses on low-level native APIs (Vulkan, D3D, OpenGL).
   • Workarounds/Alternatives:
     1. Browser DevTools: Use Chrome DevTools Performance tab for basic profiling (limited detail compared to Nsight).
     2. Emulate WebGPU with Vulkan: Use WGPU (Rust-based) to compile WebGPU to Vulkan, then profile the Vulkan backend with Nsight Graphics.
     3. SPIR-V Translation: Translate WebGPU (WGSL) shaders to SPIR-V (via SPIRV-Cross/glslang) and profile the resulting Vulkan shaders with Nsight.
     4. Chrome Internals: Use Chrome Task Manager and flags for rough GPU insights.
     5. Future: Wait for WebGPU toolchain maturity (Chrome DevTools updates, etc.).
   • Conclusion: No direct support, but browser tools and Vulkan translation (WGPU) offer viable paths for performance tuning.

3. Drafting the Summary (Iterative Process):

   • Draft 1 (Mental): Nsight Graphics doesn't support WebGPU yet because WebGPU is new and browser-based while Nsight is for native APIs. You can profile it using Chrome DevTools Performance tab, but it's not detailed. Another way is to use WGPU to turn WebGPU into Vulkan, then use Nsight on that. You can also use SPIR-V tools to translate shaders. Chrome Task Manager is another option. Eventually, WebGPU tools will get better.

   • Draft 2 (More formal): Currently, NVIDIA Nsight Graphics lacks native WebGPU support due to the API differences. However, there are workarounds. First, use Chrome DevTools for basic profiling. Second, use WGPU to translate WebGPU to Vulkan, allowing Nsight Graphics to profile the backend. Third, translate WGSL to SPIR-V for shader analysis. Fourth, use Chrome's Task Manager. Finally, wait for WebGPU toolchain maturity.

   • Draft 3 (Refining for clarity and structure):
     Current Status: NVIDIA Nsight Graphics does not natively support WebGPU, as it targets low-level native APIs (Vulkan, D3D, OpenGL) rather than browser-based WebGPU.
     Workarounds for Profiling:

     1. Browser DevTools: Use Chrome's Performance tab for basic workload insights.
     2. Vulkan Translation: Use WGPU to compile WebGPU code to Vulkan, enabling Nsight Graphics to profile the backend.
     3. SPIR-V Shaders: Translate WGSL shaders to SPIR-V for direct shader analysis within Nsight.
     4. Chrome Internals: Monitor GPU usage via Chrome Task Manager and benchmarking flags.
        Future Outlook: Native WebGPU profiling tools are expected to mature as the ecosystem grows.

4. Final Polish: Check against the original text to ensure accuracy.

   • Does it mention WGPU? Yes.
   • Does it mention SPIR-V? Yes.
   • Does it mention Chrome DevTools? Yes.
   • Does it state the lack of native support clearly? Yes.

5. Final Output Generation: (Matches Draft 3 structure but concise).

   • Headline: Nsight Graphics and WebGPU Compatibility.
   • Key Constraint: No native support.
   • Solutions: DevTools, WGPU/Vulkan, SPIR-V, Chrome Internals.
   • Conclusion: Use workarounds for now.

   Let's make it a bulleted summary for readability.

   • No Native Support: NVIDIA Nsight Graphics does not natively support WebGPU, as it is designed for low-level native APIs (Vulkan, D3D, OpenGL) rather than browser-based WebGPU.
   • Profiling Workarounds:
     • Browser DevTools: Use Chrome's Performance tab for basic GPU workload analysis.
     • Vulkan Backend: Utilize WGPU to translate WebGPU code to Vulkan,

[xiaofeihan1]

@apsonawane Do you have any other comments on this PR? If not, I’ll go ahead and merge it.