[CUDA] fp16 intB gemm

[tianleiwu]

Description

• Add fpA intB gemm kernel from WeightOnlyGroupwiseQuantGemmPlugin of TensorRT-LLM.
• Add prepacking to convert weight/scales/zero_points to adapt MatMulNBits to use the kernel.

Limitations:

• Only enable fp16 kernel. BF16 support will be added later.
• Requires zero points. The support of scales only might be added later.
• Bias is not enabled since previous MatMulNBits kernel does not support bias.
• Weight prepacking (preprocessing) is done in CPU, which could be slow for large models. We will move that part to cuda later.
• GEMM profiling is done during session creation. It could take a long time for large models. We might add cache of profiling result (and load it from file if available).

The feature is currently disabled by default. To enable it, set an environment variable ORT_FPA_INTB_GEMM=1 to enable it.

Motivation and Context

To improve performance of LLM.

Initial result shows 2.2x throughput on prompt processing and 1.25X throughput on token generation using onnxruntime-genai benchmark_e2e.py on phi-4-mini-instruct on A100.

[Copilot]

Pull Request Overview

This PR introduces a new fpA intB GEMM kernel based on the WeightOnlyGroupwiseQuantGemmPlugin from TensorRT-LLM to improve LLM throughput. Key changes include the addition of several new header files implementing specialized dequantized multistage MMA operations, updated traits for int8 and fpA intB GEMM kernels, and CMake configuration updates to support enhanced CUDA architectures and features.

Reviewed Changes

Copilot reviewed 80 out of 80 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/threadblock/default_dq_mma_multistage.h	Implements dequantized multistage MMA operations with template specializations for different quantization methods.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/threadblock/default_dq_mma.h	Declares threadblock-scoped GEMM kernels with dequantization support.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/kernel/mixed_gemm_B_layout.h	Provides layout especializations to unify weight layouts for quantized GEMM kernels.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/kernel/default_int8_traits.h	Defines architecture traits for int8 GEMM implementations.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/kernel/default_fpA_intB_traits.h	Introduces fpA intB traits to support mixed precision GEMM operations.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/device/gemm_universal_base_compat.h	Implements a compatibility layer for universal GEMM kernels supporting different modes and split-K configurations.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/gemm/collective/*.hpp and *.inl	Adds collective implementations and builders for interleaved MMA operations targeting SM90 and later architectures.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/epilogue_helpers.h and fused_activations.h	Defines helper types and fused activation functions to support various epilogue operations.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/compute_occupancy.h	Provides utilities for computing the occupancy of CUDA kernels based on shared memory usage.
onnxruntime/contrib_ops/cuda/llm/cutlass_extensions/arch/mma.h	Defines new tags and helper templates for dequantized MMA operators based on the quantization scheme.
onnxruntime/contrib_ops/cuda/llm/common/*	Adds common utilities for workspace memory alignment, CUDA runtime queries, and logging.
cmake/*	Updates CMake configuration to properly detect and set up supported CUDA architectures and features (e.g. enabling FP8/FP4, TMA support).