Add k_quant_linear mixed-precision quantization for hybrid attention …

[apsonawane]

Adds k_quant_linear as a new int4_algo_config option that promotes linear attention projections and their MLPs from INT4 to INT8. This improves accuracy for hybrid models like Qwen3.5 where linear attention recurrence accumulates quantization errors across the full sequence (unlike softmax attention which normalizes per-step).

Changes

onnxruntime-genai

• qwen.py:
  • Added make_int4_algo_config() override with k_quant_linear support
  • Moved layer_types parsing before super().__init__() so make_int4_algo_config can access it during base class initialization
  • Removed the always-on INT8 quantization for linear attention layers (now opt-in via int4_algo_config)

[Copilot]

Pull request overview

Adds an opt-in mixed-precision INT4→INT8 promotion path for Qwen3.5 hybrid (linear + full) attention models by introducing a k_quant_linear int4_algo_config mode and making layer-type metadata available during base-class initialization.

Changes:

• Parse layer_types before super().__init__() so base init-time quantization config can inspect linear-attention layers.
• Remove the always-on INT8 overrides for linear-attention projections/MLP and replace them with an opt-in make_int4_algo_config() override keyed on k_quant_linear.
• Add a Qwen3.5-specific make_int4_algo_config() implementation that promotes selected MatMul weights to INT8.

[kunal-vaishnavi]

Can we add k_quant_linear to the list of supported options when the help message is displayed?

onnxruntime-genai/src/python/py/models/builder.py

Lines 413 to 420 in 19c2f7b

	int4_algo_config = Method for int4 quantization. Default is 'default'.
	Currently supported options are: 'default', 'rtn', 'rtn_last', 'k_quant', 'k_quant_mixed', 'k_quant_last'.
	default = algo_config passed to MatMulNBitsQuantizer is None. Quantizer uses default RTN algorithm. All MatMuls are quantized as int4.(different node naming conventions to `rtn`)
	rtn = RTN algorithm for int4 quantization.
	rtn_last = RTN algorithm where only the last MatMul (/lm_head/MatMul) is quantized as int8. Other MatMuls are quantized as int4.
	k_quant = k_quant algorithm for int4 quantization.
	k_quant_mixed = k_quant algorithm with mixed precision (int4 + int8).
	k_quant_last = k_quant algorithm where only the last MatMul (/lm_head/MatMul) is quantized as int8. Other MatMuls are quantized as int4.