[Qwen3.5] Make linear attention INT8 quantization opt-in via extra_options

[Jianhui Dai (daijh)]

Change Qwen3.5 mixed-precision quantization for linear attention layers from always-on to opt-in. Users can enable INT8 promotion of linear attention nodes by passing linear_attention_int8=true in extra_options.

[Copilot]

Pull request overview

This PR makes Qwen3.5’s mixed-precision quantization behavior for linear-attention layers opt-in, so INT8 promotion for selected MatMul weights is only applied when users request it via extra_options.

Changes:

• Gates the linear-attention INT8 weight promotion behind extra_options["linear_attention_int8"].
• Updates the inline comment to describe the feature as optional and explain the rationale.

[Jianhui Dai (daijh)]

Fix #2095

[Jianhui Dai (daijh)]

Guenther Schmuelling (@guschmue) PTAL

[Akshay Sonawane (apsonawane)]

Add linear_attention_int8 to the bools list in check_extra_options in builder.py, then just simplify this to
if extra_options.get("linear_attention_int8", False)

[kunal-vaishnavi]

Since this is just affecting the MatMuls, can we instead introduce a new int4 algo config or use an existing one? In the future, there could be another op instead of linear attention and we wouldn't want to change the extra options for that new op then. The algo config change would be a more generic solution.

onnxruntime-genai/src/python/py/models/builder.py

Lines 403 to 410 in 2a2ef8c

	int4_algo_config = Method for int4 quantization. Default is 'default'.
	Currently supported options are: 'default', 'rtn', 'rtn_last', 'k_quant', 'k_quant_mixed', 'k_quant_last'.
	default = algo_config passed to MatMulNBitsQuantizer is None. Quantizer uses default RTN algorithm. All MatMuls are quantized as int4.(different node naming conventions to `rtn`)
	rtn = RTN algorithm for int4 quantization.
	rtn_last = RTN algorithm where only the last MatMul (/lm_head/MatMul) is quantized as int8. Other MatMuls are quantized as int4.
	k_quant = k_quant algorithm for int4 quantization.
	k_quant_mixed = k_quant algorithm with mixed precision (int4 + int8).
	k_quant_last = k_quant algorithm where only the last MatMul (/lm_head/MatMul) is quantized as int8. Other MatMuls are quantized as int4.

[Jianhui Dai (daijh)]

Since this is just affecting the MatMuls, can we instead introduce a new int4 algo config or use an existing one? In the future, there could be another op instead of linear attention and we wouldn't want to change the extra options for that new op then. The algo config change would be a more generic solution.

onnxruntime-genai/src/python/py/models/builder.py

Lines 403 to 410 in 2a2ef8c

	int4_algo_config = Method for int4 quantization. Default is 'default'.
	Currently supported options are: 'default', 'rtn', 'rtn_last', 'k_quant', 'k_quant_mixed', 'k_quant_last'.
	default = algo_config passed to MatMulNBitsQuantizer is None. Quantizer uses default RTN algorithm. All MatMuls are quantized as int4.(different node naming conventions to `rtn`)
	rtn = RTN algorithm for int4 quantization.
	rtn_last = RTN algorithm where only the last MatMul (/lm_head/MatMul) is quantized as int8. Other MatMuls are quantized as int4.
	k_quant = k_quant algorithm for int4 quantization.
	k_quant_mixed = k_quant algorithm with mixed precision (int4 + int8).
	k_quant_last = k_quant algorithm where only the last MatMul (/lm_head/MatMul) is quantized as int8. Other MatMuls are quantized as int4.

Thank you.
Using a new INT4 algo config is definitely the right move here.

I’m going to close this PR for now, as PR #2100 has been created as a draft for this purpose.