[#5861][autodeploy] Refactor: Quantization Transforms with Inheritance#7227

Merged

Fridah-nv merged 25 commits intoNVIDIA:mainfrom

nv-auto-deploy:user/fridah/inherit-quant2

Sep 10, 2025

Commits on Sep 8, 2025

add torch ref impl for FP8, add unit test
Fridah-nv
committed
add torch ref impl for FP4, add op map unit test
Fridah-nv
committed
split linear and bmm quantization
Fridah-nv
committed
update quantize_linear_from_config to point to the custom op
Fridah-nv
committed
separate custom op into two torch ops
Fridah-nv
committed
quantized fusion transforms, WIP for FP4
Fridah-nv
committed
add QuantizationFusionMixin class
Fridah-nv
committed
quantized sharding class for FP8 and FP4
Fridah-nv
committed
remove QuantizationImpl in sharding, remove unused methods in QuantizationImpl
Fridah-nv
committed
remove custom_quant_linear op
Fridah-nv
committed
rename custom quant ops
Fridah-nv
committed
WIP to map custom quant op to real implementation using pattern matcher
Fridah-nv
committed
fix unit tests
Fridah-nv
committed
remove unused ENUM
Fridah-nv
committed
minor updates: rabbit feedback, docstrings, code cleaning
Fridah-nv
committed
clear unit tests on blackwell; address a few comments; rename FP ops to NVFP4 ops
Fridah-nv
committed
remove include_quantization from is_linear_node
Fridah-nv
committed
address few comments: remove pattern matcher fake mode patch; remove some usage of is_fake_linear_ops
Fridah-nv
committed
update quantization transforms:Linear, BMM, MoE and MoE matching into the inheritance system
Fridah-nv
committed
remove QuantizationImpl class; remove more reference of is_quantized_op
Fridah-nv
committed
fix test_quantization_utils.py
Fridah-nv
committed
minor: address comments, uncomment skipped unit tests
Fridah-nv
committed

Commits on Sep 9, 2025

skip quant gemm fusion for perf
Fridah-nv
committed
Merge branch 'main' into user/fridah/inherit-quant2
Fridah-nv
authored
Merge branch 'main' into user/fridah/inherit-quant2
Fridah-nv
authored

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[#5861][autodeploy] Refactor: Quantization Transforms with Inheritance#7227

[#5861][autodeploy] Refactor: Quantization Transforms with Inheritance#7227
Fridah-nv merged 25 commits intoNVIDIA:mainfrom
nv-auto-deploy:user/fridah/inherit-quant2

Commits on Sep 8, 2025

add torch ref impl for FP8, add unit test

add torch ref impl for FP4, add op map unit test

split linear and bmm quantization

update quantize_linear_from_config to point to the custom op

separate custom op into two torch ops

quantized fusion transforms, WIP for FP4

add QuantizationFusionMixin class

quantized sharding class for FP8 and FP4

remove QuantizationImpl in sharding, remove unused methods in QuantizationImpl

remove custom_quant_linear op

rename custom quant ops

WIP to map custom quant op to real implementation using pattern matcher

fix unit tests

remove unused ENUM

minor updates: rabbit feedback, docstrings, code cleaning

clear unit tests on blackwell; address a few comments; rename FP ops to NVFP4 ops

remove include_quantization from is_linear_node

address few comments: remove pattern matcher fake mode patch; remove some usage of is_fake_linear_ops

update quantization transforms:Linear, BMM, MoE and MoE matching into the inheritance system

remove QuantizationImpl class; remove more reference of is_quantized_op

fix test_quantization_utils.py

minor: address comments, uncomment skipped unit tests

Commits on Sep 9, 2025

skip quant gemm fusion for perf

Merge branch 'main' into user/fridah/inherit-quant2

Merge branch 'main' into user/fridah/inherit-quant2

[#5861][autodeploy] Refactor: Quantization Transforms with Inheritance#7227Fridah-nv merged 25 commits intoNVIDIA:mainNVIDIA/TensorRT-LLM:mainfrom nv-auto-deploy:user/fridah/inherit-quant2nv-auto-deploy/TensorRT-LLM:user/fridah/inherit-quant2Copy head branch name to clipboard

Commits

Commits on Sep 8, 2025

Commits on Sep 9, 2025

[#5861][autodeploy] Refactor: Quantization Transforms with Inheritance#7227
Fridah-nv merged 25 commits intoNVIDIA:mainfrom
nv-auto-deploy:user/fridah/inherit-quant2