[Test]Add quantization test case - #8666
Conversation
Signed-off-by: Wang Kunpeng <1289706727@qq.com>
…to ut-refactor # Conflicts: # tests/ut/quantization/methods/test_kv_c8.py # tests/ut/quantization/methods/test_w4a16.py # tests/ut/quantization/methods/test_w4a4_flatquant.py # tests/ut/quantization/methods/test_w4a8.py # tests/ut/quantization/methods/test_w8a16.py # tests/ut/quantization/methods/test_w8a8_static.py # tests/ut/quantization/test_modelslim_config.py # tests/ut/quantization/test_w8a8_dynamic.py
Signed-off-by: Wang Kunpeng <1289706727@qq.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on enhancing the test infrastructure for quantization methods. It introduces new test files, reorganizes existing ones for better maintainability, and adds significant test coverage for various quantization schemes and configurations. These changes ensure better reliability and easier verification of quantization logic. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
There was a problem hiding this comment.
Code Review
This pull request introduces several enhancements and test coverage improvements for the quantization module, specifically focusing on W8A8, W4A4, and MXFP8 quantization schemes. It adds new test files for registry and configuration management, updates existing tests to use more robust mocking and real module structures, and includes utility functions for quantization parameter parsing and KV cache handling. My feedback highlights that some tests in test_w8a8_static.py rely on MagicMock for nn.Module objects, which can lead to unreliable verification; I recommend using real nn.Module instances to ensure parameter registration and tensor operations are correctly validated.
| self.assertNotIn("deq_scale", dict(layer.named_parameters())) | ||
|
|
||
| @patch.dict(os.environ, {"VLLM_ASCEND_ENABLE_NZ": "1"}) | ||
| @patch("torch_npu.npu_format_cast") | ||
| def test_process_weights_after_loading_with_nz1(self, mock_npu_format_cast): | ||
| layer = MagicMock() | ||
|
|
||
| layer.weight.data = torch.randint(-127, 128, (128, 256), dtype=torch.int8) | ||
| layer.weight.data = torch.randint(-128, 127, (128, 256), dtype=torch.int8) | ||
| layer.input_scale.data = torch.tensor([0.1]) | ||
| layer.input_offset.data = torch.tensor([0]) | ||
| layer.deq_scale = torch.tensor([0.5]) | ||
| layer.weight_scale.data = torch.randn(128, 1) | ||
| layer.weight_offset.data = torch.randn(128, 1) | ||
|
|
||
| mock_npu_format_cast.return_value = MagicMock | ||
| mock_npu_format_cast.side_effect = identity | ||
| self.method.process_weights_after_loading(layer) | ||
|
|
||
| expected_offset = torch.tensor([0]).repeat(256).to(torch.int8) | ||
| self.assertTrue(torch.equal(layer.aclnn_input_offset.data, expected_offset)) | ||
| self.assertFalse(layer.aclnn_input_offset.requires_grad) | ||
|
|
||
| self.assertFalse(layer.deq_scale.requires_grad) | ||
|
|
||
| self.assertEqual(layer.weight.data.shape, (256, 128)) | ||
| self.assertEqual(layer.weight_scale.data.shape, (128,)) | ||
| self.assertEqual(layer.weight_offset.data.shape, (128,)) | ||
| mock_npu_format_cast.assert_called_once() | ||
| self.assertTrue(isinstance(layer.deq_scale, MagicMock)) | ||
|
|
||
| @patch.dict(os.environ, {"VLLM_ASCEND_ENABLE_NZ": "2"}) | ||
| @patch("torch_npu.npu_format_cast") | ||
| def test_process_weights_after_loading_with_nz2(self, mock_npu_format_cast): | ||
| def test_process_weights_after_loading_with_nz2_and_compressed_tensors(self, mock_npu_format_cast): | ||
| layer = MagicMock() | ||
|
|
||
| layer.weight.data = torch.randint(-127, 128, (128, 256), dtype=torch.int8) | ||
| layer.weight.data = torch.randint(-128, 127, (128, 256), dtype=torch.int8) | ||
| layer.input_scale.data = torch.tensor([0.1]) | ||
| layer.input_offset.data = torch.tensor([0]) | ||
| layer.deq_scale = torch.tensor([0.5]) | ||
| layer.weight_scale.data = torch.randn(128, 1) | ||
| layer.weight_offset.data = torch.randn(128, 1) | ||
| layer.ascend_quant_method = COMPRESSED_TENSORS_METHOD | ||
|
|
||
| mock_npu_format_cast.return_value = MagicMock | ||
| mock_npu_format_cast.side_effect = identity | ||
| self.method.process_weights_after_loading(layer) | ||
|
|
||
| expected_offset = torch.tensor([0]).repeat(256).to(torch.int8) | ||
| self.assertTrue(torch.equal(layer.aclnn_input_offset.data, expected_offset)) | ||
| self.assertFalse(layer.aclnn_input_offset.requires_grad) | ||
|
|
||
| self.assertFalse(layer.deq_scale.requires_grad) | ||
|
|
||
| self.assertEqual(layer.weight.data.shape, (256, 128)) | ||
| self.assertEqual(layer.weight_scale.data.shape, (128,)) | ||
| self.assertEqual(layer.weight_offset.data.shape, (128,)) | ||
| mock_npu_format_cast.assert_called_once() | ||
| self.assertIn("deq_scale", dict(layer.named_parameters())) | ||
| self.assertFalse(isinstance(layer.deq_scale, MagicMock)) |
There was a problem hiding this comment.
The use of MagicMock for the layer object in these tests (lines 125, 149, 174) makes the verification of parameter registration and tensor computations unreliable.
- At line 125 and 174,
dict(layer.named_parameters())is performed on a mock, which returns an empty dictionary by default, meaningassertNotInpasses for the wrong reason andassertInshould technically fail unless the mock is specifically configured. - At line 149, asserting that
layer.deq_scaleis aMagicMockconfirms that the test is merely verifying mock interactions rather than the actual quantization logic or tensor values.
It is highly recommended to use a realnn.Module(similar to thebuild_layerhelper used in other test files) to ensure that parameters are correctly registered and that the mathematical operations produce valid tensors.
Signed-off-by: Wang Kunpeng <1289706727@qq.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com> Signed-off-by: zouyida2052 <zouyida2002@gmail.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com> Signed-off-by: zc02384840 <zc02384840@antgroup.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com> Signed-off-by: PiratePai <416932041@qq.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com> Signed-off-by: yangzhe-2026 <yangzhe@isrc.iscas.ac.cn>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com> Signed-off-by: ZhuQi-seu <zhuqi12@huawei.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com> Signed-off-by: nanxing <1014662416@qq.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com>
### What this PR does / why we need it? **1. Remove meaningless parameters.** **2. Reconstruct the UT directory `tests/ut/quantization` for quantization features** tests/ut/quantization/ ├── conftest_quantization.py ├── methods │ ├── test_kv_c8.py │ ├── test_registry.py │ ├── test_w4a16.py │ ├── test_w4a4_flatquant.py │ ├── test_w4a4_laos_dynamic.py │ ├── test_w4a4_mxfp4.py │ ├── test_w4a8.py │ ├── test_w8a16.py │ ├── test_w8a8_dynamic.py │ ├── test_w8a8_mxfp8.py │ ├── test_w8a8_pdmix.py │ └── test_w8a8_static.py ├── test_compressed_tensors_config.py ├── test_method_adapters.py ├── test_modelslim_config.py ├── test_quant_parser.py └── test_utils.py **3. Improve the UT coverage for quantization features.** ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? pytest --cov=vllm_ascend.quantization tests/ut/quantization/ - vLLM version: v0.19.0 - vLLM main: vllm-project/vllm@6f786f2 --------- Signed-off-by: Wang Kunpeng <1289706727@qq.com>
What this PR does / why we need it?
1. Remove meaningless parameters.
2. Reconstruct the UT directory
tests/ut/quantizationfor quantization featurestests/ut/quantization/
├── conftest_quantization.py
├── methods
│ ├── test_kv_c8.py
│ ├── test_registry.py
│ ├── test_w4a16.py
│ ├── test_w4a4_flatquant.py
│ ├── test_w4a4_laos_dynamic.py
│ ├── test_w4a4_mxfp4.py
│ ├── test_w4a8.py
│ ├── test_w8a16.py
│ ├── test_w8a8_dynamic.py
│ ├── test_w8a8_mxfp8.py
│ ├── test_w8a8_pdmix.py
│ └── test_w8a8_static.py
├── test_compressed_tensors_config.py
├── test_method_adapters.py
├── test_modelslim_config.py
├── test_quant_parser.py
└── test_utils.py
3. Improve the UT coverage for quantization features.
Does this PR introduce any user-facing change?
no
How was this patch tested?
pytest --cov=vllm_ascend.quantization tests/ut/quantization/