Enable AVX NE CONVERT for FP16 to FP32 cast #21183
Conversation
* Enable AVX_NE_CONVERT detection via CPUID. * Developed assembly kernel using the new ISA. * Integrated kernel.
|
/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline |
|
/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Linux Android Emulator QNN CI Pipeline |
|
/azp run Android CI Pipeline,iOS CI Pipeline,ONNX Runtime React Native CI Pipeline |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
Azure Pipelines successfully started running 10 pipeline(s). |
1 similar comment
|
Azure Pipelines successfully started running 10 pipeline(s). |
| @@ -1037,6 +1037,14 @@ MlasConvertHalfToFloatBuffer( | |||
| size_t Count | |||
| ); | |||
|
|
|||
| extern "C" void | |||
| MLASCALL | |||
| MlasConvertHalfToFloatBufferAVX2( | |||
There was a problem hiding this comment.
as we discussed, could you please combine the MlasConvertHalfToFloatBufferAVX2 and MlasConvertHalfToFloatBuffer function?
| @@ -0,0 +1,148 @@ | |||
| ;++ | |||
There was a problem hiding this comment.
could you please add also add a version for Linux?
|
i think the build failure of QNN CI pipeline is that it uses msvc 14.36, which doesn't support vcvtneeph2ps instruction yet. Other windows CI pipeline uses 14.40. @snnn, any ideas why QNN CI pipeline doesn't use same msvc version? |
Description
Implementation of a new cast assembly kernel that uses AVX_NE_CONVERT instructions to accelerate casting from FP16 to FP32. Added CPUID checks to determine support of the ISA.
Motivation and Context
Currently FP16 models executed on systems that lack complete FP16 operator support use single precision on every node to run the model, this means the original FP16 weights have to be casted to FP32 in order to run the model properly, this change aims to accelerate the casting by using upconvert instructions and therefore improve performance.