[vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target. #10391
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mei-ye
requested review from
a team,
Huyuwei,
Hzfengsy,
Laurawly,
MarisaKirisame,
ZihengJiang,
anijain2305,
areusch,
comaniac,
icemelon,
jcf94,
jroesch,
junrushao,
jwfromm,
kazum,
kevinthesun,
kparzysz-quic,
liangfu,
masahi,
mbrookhart,
merrymercy,
slyubomirsky,
tmoreau89,
tqchen,
vinx13,
wweic,
yzhliu and
zhiics
as code owners
Member
Member
|
Please remove the change in |
mei-ye
force-pushed
the
main
branch
3 times, most recently
from
b3b0aa9 to
e8b42df
Compare
masahi
force-pushed
the
main
branch
2 times, most recently
from
1e337c2 to
0a2e8c8
Compare
masahi
force-pushed
the
main
branch
4 times, most recently
from
8f6a79d to
cf00438
Compare
Member
|
Ok @mei-ye the CI is green now. The changes I made are:
|
masahi
approved these changes
Mar 7, 2022
…index expression.
ziqiangxu8457
pushed a commit
to ziqiangxu8457/tvm
that referenced
this pull request
Mar 9, 2022
…the vulkan SPIR-V target. (apache#10391) * [Vulkan] Add cmake change for spirv dot product * add spirv and vk runtime change * add back conv2d int8 related change * add back dense and group conv2d change * add back test_topi_conv2d_int8.py change (but don't test on vk) * check dot prod availablity in batch matmul schedule * do not run uint8 tensorization on arm * add vulkan target to conv2d int8 test but comment out on CI * do not run vk batch matmul test * Fix performance regression due to missing of constant folding in the index expression. Co-authored-by: Masahiro Masuda <[email protected]>
pfk-beta
pushed a commit
to pfk-beta/tvm
that referenced
this pull request
Apr 11, 2022
…the vulkan SPIR-V target. (apache#10391) * [Vulkan] Add cmake change for spirv dot product * add spirv and vk runtime change * add back conv2d int8 related change * add back dense and group conv2d change * add back test_topi_conv2d_int8.py change (but don't test on vk) * check dot prod availablity in batch matmul schedule * do not run uint8 tensorization on arm * add vulkan target to conv2d int8 test but comment out on CI * do not run vk batch matmul test * Fix performance regression due to missing of constant folding in the index expression. Co-authored-by: Masahiro Masuda <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@masahi
[vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target. Currently only autotvm path is supported.
Prerequisites for compilation: (1) Use VulkanSDK 1.2.198 release with SPIR-V integer dot product suppport (2) set(USE_SPIRV_KHR_INTEGER_DOT_PRODUCT ON) in config.cmake and build (3) Use a driver that supports VK_KHR_shader_integer_dot_product extension.
The compiled binary can only be run on a hardware that supports relevant ISA. This work is tested on AMD RDNA2 famillies (e.g., Rembrandt and RX6800).
To compile on a device that supports this extension, use target: vulkan -from_device=0
To compile on a device that supports int8 but does not support this extension, add "-supports_integer_dot_product=1" or "-mattr=+dotprod" to the target string.
To support pre-released vulkan and SPIR-V extensions, we need SPIR-V tool and header file from Khronos github, use the option: USE_KHRONOS_SPIRV in config.cmake.
Example to use this feature can be found in: gallery/how_to/deploy_models/deploy_prequantized.py and gallery/how_to/deploy_models/deploy_prequantized_tflite.py