[Codegen] FP4 support #17630
[Codegen] FP4 support #17630
Conversation
|
cc: @tqchen |
jinhongyii
force-pushed
the
fp4
branch
2 times, most recently
from
1ac2b1d to
ee3f17b
Compare
|
CI failed due to the ml_dtypes version, #17686 is trying to upgrade ml_dtypes |
|
hmm, the ml_dtypes>=0.5.1 was installed, but was downgraded to 0.4 during tensorflow install: https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm-docker/branches/main/runs/2062/nodes/50/steps/114/log/?start=0 |
|
|
|
one workaround is to install tensorflow w/o checking the dependencies: |
| } else if (lanes == 4) { | ||
| vec = "x4"; | ||
| } else { | ||
| LOG(FATAL) << "Only support scalar and vector types of width (2, 4, 8) for FP8"; |
There was a problem hiding this comment.
| LOG(FATAL) << "Only support scalar and vector types of width (2, 4, 8) for FP8"; | |
| LOG(FATAL) << "Only support scalar and vector types of width (2, 4) for FP4"; |
| if (type.code() == DataType::kE2M1Float) { | ||
| suffix = "_e2m1"; | ||
| } else { | ||
| LOG(FATAL) << "Unsupported FP8 type in CUDA codegen"; |
There was a problem hiding this comment.
| LOG(FATAL) << "Unsupported FP8 type in CUDA codegen"; | |
| LOG(FATAL) << "Unsupported FP4 type in CUDA codegen"; |
|
ci is green, the manually built cpu and GPU images work well. We can go ahead with the manually built images. I will follow-up to use automatic built images later once #17691 lands |
|
Thank you so much for getting the CI ready ❤️ @yongwww! |
* fp4 * fix test * fix lint * fix * Test with manually built images --------- Co-authored-by: Yong Wu <[email protected]>
3 participants
add fp4 codegen/runtime support