Enable FusedRMSNorm#78

Merged

jithunnair-amd merged 9 commits intomasterfrom

dev/hubertlu/FusedRMSNorm

Aug 5, 2022

Commits on Apr 15, 2022

FusedRMSNorm/"T5LayerNorm" based on FusedLayerNorm (#1274)

authored and
hubertlu-tw
committed
fix and generate docs for FusedRMSNorm (#1285)

eqy
authored and
hubertlu-tw
committed
[FusedRMSNorm doc] document where epsilon is added (#1295)

stas00
authored and
hubertlu-tw
committed
Fix some bugs
hubertlu-tw
committed
Optimize HostRMSNormGradient and HostApplyRMSNorm for AMD GPUs
hubertlu-tw
committed
Fix NaN issues in FusedRMSNorm
hubertlu-tw
committed

Commits on Jul 29, 2022

Update test_fused_layer_norm.py
hubertlu-tw
committed

Commits on Aug 1, 2022

Skip test_fused_layer_norm.TestAutocastFusedRMSNorm on ROCm
hubertlu-tw
committed

Commits on Aug 2, 2022

Use at::cuda::warp_size() instead of at::cuda::getCurrentDeviceProperties()->warpSize
hubertlu-tw
committed