TP-invariant Training: bitwise-identical training across TP degrees and GPU Architecture - #4740

TP-invariant Training: bitwise-identical training across TP degrees and GPU Architecture#4740

jinzex wants to merge 4 commits into

NVIDIA:devNVIDIA/Megatron-LM:devfrom

jinzex:jinzex/tp-invariant-numerics-upstreamjinzex/Megatron-LM:jinzex/tp-invariant-numerics-upstreamCopy head branch name to clipboard