Support PyTorch Tensor Parallelism #868
Description
🚀 The feature, motivation and pitch
PyTorch provides native Tensor Parallel techniques for model training.
https://docs.pytorch.org/docs/stable/distributed.tensor.parallel.html
It would be great to have it supported in liger-kernel.
Alternatives
No response
Additional context
No response