Pinned Loading

1. huggingface/transformers huggingface/transformers Public

   🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

   Python 135k 27.1k

2. huggingface/accelerate huggingface/accelerate Public

   🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

   Python 8k 970

3. huggingface/nanotron huggingface/nanotron Public

   Minimalistic large language model 3D-parallelism training

   Python 1.3k 122

4. NVIDIA/Megatron-LM NVIDIA/Megatron-LM Public

   Ongoing research training transformer models at scale

   Python 10.6k 2.4k

5. linkedin/Liger-Kernel linkedin/Liger-Kernel Public

   Efficient Triton Kernels for LLM Training

   Python 3.5k 209

6. tomaarsen/attention_sinks tomaarsen/attention_sinks Public

   Extend existing LLMs way beyond the original training length with constant memory usage, without retraining

   Python 677 40