Popular repositories Loading

1. aphrodite-engine aphrodite-engine Public

   Forked from aphrodite-engine/aphrodite-engine

   PygmalionAI's large-scale inference engine

   C++

2. exllama exllama Public

   Forked from turboderp-org/exllamav2

   A fast inference library for running LLMs locally on modern consumer-class GPUs

   Python

3. llama.cpp llama.cpp Public

   Forked from ggml-org/llama.cpp

   LLM inference in C/C++

   C++

4. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python

5. flash-attention flash-attention Public

   Forked from Dao-AILab/flash-attention

   Fast and memory-efficient exact attention

   Python

6. tabbyAPI tabbyAPI Public

   Forked from theroyallab/tabbyAPI

   An OAI compatible exllamav2 API that's both lightweight and fast

   Python