[2024-09-04] SGLang v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision
[2024-07-25] Achieving Faster Open-Source Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM)
[2024-02-05] Fast JSON Decoding for Local LLMs with Compressed Finite State Machine
[2024-01-17] Fast and Expressive LLM Inference with RadixAttention and SGLang
[2024-10-10] Efficient LLM Inference with SGLang
[2024-10-05] SGLang Double Sparsity
[2024-09-22] SGLang DeepSeek MLA
SGLang v0.2: Faster Interface and Runtime for LLM Inference
Notes: It's working in progress.
[NeurIPS 24] SGLang: Efficient Execution of Structured Language Model Programs
