Skip to content

Bump FlashInfer to 0.6.16.post4 #33092

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
elvischenv wants to merge 5 commits into from
Closed

Bump FlashInfer to 0.6.16.post4 #33092

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7b650ec
Bump FlashInfer to 0.6.16
elvischenv Jul 31, 2026
c0412ba
Use FlashInfer 0.6.16.post1 for the tvm-ffi ABI fix
elvischenv Aug 6, 2026
a7b0b8c
Move to FlashInfer 0.6.16.post2
elvischenv Aug 7, 2026
f730e3f
Move to FlashInfer 0.6.16.post4
elvischenv Aug 11, 2026
3c6d972
fix fast prefill plan api breakage
elvischenv Aug 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ ARG PIP_DEFAULT_INDEX
ARG UBUNTU_MIRROR
ARG GITHUB_ARTIFACTORY=github.com
ARG INSTALL_FLASHINFER_JIT_CACHE=0
ARG FLASHINFER_VERSION=0.6.15.post1
ARG FLASHINFER_VERSION=0.6.16.post4
ARG TRTLLM_GEN_MOE_CUBIN_URL="https://github.com/sgl-project/whl/releases/download/trtllm_gen_moe_cubin_20260617/trtllm_gen_moe_cubin_pool_20260617_v0613rc1.zip"
ARG TRTLLM_GEN_MOE_CUBIN_SHA256="4900501cbe782a76b08a5858f9f07152287b97cb68114466dac286366b66c192"
ARG TRTLLM_GEN_MOE_CUBIN_ARCHIVE_ROOT="trtllm_gen_moe_cubin_pool_20260617_v0613rc1"
Expand Down
2 changes: 1 addition & 1 deletion python/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ dependencies = [
"einops",
"fastapi",
"flash-attn-4>=4.0.0b18",
"flashinfer_python[cu13]==0.6.15.post1", # keep it aligned with jit-cache version in Dockerfile
"flashinfer_python[cu13]==0.6.16.post4", # keep it aligned with jit-cache version in Dockerfile
"gguf",
"humming-kernels[cu13]==0.1.10",
"interegular",
Expand Down
2 changes: 1 addition & 1 deletion python/sglang/srt/entrypoints/engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1653,7 +1653,7 @@ def _set_envs_and_config(server_args: ServerArgs):
if server_args.attention_backend == "flashinfer":
assert_pkg_version(
"flashinfer_python",
"0.6.15.post1",
"0.6.16.post4",
"Please uninstall the old version and "
"reinstall the latest version by following the instructions "
"at https://docs.flashinfer.ai/installation.html.",
Expand Down
1 change: 1 addition & 0 deletions python/sglang/srt/layers/attention/flashinfer_backend.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ def fast_prefill_plan(
fixed_split_size if fixed_split_size is not None else -1,
False, # disable_split_kv
0, # num_colocated_ctas
0, # uniform_q_len
]
self._plan_info = self._cached_module.plan(*args)

Expand Down
2 changes: 1 addition & 1 deletion python/sglang/srt/utils/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1999,7 +1999,7 @@ def check_pkg_version_at_least(pkg: str, min_version: str) -> bool:

Args:
pkg: Package name (distribution name, e.g., "flashinfer-python")
min_version: Minimum version required (e.g., "0.6.15.post1")
min_version: Minimum version required (e.g., "0.6.16.post4")

Returns:
True if package is installed and version >= min_version, False otherwise
Expand Down
Loading