Skip to content

[Core] Use FlashInfer for pre-SM100 NVFP4 KV cache updates #100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lesj0610 wants to merge 49 commits into
base: main
Choose a base branch
from
Open

[Core] Use FlashInfer for pre-SM100 NVFP4 KV cache updates #100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
bf6aa1c
Use FlashInfer for pre-SM100 NVFP4 KV cache updates
lesj0610 Jun 24, 2026
3ceeb08
Support native FlashInfer NVFP4 KV planning
lesj0610 Jun 26, 2026
bce74cb
Bump FlashInfer dependency for NVFP4 slot mapping
lesj0610 Jun 26, 2026
24044a6
Cache FlashInfer NVFP4 slot mapping writer
lesj0610 Jun 26, 2026
1a2ee4a
Merge branch 'main' into lesj/flashinfer-nvfp4-kv-cache-20260624
lesj0610 Jun 27, 2026
11e4793
Allow Gemma4 auto attention backend selection
lesj0610 Jun 27, 2026
d2534b5
Add FlashInfer NVFP4 paged prefill path
lesj0610 Jun 27, 2026
7ed11a9
Harden NVFP4 FA2 prefill path
lesj0610 Jun 28, 2026
fcfbea6
Warn on missing NVFP4 paged dequant
lesj0610 Jun 28, 2026
5a977a9
Guard NVFP4 FA2 prefill for mixed head dims
lesj0610 Jun 28, 2026
7efddfc
Guard NVFP4 FA2 prefill for large heads
lesj0610 Jun 28, 2026
20c2df7
Fix mixed-head NVFP4 split views
lesj0610 Jun 28, 2026
b3de160
Merge branch 'main' into lesj/flashinfer-nvfp4-kv-cache-20260624
lesj0610 Jun 28, 2026
0c32dda
Fix FlashInfer mixed-head KV cache typing
lesj0610 Jun 28, 2026
0aa1f83
Restore DiffusionGemma Triton fallback
lesj0610 Jun 29, 2026
1f369a6
Gate NVFP4 query dtype on trtllm-gen
lesj0610 Jul 3, 2026
5646187
Drop Gemma4 selector copy from NVFP4 PR
lesj0610 Jul 3, 2026
b5585ad
Drop invalid FlashInfer version bump
lesj0610 Jul 3, 2026
399e1b7
Centralize attention KV cache shape helpers
lesj0610 Jul 3, 2026
762f0dc
Cache FlashInfer NVFP4 KV views
lesj0610 Jul 3, 2026
aaaf154
Tighten NVFP4 KV split stride checks
lesj0610 Jul 3, 2026
7bf65f2
Log NVFP4 FA2 prefill workspace reservation
lesj0610 Jul 3, 2026
25171d9
Merge origin/main into NVFP4 FlashInfer branch
lesj0610 Jul 4, 2026
5ee3d55
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Jul 4, 2026
a5872ea
Move cache dtype selection to attention specs
lesj0610 Jul 4, 2026
645dc5e
Merge origin/main into FlashInfer NVFP4 KV cache PR
lesj0610 Jul 6, 2026
2c2bc0f
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Jul 7, 2026
cd06010
Gate native NVFP4 updates on TRTLLM availability
lesj0610 Jul 8, 2026
da00dd5
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Jul 8, 2026
3d4bd85
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Jul 10, 2026
dd02f6c
[Bugfix] Guard FlashInfer stacked KV cache layout contract
lesj0610 Jul 15, 2026
8941e04
[Bugfix] Preserve FlashInfer cascade KV cache contract
lesj0610 Jul 15, 2026
16e3e21
[Perf] Avoid full-context NVFP4 prefill scratch
lesj0610 Jul 15, 2026
edb4741
Merge origin/main into FlashInfer NVFP4 KV cache
lesj0610 Jul 16, 2026
b28448f
Merge origin/main into FlashInfer NVFP4 KV cache
lesj0610 Jul 21, 2026
805af67
Merge upstream main into NVFP4 KV cache PR
lesj0610 Aug 1, 2026
7f7fff7
Merge upstream main into NVFP4 KV cache PR
lesj0610 Aug 8, 2026
e505363
Reject NVFP4 scale-search dtypes without native store
lesj0610 Aug 8, 2026
3fab113
Merge upstream main into NVFP4 KV cache PR
lesj0610 Aug 12, 2026
c5599fd
Merge upstream main into NVFP4 KV cache PR
lesj0610 Aug 14, 2026
3e70b1f
Merge upstream main into NVFP4 KV cache PR
lesj0610 Aug 15, 2026
7daad07
Merge upstream main into NVFP4 KV cache PR
lesj0610 Aug 22, 2026
d165855
Merge branch 'main' into lesj/flashinfer-nvfp4-kv-cache-20260624
lesj0610 Aug 23, 2026
ce6ad7c
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
Aug 26, 2026
f7b8cdc
test: adapt NVFP4 FlashInfer tests to upstream layout and sinks contr…
Aug 26, 2026
6d87675
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Aug 29, 2026
05391d2
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Sep 2, 2026
667913d
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Sep 8, 2026
0241e32
Merge remote-tracking branch 'origin/main' into lesj/flashinfer-nvfp4…
lesj0610 Sep 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading