Skip to content

feat: simulate grouped prefix caching in the vLLM scheduler and plan it from probe evidence - #781

Open
Arsene12358 wants to merge 4 commits into
yimingl/aisim-2289-simulate-prefix-caching-for-grouped-fpmfrom
yimingl/aisim-2292-grouped-prefix-caching-scheduler
Open

feat: simulate grouped prefix caching in the vLLM scheduler and plan it from probe evidence#781
Arsene12358 wants to merge 4 commits into
yimingl/aisim-2289-simulate-prefix-caching-for-grouped-fpmfrom
yimingl/aisim-2292-grouped-prefix-caching-scheduler

fix: apply the Mamba align split only to ranks with a Mamba state group

d23b71f
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar