[Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to further optimize block_pool

[Jialin]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Most of the block_pool operators are on critical path

• forward path is hard blocked by kv block allocation
• decode cycle end is hard blocked by kv block free

In this PR, we're focusing on further optimization these 2 operators.

Bulk popleft instead of popleft n times

Originally, in block_pool.get_new_blocks, we popped blocks one at a time, which would triggered the second block to fake head connections (which are unnecessary operations as the second block might be popped right after this).

As we knew total number of blocks to pop ahead, we could simply introduce popleft_n for buck popleft. Overall, the number link list operations to linked list of popleft_n would only be half of n popleft.

Bulk append instead of append n times

Similar, in block_pool.free_blocks, we invoke append one at a time. Introducing bulk append would also cut link list operations by half.

Test Plan

• Evaluate with benchmark scripts
• Evaluate with benchmark_blockpoll
• New Unit Test for append_n and popleft_n are added

Test Result

benchmark scripts

• Get new blocks improved from 0.15ms to 0.008ms
• Free new blocks improved from 33us to 9us

After

Before

benchmark_blockpool

As expected, get_blocks and free_blocks times are cut in half.

After

Before

(Optional) Documentation Update

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Jialin]

resolve #21141

[gemini-code-assist]

Code Review

This pull request introduces popleft_n and append_n methods to FreeKVCacheBlockQueue for bulk operations, optimizing get_new_blocks and free_blocks in BlockPool. Benchmark results show significant improvements. To enhance robustness, I've suggested materializing the ordered_blocks iterable to a list in free_blocks to prevent potential OOM errors.

[houseroad]

Looks good to me. Impressive results, and two nits to consider to address.

[njhill]

Thanks @Jialin!