Fix decode bucket filter issues from #1122

[yangulei]

Fixes

Two bugs introduced by #1122 (commit f24f3f9):

1. IndexError when using file-based bucketing (GAUDISW-248587)

When VLLM_BUCKETING_FROM_FILE is used (e.g. GraniteMoeHybrid model), ctx_range is passed as an empty list to generate_buckets(). The num_ctx_tokens_less_or_equal_batched_max_model_len filter accessed ctx_range[0] unconditionally, causing IndexError: list index out of range.

Fix: Safe access with fallback to 0 when ctx_range is empty.

2. Contiguous PA decode buckets incorrectly filtered (GAUDISW-248598)

The ctx filter was applied to contiguous PA decode buckets, incorrectly dropping valid buckets. For example, with max_model_len=2048, block_size=256, max_num_seqs=256, bucket (256, 1, 2112) was filtered because 2112 > ceil(2048/256)*256 = 2048, but 2112 is a valid user-configured VLLM_DECODE_BLOCK_BUCKET_MAX.

Fix: Remove the ctx filter from contiguous PA decode buckets. For contiguous PA, the block range is already bounded by max_blocks in the bucketing strategies.

Tests

• Added test_file_buckets_with_empty_ctx_range_no_crash — reproduces the server.log IndexError
• Added test_contiguous_pa_decode_buckets_not_filtered_by_ctx — reproduces the std_out.txt issue
• Narrowed test_decode_buckets_satisfy_ctx_filter to non-contiguous PA only
• Updated docstrings

Signed-off-by: Youlei Yang youlei.yang@intel.com

[Copilot]

Pull request overview

Fixes two regressions introduced in #1122 affecting decode bucket filtering: (1) avoid IndexError when file-based bucketing passes an empty ctx_range, and (2) stop incorrectly filtering valid contiguous-PA decode buckets based on the ctx filter (since contiguous-PA block ranges are already bounded by max_blocks).

Changes:

• Guard ctx_range[0] access in num_ctx_tokens_less_or_equal_batched_max_model_len by deriving a safe ctx_min when ctx_range is empty.
• Remove the ctx filter from contiguous-PA decode bucket filtering.
• Add/adjust unit tests to cover the empty-ctx_range file-bucketing path and contiguous-PA decode bucket retention.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
vllm_gaudi/extension/bucketing/common.py	Makes ctx filtering safe for empty ctx_range and disables ctx filtering for contiguous-PA decode buckets.
tests/unit_tests/test_bucketing.py	Adds regression tests for the two reported issues and scopes ctx-filter assertions to non-contiguous PA.

[github-actions]

✅ CI Passed

All checks passed successfully against the following vllm commit:
54f548e9e58087f0155e4e164e416ad7efdfde6d

[github-actions]

✅ CI Passed

All checks passed successfully against the following vllm commit:
54f548e9e58087f0155e4e164e416ad7efdfde6d

[yangulei]

@iboiko-habana @kamil-kaczor
Please help to review this PR. It has been validated by the reporters for GAUDISW-248598 and GAUDISW-248587.
Thanks!

[adobrzyn]

Finding 1 🔴 Critical · vllm_gaudi/extension/bucketing/common.py:L451

With ctx_min = ctx_range[0] if ctx_range else 0, the file-bucketing path (ctx_range == []) sets ctx_min = 0, so the condition ctx > ctx_min is true for every bucket with ctx > 0 and the filter is now applied to user-provided file buckets — silently rejecting any whose ctx > ceil(max_model_len/block_size) * bs.

Worked example using parameters from the new test_file_buckets_with_empty_ctx_range_no_crash (max_model_len=131072, block_size=528, non-contiguous PA, file_buckets=[(1,1,256), (1,1,512), (2,1,256), (2,1,512), (32,1,2424)]):

• ceil(131072/528) = 249
• (1,1,256): 1*249=249, 256 > 249 → dropped
• (1,1,512): 512 > 249 → dropped
• (2,1,256): 2*249=498, 256 ≤ 498 → kept
• (2,1,512): 512 > 498 → dropped
• (32,1,2424): 32*249=7968, 2424 ≤ 7968 → kept

3 of 5 explicitly-configured file buckets are silently filtered out. The new test only asserts len(buckets) > 0, which masks this. The num_ctx_tokens_less_or_equal_batched_max_model_len filter from #1122 was designed to limit auto-generated warmup explosion — it should not be silently rejecting buckets the user explicitly configured via VLLM_BUCKETING_FROM_FILE.

Suggestion: Skip filters entirely for file-provided buckets (file = explicit user intent), e.g. in generate_buckets:

filters = [] if file_buckets else get_filters(is_prompt, use_merged_prefill, use_contiguous_pa)

This preserves #1122's filter for auto-generated buckets (its actual target) and avoids any behavior change on the file path beyond "don't crash". Also strengthen the regression test to assert all five file buckets are preserved:

assert len(buckets) == len(file_buckets)
for fb in file_buckets:
    assert fb in buckets

---

[- Reviewed by Awesome ChlOpus]

[yangulei]

Finding 1 🔴 Critical · vllm_gaudi/extension/bucketing/common.py:L451

With ctx_min = ctx_range[0] if ctx_range else 0, the file-bucketing path (ctx_range == []) sets ctx_min = 0, so the condition ctx > ctx_min is true for every bucket with ctx > 0 and the filter is now applied to user-provided file buckets — silently rejecting any whose ctx > ceil(max_model_len/block_size) * bs.

Worked example using parameters from the new test_file_buckets_with_empty_ctx_range_no_crash (max_model_len=131072, block_size=528, non-contiguous PA, file_buckets=[(1,1,256), (1,1,512), (2,1,256), (2,1,512), (32,1,2424)]):

• ceil(131072/528) = 249
• (1,1,256): 1*249=249, 256 > 249 → dropped
• (1,1,512): 512 > 249 → dropped
• (2,1,256): 2*249=498, 256 ≤ 498 → kept
• (2,1,512): 512 > 498 → dropped
• (32,1,2424): 32*249=7968, 2424 ≤ 7968 → kept

3 of 5 explicitly-configured file buckets are silently filtered out. The new test only asserts len(buckets) > 0, which masks this. The num_ctx_tokens_less_or_equal_batched_max_model_len filter from #1122 was designed to limit auto-generated warmup explosion — it should not be silently rejecting buckets the user explicitly configured via VLLM_BUCKETING_FROM_FILE.

Suggestion: Skip filters entirely for file-provided buckets (file = explicit user intent), e.g. in generate_buckets:

filters = [] if file_buckets else get_filters(is_prompt, use_merged_prefill, use_contiguous_pa)

This preserves #1122's filter for auto-generated buckets (its actual target) and avoids any behavior change on the file path beyond "don't crash". Also strengthen the regression test to assert all five file buckets are preserved:

assert len(buckets) == len(file_buckets)
for fb in file_buckets:
    assert fb in buckets

[- Reviewed by Awesome ChlOpus]

Done by Skip filters for file-based bucketing and refactor tests.

[github-actions]

✅ CI Passed

All checks passed successfully against the following vllm commit:
54f548e9e58087f0155e4e164e416ad7efdfde6d

[kamil-kaczor]

lgtm