[Test] Add worker tests for hidden-state prefix cache

[hongzhi-gao]

Purpose

Add worker-level correctness tests for hidden-state prefix caching (OmniTensorPrefixCache), aligned with the RFC goal of returning full hidden-state sequences (cached prefix + newly computed tokens) for downstream use when prefix caching is enabled (#2164).

This PR does not change runtime behavior; it reduces regression risk for the merge path (GPU → CPU coercion, block-table lookup) and for dual prefix-hit batches, which are not fully covered by tests/core/test_prefix_cache.py.

Test Plan

• Files added: tests/worker/test_hidden_state_prefix_cache.py
• Commands (local):

  pytest -q tests/worker/test_hidden_state_prefix_cache.py
  # Optional: CPU-only (skip CUDA test), e.g. environments without a GPU:
  pytest -q tests/worker/test_hidden_state_prefix_cache.py -m "not cuda"

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

BLOCKER scan (test-only PR):

• Correctness of test logic: PASS

The test coverage looks good:

• Proper mocking of vLLM's InputBatch.block_table
• Tests GPU → CPU coercion for cache operations
• Tests dual prefix-hit scenarios (shared and distinct block tables)
• Appropriate pytest markers (cuda, cpu, L4)
• Clear assertions for both shape and value correctness

VERDICT: COMMENT (no blockers)

---

Minor nit: The test file path uses but the test is in . Consider if this should be or if is appropriate here.

[amy-why-3459]

Thank you very much for your contribution. @yenuo26 PTAL

[hongzhi-gao]

BLOCKER scan (test-only PR):

• Correctness of test logic: PASS

The test coverage looks good:

• Proper mocking of vLLM's InputBatch.block_table
• Tests GPU → CPU coercion for cache operations
• Tests dual prefix-hit scenarios (shared and distinct block tables)
• Appropriate pytest markers (cuda, cpu, L4)
• Clear assertions for both shape and value correctness

VERDICT: COMMENT (no blockers)

Minor nit: The test file path uses but the test is in . Consider if this should be or if is appropriate here.

Thanks for the nit! I moved the test from tests/worker/test_hidden_state_prefix_cache.py to tests/core/test_hidden_state_prefix_cache.py to align with layer ownership.
Test logic/coverage remains unchanged. PTAL again. @hsliuustc0106

[hongzhi-gao]

Thank you very much for your contribution. @yenuo26 PTAL

Thanks for the review and coordination.

[Copilot]

Pull request overview

Adds new correctness tests covering hidden-state prefix caching merge behavior in OmniTensorPrefixCache, focusing on the “cached prefix + newly computed tail” merge path and multi-request prefix-hit scenarios.

Changes:

• Add a CUDA-focused test that exercises GPU→CPU coercion during cache update/read and validates merged hidden-state sequence correctness.
• Add a CPU test that validates dual prefix-hit behavior for requests sharing the same cached blocks vs using distinct cached blocks.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.