Fix test_kv_sharing_fast_prefill flakiness

[sarckk]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Fixes #22104

This unit test, added in #21590, checks for exact outputs from two LLM instances. However, due to inherent randomness in vLLM (e.g. scheduling), sometimes the test fails, making it flaky. Following vLLM's reproducibility guide, make this less flaky.

The key was to turn off multiprocessing to make scheduling deterministic with VLLM_ENABLE_V1_MULTIPROCESSING=0. However, this requires further changes to release GPU memory for second LLM instance (as unlike with multiprocessing, there is no separate worker process that gets killed to free memory).

Follow up required: had to disable enforce_eager=False test case as deleting model and KV caches ref doesn't seem to remove all references, so memory doesn't get freed.

Test Plan

Ran pytest tests/v1/e2e/test_kv_sharing_fast_prefill.py::test_kv_sharing_fast_prefill 10 times

Test Result

Before PR, some runs would fail. After PR, all 10 runs pass.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request effectively addresses the flakiness of test_kv_sharing_fast_prefill by enforcing deterministic execution. The method of disabling multiprocessing and setting seeds is appropriate for achieving reproducibility.

My primary feedback concerns the new cleanup function. While it solves the immediate memory release issue, its approach of directly manipulating deep internal state of the LLM engine is fragile and creates a maintenance risk. I've added a comment recommending the creation of a proper teardown API on the LLM class as a more robust long-term solution. The current implementation is acceptable as a temporary measure to fix the test, but this technical debt should be addressed.

[sarckk]

@DarkLight1337 is there a recommended way to release GPU memory between non-multiprocessing LLM instances? I tried solutions in:

#1908
#16543
#6544
#3281

but none of them worked, and the only thing that worked was a hacky solution (see gemini comment above) that involves deleting references to model and KV caches from the model runner.

[sarckk]

CI test failure seems unrelated:

[2025-08-01T22:33:04Z] ______________ ERROR collecting tests/v1/engine/test_async_llm.py ______________
--
  | [2025-08-01T22:33:04Z] ImportError while importing test module '/vllm-workspace/tests/v1/engine/test_async_llm.py'.
  | [2025-08-01T22:33:04Z] Hint: make sure your test modules/packages have valid Python names.
  | [2025-08-01T22:33:04Z] Traceback:
  | [2025-08-01T22:33:04Z] /usr/lib/python3.12/importlib/__init__.py:90: in import_module
  | [2025-08-01T22:33:04Z]     return _bootstrap._gcd_import(name[level:], package, level)
  | [2025-08-01T22:33:04Z] v1/engine/test_async_llm.py:19: in <module>
  | [2025-08-01T22:33:04Z]     from vllm.v1.engine.async_llm import AsyncLLM
  | [2025-08-01T22:33:04Z] /usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py:32: in <module>
  | [2025-08-01T22:33:04Z]     from vllm.v1.engine.core_client import EngineCoreClient
  | [2025-08-01T22:33:04Z] /usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py:32: in <module>
  | [2025-08-01T22:33:04Z]     from vllm.v1.engine.core import EngineCore, EngineCoreProc
  | [2025-08-01T22:33:04Z] /usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py:22: in <module>
  | [2025-08-01T22:33:04Z]     from vllm.executor.multiproc_worker_utils import _add_prefix
  | [2025-08-01T22:33:04Z] E   ImportError: cannot import name '_add_prefix' from 'vllm.executor.multiproc_worker_utils' (/usr/local/lib/python3.12/dist-packages/vllm/executor/multiproc_worker_utils.py)

[DarkLight1337]

Can you merge from main to fix CI?