[CI] Fix custome offline ci issue, V1 Others Test bug#34913
[CI] Fix custome offline ci issue, V1 Others Test bug#34913yewentao256 wants to merge 2 commits intomainfrom
V1 Others Test bug#34913Conversation
Signed-off-by: yewentao256 <zhyanwentao@126.com>
There was a problem hiding this comment.
Code Review
This pull request addresses a CI failure in the custom offline logits processor tests. The issue stems from vLLM's automatic attention backend selection triggering CUDA initialization, which then forces the multiprocessing start method to 'spawn', conflicting with the test's requirement for 'fork'. The fix cleverly circumvents this by explicitly setting the attention backend to 'TRITON_ATTN'. Additionally, the PR commendably improves the test's correctness by propagating extra keyword arguments to the reference LLM instance, ensuring a more accurate comparison. The changes are sound and effective. I have one suggestion to further improve the code's maintainability by removing a magic string.
|
This will be fixed by #34818 |
|
@MatthewBonanni Thanks, I am thinking we can still have this PR just in case we might have similar issue in the future, |
|
34818 allows this test to run with I feel that this PR introduces a false pass. As it exists on main, the test correctly fails and catches a bug (even though it's not necessarily what the test was designed to catch) - this PR prevents it from failing and catching the bug |
|
@MatthewBonanni OK, please combine this diff in your PR. |
Purpose
Fixes https://buildkite.com/vllm/ci/builds/52111/steps/canvas?jid=019c71de-e90f-46a7-9da5-cfea9f4a0660
We need to
fork, but if we let vLLM automatically choose attn backend,It will be set to spawn, which causes the issue
Test