[CI/Build] Temporary fix to LM Eval Small Models

[zhewenl]

Purpose

This PR provides a temp fix to broken CI: LM Eval Small Models, the test is failing with GSM8K evals on Qwen1.5-MoE-W4A16-CT-tp1:

[2025-11-05T06:16:56Z]             # Verify accuracy is within tolerance
--
  | [2025-11-05T06:16:56Z] >           assert measured_accuracy >= expected_accuracy - RTOL, (
  | [2025-11-05T06:16:56Z]                 f"Accuracy too low: {measured_accuracy:.3f} < "
  | [2025-11-05T06:16:56Z]                 f"{expected_accuracy:.3f} - {RTOL:.3f}"
  | [2025-11-05T06:16:56Z]             )
  | [2025-11-05T06:16:56Z] E           AssertionError: Accuracy too low: 0.000 < 0.450 - 0.080
  | [2025-11-05T06:16:56Z] E           assert 0.0 >= (0.45 - 0.08)
  | [2025-11-05T06:16:56Z]
  | [2025-11-05T06:16:56Z] evals/gsm8k/test_gsm8k_correctness.py:86: AssertionError

After bisecting, we found this PR could contribute to failing tests: #26440 - disabling the feature with VLLM_DISABLE_SHARED_EXPERTS_STREAM=1 can make test passed consistently.

Note: this is only easy to reproduce on Nvidia L4(what CI is using), H100/MI300 is very difficult to repro this: pytest -s -v 'tests/evals/gsm8k/test_gsm8k_correctness.py::test_gsm8k_correctness_param[Qwen1.5-MoE-W4A16-CT-tp1]'

Some hypothesis on why is this failing:

• Quantization Incompatibility? The test is using W4A16 quantization, so the previous PR might break tensor properties with hidden_states.clone()?
• It might be breaking memory layout in Qwen2 MoE model forward pass?

Test Plan

With VLLM_DISABLE_SHARED_EXPERTS_STREAM=1, tested 10 times(10/10 are passing): https://gist.github.com/zhewenl/a23ead4262f500b764ecd56c8c4c213d

Without disabling the env var, 3/3 are failing: https://gist.github.com/zhewenl/01a70feea4de15f6258dbecd76415a22

[zhewenl]

On a side note, since the root cause and the impact is still unknown - shall we temporally disable VLLM_DISABLE_SHARED_EXPERTS_STREAM ? It's enabled globally as default:

vllm/vllm/envs.py

Line 221 in da786e3

VLLM_DISABLE_SHARED_EXPERTS_STREAM: bool = False

[vadiklyutiy]

Do I understand correctly that we add VLLM_DISABLE_SHARED_EXPERTS_STREAM=1 for all tests?

[zhewenl]

Do I understand correctly that we add VLLM_DISABLE_SHARED_EXPERTS_STREAM=1 for all tests?

For all the tests under LM Eval Small Models

[yeqcharlotte]

ok does this mean the failure is specific to L4 gpu? i think the duo stream still need some ci coverage.

can we reorganize these testing pipelines so we can separate testing that must be run on H100/B200 vs. L4. @hl475 @rzabarazesh please consider these in the test refactoring!

[hl475]

thanks!

not sure if feasible - can we add something like env: VLLM_DISABLE_SHARED_EXPERTS_STREAM=1 into https://github.com/vllm-project/vllm/blob/b158df28139d134c2a43680104418eaa0d58e91c/tests/evals/gsm8k/configs/Qwen1.5-MoE-W4A16-CT.yaml , and then propagate it to the pytest which can use monkeypatch for the environment variables ? In this way, we can make the impact only to this test.

[mgoin]

Can you please add it to the Blackwell lm eval job too?

[vadiklyutiy]

I think we should add it on pytest level. Otherwise running tests in CI/buildkite environment will be different from running pytest locally.

[vadiklyutiy]

I can confirm that adding VLLM_DISABLE_SHARED_EXPERTS_STREAM=1 fix the issue(did a lot of runs).

[zhewenl]

@mgoin / @hl475 / @vadiklyutiy thanks for your suggestions! Updated the logic to take in env vars so both L4/Blackwell can use the same config wo/ setting it in different test suite.