Skip to content

[CI] Fix vLLM integration CI (fp8 MoE, continue decode, routed-experts tests) - #3924

Merged
kwang3939 merged 1 commit into
mainfrom
kwang3939/fix_ci_integration
Oct 28, 2026
Merged

kwang3939 merged 1 commit into
mainfrom
kwang3939/fix_ci_integration

Conversation

@kwang3939

Copy link
Copy Markdown
Collaborator

Description

Fixes three breakages against upstream vLLM HEAD:

  1. fp8 MoE AttributeError (DeepSeek torchax E2E): vLLM #57092 reads self.per_act_token_quant in Fp8MoEMethod.get_fused_moe_quant_config. VllmFp8MoEMethod replaces vLLM's __init__, so the attribute was never set. Set it to False; the TPU kernels do not use the resulting quant config.
  2. Continue Decode placeholder underflow (test_sched_utils.py): vLLM #61589 moved num_output_placeholders -= N and its assert from AsyncScheduler into the base Scheduler, so it now runs before the patch's post-call compensation. Pre-compensate by N - 1 before calling the base instead, which works on both LKG and HEAD. Also fix the test fixture so it does not pin the unpatched base onto AsyncScheduler.
  3. Routed-experts connector tests (test_routed_experts_slots.py): vLLM #62591 changed AuxOutputSchedulerConnector.__init__ to take a KVCacheManager. Pass a minimal stub from the test helper. Test-only.

Tests

Tested with:

python3 -m pytest -s -v tests/layers/vllm/test_fp8.py::test_fused_moe
python3 -m pytest tests/core/test_sched_utils.py tests/runner/test_routed_experts_slots.py

Checklist

Before submitting this PR, please make sure:

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have made or will make corresponding changes to any relevant documentation.

Signed-off-by: Kewei Wang <keweiwang@google.com>
@github-actions

Copy link
Copy Markdown

Description

Start with a short description of what the PR does and how this is a change from
the past.

The rest of the description includes relevant details and context, examples:

  • why is this change being made,
  • the problem being solved and any relevant context,
  • why this is a good solution,
  • some information about the specific implementation,
  • shortcomings of the solution and possible future improvements.

If the change fixes a Github issue, please include a link, e.g.,:
FIXES: #123456

Tests

Please describe how you tested this change, and include any instructions and/or
commands to reproduce.

Checklist

Before submitting this PR, please make sure:

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have made or will make corresponding changes to any relevant documentation.

@kwang3939
kwang3939 enabled auto-merge (squash) October 28, 2026 18:52
@kwang3939
kwang3939 merged commit 7840339 into main Oct 28, 2026
53 of 54 checks passed
wenxindongwork added a commit that referenced this pull request Oct 29, 2026
…n engines on failure

- Bump .buildkite/vllm_lkg.version 2fb320d8 -> 4541fcdd (vLLM main as of
  2026-10-29 17:07 UTC). The fp8 MoE `per_act_token_quant` and continue-decode
  scheduler incompatibilities reported from tpu-vllm-integration build #151
  were already fixed on main by #3924; today's integration build #157 (main
  + vLLM HEAD) passes the JAX unit tests and all Runai Model Streamer
  correctness tests, so this moves the pin forward to the current head.
- tests/e2e/test_runai_model_streamer_loader.py: wrap generate() in
  try/finally so the engine core is always shut down. Previously a failure
  in the first LLM left its EngineCore process alive, and the second LLM in
  the same test then died with EngineDeadError, masking the real error.

Signed-off-by: wenxindongwork <wenxindong@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants