Skip to content

[CI] Pass the scale-out endpoint flag in EC E2E - #56819

Merged
DarkLight1337 merged 2 commits into
vllm-project:mainfrom
khluu:agent/fix-scaleout-enable-flag
Sep 15, 2026
Merged

DarkLight1337 merged 2 commits into
vllm-project:mainfrom
khluu:agent/fix-scaleout-enable-flag

Conversation

@khluu

@khluu khluu commented Sep 14, 2026

Copy link
Copy Markdown
Member

Summary

  • replace the removed VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 setting in the scale-out EC E2E launcher
  • pass --enable-scale-out to both ordinary vllm serve processes that need /inference/v1/generate

Why

Merged PR #55176 replaced the environment gate with the vllm serve --enable-scale-out flag, but the L4 Scale-out EC E2E script still uses the removed environment variable. Main build #88790 therefore started healthy encode and prefill servers without the generate route. Both the original job and its automatic retry failed the first encode request with HTTP 404.

The exact Scale-out EC job did not run in PR #55176's green build #88768. This script is one of the job's direct source dependencies, so this change also forces exact PR-CI coverage of the repaired invocation.

Duplicate check

This is not a duplicate. I reviewed issue #55117 and its comments, searched open PRs by the issue number and by enable-scale-out, Scale-out EC Connector E2E, and the 404 signature. The nearby open results concern Rust route support or unrelated scale-out features; none updates this E2E launcher.

Validation

  • bash -n tests/entrypoints/scale_out/ec_integration/run_scale_out_ec_e2e_test.sh
  • shellcheck tests/entrypoints/scale_out/ec_integration/run_scale_out_ec_e2e_test.sh
  • pre-commit run --files tests/entrypoints/scale_out/ec_integration/run_scale_out_ec_e2e_test.sh
  • git diff --check
  • static assertion: exactly two --enable-scale-out arguments and no removed environment-variable reference remain in the script

All pass locally. The exact two-L4 E2E job is required before merge. No model evaluation was run because this changes only the CI harness invocation, not serving output or model code.

AI assistance

OpenAI Codex was used to diagnose the main-CI failure, prepare this patch, and run the listed checks. A human submitter must review every changed line and understand the endpoint-registration behavior before merge.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
@khluu

khluu commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Exact hardware validation is terminal green at c3bf36938241ec1d092147fffdcd57254d32feb6.

  • Build: #88826 — passed
  • Exact lane: L4 Scale-out EC Connector E2E — exit 0 in 3m45s
  • Evidence: baseline and scale-out services started successfully; the single-image, two-image, and text-only scale-out outputs all matched the baseline.

Lightweight checks and the exact two-L4 machine gate are now green. The remaining gate is human review.

Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
@mergify

mergify Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Documentation preview: https://vllm--56819.org.readthedocs.build/en/56819/

@mergify mergify Bot added the documentation Improvements or additions to documentation label Sep 14, 2026
@tlrmchlsmth
tlrmchlsmth marked this pull request as ready for review September 14, 2026 16:57

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@tlrmchlsmth

Copy link
Copy Markdown
Member

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #88875 for commit b1452f4c1d39.

@khluu

khluu commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Fresh main evidence strengthens the gate for this fix:

The isolated exact lane on this PR remains green in #88826. This is now repeated deterministic main breakage, not a one-off runner failure.

@DarkLight1337
DarkLight1337 merged commit caf56ce into vllm-project:main Sep 15, 2026
32 checks passed
ItsRoy69 pushed a commit to ItsRoy69/vllm that referenced this pull request Sep 15, 2026
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
keneoneth pushed a commit to keneoneth/vllm that referenced this pull request Sep 16, 2026
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants