[EC Connector] Added Build Connector Worker Meta for EC Connector by omerpaz95 · Pull Request #49585 · vllm-project/vllm

omerpaz95 · 2026-07-23T12:33:50Z

Purpose

Why this is still needed after #38390. The PR implemented the V2 model runner EC Connector, but EC still has no worker -> scheduler metadata channel. KV connectors have a complete one: build_connector_worker_meta() -> KVConnectorOutput.kv_connector_worker_meta -> KVOutputAggregator -> scheduler-side KV connector.
EC had none of those three pieces, so a worker-sideECConnector has no way to report per-step state — e.g. "this mm_hash is now durably stored" — back to its scheduler-side EC connector. That blocks the CPU-offload based on CUDA events and NIXL P2P EC connectors, which need exactly this report (We are retiring the StepTracker in the CPU EC Connector, because using CUDA events is the right way to handle "copy done" notifications, instead of waiting until the batch is done).

This PR adds that channel, and fixes the exit paths where the worker's ec_connector_output was computed and then dropped before reaching the scheduler.

The channel

ec_connector/base.py: ECConnectorWorkerMetadata ABC with an abstract aggregate(), plus ECConnectorBase.build_connector_worker_meta().
v1/outputs.py: ec_connector_worker_meta field on ECConnectorOutput.
ec_connector/utils.py: ECOutputAggregator, mirroring KVOutputAggregator - merges the EC output of the rank that ran the connector onto the rank whose ModelRunnerOutput reaches the scheduler.
engine/core.py, executor/abstract.py, executor/multiproc_executor.py: wiring so the aggregator runs when an EC connector is configured.
Both runners (MRv1, MRv2) populate the field via build_connector_worker_meta().

Default behavior is unchanged for existing connectors: nothing overrides
build_connector_worker_meta() yet, so it returns None.

Test Plan

# Existing EC connector unit tests (unaffected areas)
.venv/bin/python -m pytest tests/v1/ec_connector/unit/test_metadata.py \
  tests/v1/ec_connector/unit/test_ec_example_connector.py \
  tests/v1/ec_connector/unit/cpu/test_connector.py -v

## Test Result

tests/v1/ec_connector/unit/test_metadata.py ..................... PASSED
tests/v1/ec_connector/unit/test_ec_example_connector.py ......... PASSED
tests/v1/ec_connector/unit/cpu/test_connector.py ................ PASSED
36 passed, 2 warnings

pre-commit: ruff check / ruff format / mypy / SPDX headers / etc. — all Passed

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Signed-off-by: yewentao256 <zhyanwentao@126.com>

…rker_meta

yewentao256

Let's wait until #38390 landed
Same for #49579 and #49582

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

yewentao256

LGTM, thanks for the iterations!

gty111 · 2026-08-14T00:20:23Z

/ci run

github-actions · 2026-08-14T00:20:41Z

✅ Triggered Buildkite CI #83812 for commit 9ac0d01b7894.

gty111 · 2026-08-14T00:54:10Z

The failed CI is related to this PR. Need to fix the tests.

mergify · 2026-08-14T06:33:16Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @omerpaz95.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

…rker_meta Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

gty111 · 2026-08-14T11:09:11Z

/ci run

github-actions · 2026-08-14T11:09:28Z

✅ Triggered Buildkite CI #83886 for commit 9c13dbc473d2.

gty111 · 2026-08-14T14:30:45Z

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

gty111 · 2026-08-14T14:52:15Z

/ci run

github-actions · 2026-08-14T14:52:34Z

✅ Triggered Buildkite CI #83938 for commit 2cf705c8a108.

…rker_meta

gty111 · 2026-08-14T22:40:54Z

/ci retry

github-actions · 2026-08-14T22:41:11Z

✅ Triggered Buildkite CI #83985 for commit c8a9b6eb66f7, running 1 failed step(s) from Buildkite CI #83938.

…lm-project#49585) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: omerpaz95 <omerpaz95@gmail.com> Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Co-authored-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

…lm-project#49585) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: omerpaz95 <omerpaz95@gmail.com> Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Co-authored-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Signed-off-by: Wyett <wyettzeng@gmail.com>

…lm-project#49585) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: omerpaz95 <omerpaz95@gmail.com> Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Co-authored-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>

…lm-project#49585) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: omerpaz95 <omerpaz95@gmail.com> Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Co-authored-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Signed-off-by: khushali9 <khushali.desai9@gmail.com>

yewentao256 and others added 14 commits March 27, 2026 19:19

e/p/d disaggregation support for MRv2

2c2c65e

Signed-off-by: yewentao256 <zhyanwentao@126.com>

update

d5ad859

Signed-off-by: yewentao256 <zhyanwentao@126.com>

fix precommit

e434e5d

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Merge branch 'main' into wentao-epd-support-for-MRv2

920e7fb

Merge branch 'main' into wentao-epd-support-for-MRv2

b66ab0f

Merge branch 'main' into wentao-epd-support-for-MRv2

135b67b

Merge branch 'main' into wentao-epd-support-for-MRv2

7deae32

Merge branch 'main' into wentao-epd-support-for-MRv2

473f798

Merge branch 'main' into wentao-epd-support-for-MRv2

db887e7

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Merge branch 'main' into wentao-epd-support-for-MRv2

848e821

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Merge branch 'main' into wentao-epd-support-for-MRv2

e794c38

Merge branch 'main' into wentao-epd-support-for-MRv2

a23f528

Signed-off-by: yewentao256 <zhyanwentao@126.com>

update

d7e63e8

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Added Build Conector Worker Meta for EC Connector

43d903d

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

omerpaz95 requested a review from njhill as a code owner July 23, 2026 12:33

claude Bot reviewed Jul 23, 2026

View reviewed changes

mergify Bot added the v1 label Jul 23, 2026

yewentao256 and others added 3 commits July 23, 2026 14:47

reduce mrv2 change

251c90e

Signed-off-by: yewentao256 <zhyanwentao@126.com>

reduce mrv2 code change

69b33fe

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Merge branch 'main' into wentao-epd-support-for-MRv2

6c5090a

orozery added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 27, 2026

omerpaz95 added 2 commits July 27, 2026 18:44

Merge branch 'main' into worktree-ec_connector_add_build_connector_wo…

842d1d0

…rker_meta

Merge branch 'main' into worktree-ec_connector_add_build_connector_wo…

ab7683d

…rker_meta

yewentao256 reviewed Jul 28, 2026

View reviewed changes

Added a path for the empty step.

c3cf0f5

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

depthfirst-app Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread vllm/v1/worker/gpu_model_runner.py Outdated

omerpaz95 added 4 commits July 28, 2026 18:32

Added ECOutputAggregator for multiproc support.

3e22b0b

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

Corrected PP semantics in EC output aggregation.

397cacc

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

More bugfixing.

40b30ac

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

Fix ec_connector_output dropped on producer-only encoder-only step.

5cb71cc

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

removed a test.

9ac0d01

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

yewentao256 approved these changes Aug 13, 2026

View reviewed changes

mergify Bot added the needs-rebase label Aug 14, 2026

Fix tests to pass ci.

b0004d2

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

omerpaz95 requested review from ApostaC, ivanium and orozery as code owners August 14, 2026 10:42

Merge branch 'main' into worktree-ec_connector_add_build_connector_wo…

9c13dbc

…rker_meta Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

mergify Bot removed the needs-rebase label Aug 14, 2026

Fix eplb CI

2cf705c

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>

Merge branch 'main' into worktree-ec_connector_add_build_connector_wo…

c8a9b6e

…rker_meta

gty111 mentioned this pull request Aug 15, 2026

EPD (EC connector) Tracker #52409

Open

ywang96 merged commit fa9d67f into vllm-project:main Aug 15, 2026
9 of 11 checks passed

sierraisland mentioned this pull request Aug 18, 2026

[DPScheduler] Add missing ec_connector attribute to DPScheduler vllm-project/tpu-inference#3418

Merged

Akine-Ko mentioned this pull request Sep 2, 2026

[Refactor][EC Connector] Add backend extension points to ECCPUWorker #54033

Merged

Uh oh!

Uh oh!

Conversation

omerpaz95 commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

The channel

Test Plan

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

yewentao256 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yewentao256 left a comment

Choose a reason for hiding this comment

Uh oh!

gty111 commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

gty111 commented Aug 14, 2026

Uh oh!

mergify Bot commented Aug 14, 2026

Uh oh!

gty111 commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

gty111 commented Aug 14, 2026

Uh oh!

gty111 commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

gty111 commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

omerpaz95 commented Jul 23, 2026 •

edited

Loading