Repository navigation
feat(epp): data-parallel worker registration in the standalone reflector - #22
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
Phase 4 of the runtime-optional frontend roadmap: the EPP pod reflector is the Kubernetes catalog feeder, and it registered every pod as a single data-parallel rank (the hardcoded data_parallel_size: 1 in selector.rs). - EppStandaloneConfig gains DYN_EPP_DATA_PARALLEL_SIZE (default 1) and DYN_EPP_KV_EVENT_PORT_STRIDE (default 1). Rank r publishes KV events on kv_event_port + r * stride; the reflector emits one KV-event endpoint per rank (RawWorker.kv_events_endpoints) and the registration carries data_parallel_size, which the selector passes through with data_parallel_start_rank 0. - Adds the session_context field the EPP's booking request was missing since the structured session context landed in the selection service, so the ext-proc crate builds again on this stack. Tests: strided per-rank endpoints for a 3-rank pod; existing reflector, adapter, and selector tests updated to the per-rank shape (144 pass). Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
had a problem deploying
to
external_collaborator
September 3, 2026 09:18 — with 
GitHub Actions
Failure
GitHub Actions
Failure
github-actions
Bot
added
feat
documentation
Owner
Author
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 4 (Kubernetes feeder) of the runtime-optional frontend roadmap. Stacked on #21.
Summary
The EPP's standalone pod reflector is the Kubernetes catalog feeder, and it registered every pod as one data-parallel rank (
selector.rshardcodeddata_parallel_size: Some(1), the roadmap'sselector.rs:199-201anchor).EppStandaloneConfig:DYN_EPP_DATA_PARALLEL_SIZE(default 1, validated >= 1) andDYN_EPP_KV_EVENT_PORT_STRIDE(default 1). Rankrpublishes KV events onkv_event_port + r * stride.pod_discovery:RawWorker.kv_events_endpointsis now per rank (KvEventPorts::endpoints), IPv6 bracketing preserved.topology_adapter/selector:WorkerRegistration.data_parallel_sizeflows into the coreWorkerRequestwithdata_parallel_start_rank: 0.session_context: Nonefield the EPP'sSelectAndReserveRequestliteral was missing since feat(kv-router): wire shared-cache hits and full session context into selection #9 introduced the structured session context; thedynamo-ext-proccrate did not compile on the stack until this commit (none of the earlier PRs' validation built it). Flagging so it can be folded into feat(kv-router): wire shared-cache hits and full session context into selection #9 if preferred.vanilla-vllm-onramp.mdxenv table.Validation
cargo test -p dynamo-ext-proc: 144 + 1 passed (newdata_parallel_ranks_publish_on_strided_ports; reflector/adapter/selector tests updated to the per-rank shape).cargo clippy -p dynamo-ext-proc --all-targets: clean.cargo fmt,docs_lint.pyclean.