Repository navigation
[NIXL] Drop unreachable producer-side notif counting in push mode - #62210
Merged
njhill merged 2 commits intoOct 17, 2026
Merged
Conversation
In push mode, the only notifs D sends to P are PUSH_REG, which the writer thread consumes, and HB, which _get_new_notifs handles before the counting. Only pull_worker.py fills _pending_recv_notifs, so no code path sends P a plain "<req_id>:<n>" notif. The producer-side branch of NixlPushConnectorWorker._get_new_notifs was carried over from pull mode and never runs: P reports finished sends from its WRITE handles in get_transfer_results. Remove the branch, its guard, the transfer_topo assert it needed and the counter cleanup for finished pushes. _get_new_notifs now counts only consumer-side completion notifs and always returns an empty set. A completion notif for a request that is also being sent (one request both loading and saving on a kv_both instance) is now counted as a load instead of as a send. Tests ----- test_nixl_push_connector.py no longer sets transfer_topo or checks the return value of _get_new_notifs; the consumer-side assertions are unchanged. Validation ---------- - test_nixl_push_connector.py and test_nixl_heartbeat.py: 103 passed. test_nixl_connector.py: 182 passed; the two test_abort_timeout_on_prefiller cases fail locally on main as well. - With a tripwire in the removed branch, the push, heartbeat and NIXL connector unit tests pass without reaching it. - On GPU, the tripwire was not reached in end-to-end push runs with TP1 -> TP1, producer TP2 -> consumer TP1 and producer TP1 -> consumer TP2, and those runs met their expected results. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: David Cheung <d7cheung@gmail.com>
sawsa307
requested review from
ApostaC,
NickLucche,
ivanium,
orozery,
wzhao18 and
xuechendi
as code owners
October 17, 2026 18:38
Contributor
Author
|
cc @njhill |
2 of 6 tasks
njhill
approved these changes
Oct 17, 2026
Member
|
/ci run |
|
✅ @sawsa307, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #95617 for commit ℹ️ This PR is 4 commits behind upstream |
njhill
enabled auto-merge (squash)
October 17, 2026 19:11
michael-gpu
pushed a commit
to practical-inference/vllm-ampere-extended
that referenced
this pull request
Oct 20, 2026
…lm-project#62210) Signed-off-by: David Cheung <d7cheung@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: vllm-agent <claw@inferact.ai>
sohom-cs
added a commit
to sohom-cs/vllm
that referenced
this pull request
Oct 29, 2026
Push mode learns of finished sends from its WRITE handles (vllm-project#62210), so the push test now completes rank 0's WRITE instead of mocking _get_new_notifs. The pull test sends rank 0 a consumer notification through the NIXL wrapper instead of patching the private method. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Sohom Chakraborty <16609933+sohom-cs@users.noreply.github.com>
5 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Remove the producer-side branch of
NixlPushConnectorWorker._get_new_notifs, which no push-mode code path can reach. Split out of #61488 / #61580 as an independent simplification, as suggested in #61580 (comment).Claims
"<req_id>:<n>"notif arriving at P, which push mode never sends._get_new_notifsnow only counts consumer-side completion notifs and always returns an empty set; P keeps reporting finished sends from its WRITE handles inget_transfer_results.Validation
pytest tests/v1/kv_connector/unit/test_nixl_push_connector.py tests/v1/kv_connector/unit/test_nixl_heartbeat.py: 103 passed.pytest tests/v1/kv_connector/unit/test_nixl_connector.py: 182 passed. The twotest_abort_timeout_on_prefillercases fail locally on main as well.push_worker.pychange on the [KV Transfer] Connector-agnostic KV checksums for P/D disaggregation #61488 branch.Details
In push mode, D sends P only two kinds of notifs:
PUSH_REG, consumed by the writer thread before notifs are forwarded to_get_new_notifs;HB:, handled at the top of_get_new_notifsbefore the counting.Plain
"<req_id>:<n>"notifs to P come from_pending_recv_notifs, which onlypull_worker.pyfills. P → D completion notifs are attached to the push WRITEs and are counted on D, which is the branch this PR keeps.The removed code is the producer-side counting copied from pull mode, the
transfer_topoassert it needed, and the counter cleanup for finished pushes inget_transfer_results. One edge case changes: on akv_bothinstance where one request is both loading and saving, a completion notif for it is now counted as a load instead of a send. Push mode never sends such a notif to the producer side.Not a duplicate: #55470 also edits
_get_new_notifs(it adds a failed-WRITE notif path) but keeps the producer-side branch. Whichever merges second needs a small rebase.Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
🤖 Generated with Claude Code