[Core] Add NIXL omni connector

[yuanwu2017]

Purpose

Adds NixlConnector, an optional OmniConnectorBase implementation for moving arbitrary stage payloads across nodes through NIXL.

RFC: #6160

When to Choose It

Use NixlConnector for intra-node or inter-node stage transfer when a deployment already uses NIXL for vLLM P/D, needs UCX transport coverage (RDMA, TCP, and accelerator-native paths), or targets Intel XPU through UCX Level Zero.

Connector	Best fit	Constraint compared with NIXL
SharedMemoryConnector	Colocated stages	POSIX shared memory only; not cross-node
Mooncake connectors	Deployments standardised on Mooncake	Requires Mooncake runtime and its memory model
Mori connector	Deployments standardised on Mori	Requires Mori IOEngine
Yuanrong connectors	Ascend/Yuanrong deployments	NPU- and Datasystem-specific
NixlConnector	Cross-node, NIXL/UCX-enabled deployments	Requires NIXL plus a usable UCX backend and a small ZMQ control plane

NIXL is not a replacement for existing connectors. It remains an optional lazy import and is only constructed when a deployment YAML names it. It implements Omni put/get for arbitrary payloads; it cannot reuse vLLM P/D's NIXL connector because that interface operates on block-table, layer-wise KV semantics.

Design and Deployment Constraints

• Control plane: producer ZMQ ROUTER, consumer REQ. A consumer obtains metadata with GET_META{key} and reports XFER_DONE after successful consumption. Callers that forward put() metadata directly do not need a handshake socket. Ordinary cross-stage payload requests carry the producer endpoint through the scheduler request path.
• Data plane: consumer-initiated NIXL READ. The producer registers source memory; the consumer allocates destination tensors on its own device and pulls data.
• Ports: zmq_port is a base port. The centralized resolver derives a unique endpoint from purpose (request_forwarding, kv_transfer, or orchestrator), producer stage, replica, and TP local rank. This prevents collisions between TP workers and connector purposes.
• Payloads: tensors, structured Python objects with tensor leaves, and serialized objects. Metadata schema v1 groups descriptors by NIXL memory type and retains global tensor indices, so structured payloads may safely combine CPU and accelerator tensors.
• Ownership: source registrations have a configurable lease (lease_seconds, default 3600; VLLM_OMNI_NIXL_LEASE_S). A reaper releases abandoned registrations even on an idle producer.
• Timeout safety: NIXL 1.3 has no supported cancellation API. On a receive timeout, the connector retains buffers, registrations, descriptor lists, transfer handles, and the remote agent until NIXL reaches a terminal state; close() waits before releasing them. This prevents native DMA into deregistered memory.
• Installation: CUDA deployments can install the published nixl package. docker/Dockerfile.xpu uses the upstream vLLM XPU image, which already ships a matched torch-xpu, UCX Level Zero, and NIXL runtime; this PR adds no custom UCX/NIXL wheel build.

Validation Matrix

Scenario	Coverage	Result
ZMQ metadata handshake and completion cleanup	Stubbed NIXL unit tests	Passed
Tensor, object, and structured payload normalization	Stubbed NIXL unit tests	Passed
Mixed CPU/accelerator descriptor grouping	test_structured_payload_groups_descriptors_by_memory_type	Passed
Producer lease expiry while idle	test_idle_producer_lease_expires_without_another_put	Passed
Timeout, deferred native-resource ownership, and shutdown	NIXL timeout lifecycle tests	Passed
TP=1 / TP>1 port isolation	adapter and worker resolver tests	Passed
Heterogeneous TP source-rank endpoint selection	TP rank-aware tests	Passed
Ordinary payload producer endpoint propagation	coordinator and worker endpoint tests	Passed
Intra-node real NIXL transfer	Two-process native NIXL/UCX, GPU 0 to GPU 1	Passed
Inter-node RDMA/TCP transfer	Not run across hosts	Not validated
Real CUDA GPU transfer	Two CUDA devices with native NIXL/UCX	Passed
Real Intel XPU transfer	Not run on target hardware	Not validated
Representative connector integration	Actual OmniPayloadStruct with mixed CPU/GPU leaves	Passed

Validation run in the CUDA test image:

235 passed, 3 subtests passed

Native smoke: 1 passed; focused NIXL and affected transport suites included above. Ruff checks, Ruff format checks, and git diff --check passed.

Non-goals

• Broadcast from one producer to multiple consumers.
• Replacing existing Omni connectors.
• Making NIXL a required dependency.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

open an RFC first

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/omni_connector.md.

Module owners: @princepride @yuanheng-zhao @xuechendi @natureofnature @fake0fan

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[yuanwu2017]

open an RFC first

Ok. Let me finish the integration with #5885.

[yuanwu2017]

open an RFC first

Done — filed as #6160.

I've also filled in the PR description. Two things worth flagging for review:

• This PR is the connector only. The first consumer (MiniMax-H3 text-encoder → diffusion conditioning, on top of [Feature] Support text encoder Disaggregation for H3 #5885) is kept in a separate branch so this one stays reviewable on its own.
• Validate the Wan mode with Feat/wan EG/EGD disagg #6285

[yuanwu2017]

H3 NIXL E2E retest after main integration (2026-09-09)

The successful test used the tree now published as 0a54596 in #6264, stacked on #6093 head 7da3f90. The run used 8702c8009 plus the five-file configuration fix subsequently committed as 0a5459642; the committed diff was compared byte-for-byte with the tested patch. This is integration E2E evidence for the connector together with the downstream stage-payload feature, not a claim that standalone #6093 provides the H3 payload integration.

Results

• Six same-host A100 80GB GPUs; text encoder TP2, diffusion TP4 / Ulysses1 / VAE patch parallel4.
• Directly loaded the repository's vllm_omni/deploy/minimax_h3_disaggregated.yaml, unchanged; only Stage 1 parallel settings were overridden through --stage-overrides. This does not validate the YAML's unmodified TP1/Ulysses4 defaults.
• Original orange-tabby request: 8 steps, seed42, 1344x768, 107 frames, 24 FPS.
• HTTP 200, video/mp4, 5,108,715 bytes, 78.350852 seconds request time.
• H.264 video + AAC audio; encoded duration 4.458333 seconds. Full ffmpeg -xerror decode and preview extraction passed.
• One matching NIXL put/get, 358775 bytes, key video_sync-8be48ef374af9f47-8fca2926_0_0; producer Worker_TP0, consumer DiffusionWorker_TP0.
• Video SHA256: 68b69e1e7d8de6bc3e6c1c7c143f39ed8cddf16f3fc53d32227b7767bcacb34c.
• Expanded regressions on the fixed tree: 1278 passed, 6 skipped, 3 subtests passed. Ruff lint/format and git diff --check passed.
• Separate native NIXL smoke on pre-config-fix 8702c8009: 1 passed in 31.23s. The subsequent five-file fix did not change the native connector implementation.
• All six test GPUs were released after the run.

An earlier attempt failed during text-encoder loading because an external managed model was automatically restarted onto GPU0. The successful retry above ran after that service was stopped. That failed attempt is not counted as a successful E2E.

Update / author self-review

#6264's nine feature commits were restacked onto the current #6093 head, retaining the upstream H3 conditioning contract and native paged-KV request plumbing. The final configuration fix preserves stage payload keys through the centralized diffusion projection, serialization, and runtime mapping, with topology defaults and explicit deploy/CLI precedence (including empty overrides). It also registers/documents the two existing NIXL environment variables and updates their inventory tests. These five-file fixes are in #6264; this comment does not claim they were separately added to #6093. The native paged-KV and full-payload protocols remain distinct.

Reproduction commands

Run from a checkout of #6264 at 0a5459642, not from standalone #6093. Runtime used: local image vllm-omni-pr6093:integration-20260908 (vLLM 0.28.0 / Python 3.12, with native NIXL/UCX installed). This image tag is a local test artifact, not a published pullable image; use an equivalently provisioned runtime if reproducing elsewhere. Set HF_CACHE to the Hugging Face cache containing the pinned H3 snapshot. Physical GPUs 0,1,3,4,5,6 must be free; they map to container logical GPUs 0..5.

export HF_CACHE=/path/to/huggingface/cache
export IMAGE=vllm-omni-pr6093:integration-20260908
mkdir -p h3-e2e-output

docker run -d --name h3-nixl-e2e \
  --gpus '"device=0,1,3,4,5,6"' --network host --ipc host \
  -v "$PWD:/workspace:ro" -v "$PWD/h3-e2e-output:/artifacts" \
  -v "$HF_CACHE:/root/.cache/huggingface:ro" -w /workspace \
  -e PYTHONPATH=/workspace -e PYTHONDONTWRITEBYTECODE=1 \
  -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5 \
  -e HF_HOME=/root/.cache/huggingface -e VLLM_LOGGING_LEVEL=DEBUG \
  -e UCX_RCACHE_MAX_UNRELEASED=1024 \
  "$IMAGE" python -m vllm_omni.entrypoints.cli.main serve \
  /root/.cache/huggingface/hub/models--MiniMaxAI--MiniMax-H3/snapshots/42ed227ee7df40d41602854ae760620d6eb651fe/FL2VA \
  --omni --host 127.0.0.1 --port 8010 --disable-log-stats \
  --deploy-config vllm_omni/deploy/minimax_h3_disaggregated.yaml \
  --stage-overrides '{"1":{"tensor_parallel_size":4,"ulysses_degree":1}}'

trap 'docker stop --time 30 h3-nixl-e2e' EXIT
curl --noproxy '*' --fail --silent --show-error \
  --retry 180 --retry-connrefused --retry-delay 10 \
  --retry-max-time 1800 --max-time 5 http://127.0.0.1:8010/health

curl --noproxy '*' --fail --silent --show-error --max-time 1800 \
  -D h3-e2e-output/orange-cat.headers -o h3-e2e-output/orange-cat.mp4 \
  -w 'http_code=%{http_code}\ncontent_type=%{content_type}\nsize_download=%{size_download}\ntime_total=%{time_total}\n' \
  http://127.0.0.1:8010/v1/videos/sync \
  -F 'prompt=A clearly visible orange tabby cat walking through bright green grass in warm daylight, detailed fur, natural motion, vibrant colors, wide cinematic shot, with matching outdoor ambient sound.' \
  -F 'width=1344' -F 'height=768' -F 'num_frames=107' -F 'fps=24' \
  -F 'num_inference_steps=8' -F 'seed=42' \
  -F 'extra_params={"task":"t2va","duration":4.0,"aspect_ratio":"16:9","flow_shift":12.0,"audio_flow_shift":3.0}'

docker exec h3-nixl-e2e ffprobe -v error \
  -show_entries format=duration,size \
  -show_entries stream=codec_type,codec_name,width,height,nb_frames,r_frame_rate \
  -of json /artifacts/orange-cat.mp4
docker exec h3-nixl-e2e ffmpeg -v error -xerror \
  -i /artifacts/orange-cat.mp4 -f null -
docker exec h3-nixl-e2e ffmpeg -v error -ss 2 \
  -i /artifacts/orange-cat.mp4 -frames:v 1 /artifacts/preview.png
docker logs h3-nixl-e2e > h3-e2e-output/server.log 2>&1
grep -E 'NixlConnector (put|get).*video_sync' h3-e2e-output/server.log
sha256sum h3-e2e-output/orange-cat.mp4
docker stop --time 30 h3-nixl-e2e
trap - EXIT

The test host additionally passed its HTTP(S) proxy through to the container; configure your own proxy if required. No custom deployment YAML is needed.

Separate native smoke command (run when its two GPUs are free):

docker run --rm --gpus '"device=1,3"' --network host --ipc host \
  -v "$PWD:/workspace:ro" -w /workspace \
  -e PYTHONPATH=/workspace -e PYTHONDONTWRITEBYTECODE=1 \
  -e UCX_RCACHE_MAX_UNRELEASED=1024 \
  "$IMAGE" python -m pytest -vv -s -rA -o addopts='' -p no:cacheprovider \
  tests/distributed/omni_connectors/test_nixl_connector_native.py

Scope: real same-host CUDA/NIXL transfer with heterogeneous producer/consumer TP. Inter-node transfer, Intel XPU, and real Mooncake RDMA E2E were not validated. Mooncake/paged-KV coexistence is covered by regression tests, not by this H3 media run. H3 exercises an AR producer and diffusion consumer; the separate diffusion-producer/inline-drop branch is unit-tested, not exercised by this two-stage E2E.

Expanded regression command

Run separately in the same runtime, without GPU exposure. Redirecting the CI fixture helper's generated directory to /tmp is necessary with the read-only source mount; otherwise fixture generation fails for environmental reasons.

docker run --rm \
  -v "$PWD:/workspace:ro" -w /workspace \
  -e PYTHONPATH=/workspace -e PYTHONDONTWRITEBYTECODE=1 \
  -e CUDA_VISIBLE_DEVICES= \
  "$IMAGE" python -c '
import sys
from pathlib import Path
import tests.helpers.stage_config as stage_config
import pytest
stage_config._CI_GENERATED_DIR = Path("/tmp/generated-ci")
raise SystemExit(pytest.main(sys.argv[1:]))
' -q -o addopts='' -p no:cacheprovider \
  tests/distributed/omni_connectors \
  --ignore=tests/distributed/omni_connectors/test_nixl_connector_native.py \
  --ignore=tests/distributed/omni_connectors/test_mooncake_transfer_engine_rdma.py \
  --ignore=tests/distributed/omni_connectors/test_bagel_mooncake_connector.py \
  --ignore=tests/distributed/omni_connectors/test_bagel_shared_memory_connector.py \
  --ignore=tests/distributed/omni_connectors/test_mooncake_tcp_completion.py \
  tests/engine/test_orchestrator_kv_sender_info.py \
  tests/worker/test_omni_connector_mixin.py \
  tests/diffusion/test_diffusion_stage_payload.py \
  tests/diffusion/test_inline_stage_diffusion_client.py \
  tests/diffusion/test_stage_payload_kv_coexistence.py \
  tests/diffusion/test_diffusion_request.py \
  tests/diffusion/test_diffusion_config_fields.py \
  tests/diffusion/diffusion_kv \
  tests/model_executor/stage_input_processors/test_minimax_h3.py \
  tests/config

Observed result on the fixed tree: 1278 passed, 6 skipped, 2 warnings, 3 subtests passed in 95.55s. Apply your proxy settings to the container where model-config downloads are required. The explicitly excluded native/hardware suites are not included in this passing count.

[natureofnature]

Review of 7da3f90a2efc14350af1641a6e1c2b4fdba79146: I found three correctness/lifecycle issues and one configuration regression that should be addressed before merging.

P1: Intermediate stages cannot publish metadata for their outgoing payloads

For a chain such as Thinker -> Talker -> Code2Wav, get_stage_connector_spec() selects the intermediate stage's incoming-edge configuration with role="receiver". The resolver removes its local zmq_port, and _init_handshake() does not start a metadata listener. However, the same connector also handles outgoing put() calls, whose returned transfer metadata is not forwarded by the adapter/mixin send path. The next stage therefore has no way to retrieve those descriptors.

Please preserve both incoming and outgoing endpoint responsibilities for intermediate stages and add a three-stage regression test. Simply changing the role to sender would not preserve the incoming-edge contract.

P1: Async-chunk prewarming bypasses producer endpoint propagation

The new payload_sender_info assignment is wired into ordinary forwarding, but _prewarm_async_chunk_stages() constructs requests separately and leaves that field unset. The adapter/mixin receive path consequently falls back to its configured endpoint. When producer and consumer replica IDs differ, the receiver-derived port can point to the wrong producer replica, leaving the request waiting for data.

Please attach the bound upstream producer's endpoint during prewarming and test the actual submitted request with unequal producer/consumer replica IDs. Testing an adapter request with manually populated sender info does not cover this entry point.

P1: Producer lease expiry can release memory during an active remote READ

_handle_handshake_message() returns descriptors without pinning or renewing the source allocation. _cleanup_expired_pending() then removes and deregisters it at the original deadline, regardless of whether the consumer has completed its READ. A consumer obtaining metadata shortly before expiry can therefore still be reading when reclamation occurs.

The receiver-side deferred-transfer handling protects destination resources, but not this producer allocation. Increasing the TTL reduces the probability without establishing memory-lifetime safety. Please distinguish unclaimed payload expiry from active-transfer ownership and cover expiry while a consumer transfer remains PROC.

P2: Explicit sender_zmq_port overrides are discarded

resolve_connector_spec() unconditionally assigns sender_zmq_port. For example, a receiver specifying sender_zmq_port=55000 but no zmq_port resolves to 50051 for producer stage 0/rank 0/replica 0. The previous generic initialization used setdefault, preserving this override. The shared resolver also affects existing transfer-engine connectors.

Please preserve explicit sender endpoints and derive defaults only for absent fields.

Verification scope: I inspected the current call paths and executed four dependency-stubbed control-flow probes using the unchanged method bodies. They confirmed the missing intermediate listener, missing prewarm sender info, source deregistration after metadata delivery without a completion ACK, and overwritten explicit port. I did not run native NIXL/GPU E2E tests or reproduce an actual RDMA corruption event; the probes establish the control-flow gaps, not hardware-level outcomes.

[yuanwu2017]

Addressed the new review in 4146ce7, then integrated main aff7d64 in 882026b. Both commits are signed off and pushed. Main's connector-mixin facade/split is preserved; the NIXL-specific changes are carried in the split runtime and payload-transport modules.

@natureofnature — the four findings are addressed:

1. Intermediate stages: retain the incoming sender endpoint plus a separate outgoing bind configuration/listener. Stage-client advertisement uses that outgoing endpoint (including default ports). Added three-stage configuration/listener regression for ordinary and async settings, plus worker routing tests; this is not a full three-stage model E2E claim.
2. Async prewarming: attach the request-bound upstream producer's endpoint to the actual submitted prewarm request. Regression tests use unequal producer/consumer replica IDs and concurrent bindings.
3. Source ownership: metadata acquisition atomically installs a generation-scoped claim before exposing descriptors, including the direct-metadata path. Only unclaimed payloads expire; claimed allocations survive expiry, cleanup, and deferred close until matching completion. Duplicate/stale ACKs cannot release another claim. Deterministic fake-agent tests hold PROC and cover terminal/error/timeout/unknown states, deferred cleanup and late ACKs. This deliberately favors safety over reclamation: an abandoned/no-ACK claim can retain memory indefinitely; TTL is not native READ cancellation. Current-generation direct metadata still requires the control-plane claim.
4. Explicit endpoint overrides: preserve sender_zmq_port; compute defaults only when absent, with NIXL and Mooncake regression coverage.

Also adopted @hsliuustc0106's compute_connector_zmq_port naming, and retained request-scoped endpoint propagation through both ordinary and async-chunk consumers. Zero-byte tensor leaves keep their shape/dtype semantics without invalid zero-byte DMA registrations.

Validation on the final integrated tree

• Matching runtime: Python 3.12.3, vLLM 0.29.0 / Torch 2.13.0+cu130 / NIXL 1.3.2, derived from vllm/vllm-openai:v0.29.0 (local test image vllm-omni-pr6093:v029-merged-v029-20260910T083620Z-attempt3, SHA256 2d581fd28985715b4c7f4b53940fbe28a9adcc2f107a3be786256f81da973c50). Local image is not a publicly pullable artifact.
• Expanded connector + worker-mixin + orchestrator sender-info + config selection: 1046 passed, 2 failed, 4 skipped, 3 subtests passed. Both failures are the frontend-log-stats config test's True/False cases on UnspecifiedOmniPlatform in a CPU-only container. The same failures reproduce on pinned main in the same image/isolation (611 passed, 2 failed, 1 skipped for main's config directory). No new failures within this selection; not an all-suite pass.
• Real two-process/two-GPU native structured transfers: 2 passed, covering both GET_META and direct metadata, mixed CPU/CUDA leaves, a zero-sized CUDA tensor, source retention across forced expiry/cleanup, and final zero pending registrations. Both runs recorded native states ['PROC', 'DONE']. The test records rather than requires PROC, so this is not a guaranteed hardware timeout/cancellation test; deterministic lifecycle coverage is in the unit tests.
• Ruff lint/format and diff checks passed. Source was mounted read-only; all owned test containers removed. No model E2E or cross-host/RDMA-fabric validation is claimed here. The new [Feature] Transfer AR-to-diffusion payloads over NIXL #6264 H3 E2E remains pending.

Reproduction selections (inside the matching test environment, from repository root):

export HTTP_PROXY=http://proxy-ir.intel.com:912 HTTPS_PROXY=http://proxy-ir.intel.com:912
export http_proxy="$HTTP_PROXY" https_proxy="$HTTPS_PROXY"
export NO_PROXY=localhost,127.0.0.1,::1 no_proxy=localhost,127.0.0.1,::1
python3 -m pytest -o addopts='' -p no:cacheprovider -o faulthandler_timeout=120 \
  tests/distributed/omni_connectors \
  --ignore=tests/distributed/omni_connectors/test_nixl_connector_native.py \
  --ignore=tests/distributed/omni_connectors/test_mooncake_transfer_engine_rdma.py \
  --ignore=tests/distributed/omni_connectors/test_bagel_mooncake_connector.py \
  --ignore=tests/distributed/omni_connectors/test_bagel_shared_memory_connector.py \
  --ignore=tests/distributed/omni_connectors/test_mooncake_tcp_completion.py \
  tests/engine/test_orchestrator_kv_sender_info.py \
  tests/worker/test_omni_connector_mixin.py tests/config

# Separate run with two available GPUs:
UCX_RCACHE_MAX_UNRELEASED=1024 python3 -m pytest -o addopts='' -p no:cacheprovider -vv -s -rA \
  tests/distributed/omni_connectors/test_nixl_connector_native.py

For the actual read-only-container runs, an external guarded launcher redirects generated CI fixtures to /tmp/generated-ci; it does not override platforms or mask test failures. The same upper/lower proxy variables and localhost bypass were passed explicitly to Docker.

[hsliuustc0106]

CI failed

[yuanwu2017]

CI follow-up for e4b93ad476127809d2238287f1d9e26b9cfd6de6: pushed test-only fixes in cd8a7b3248a5005b069b1820d621976d4d8db83a.

NVIDIA failures addressed

• Engine/Entrypoints: test_async_prewarm_skips_outgoing_only_stage used a pool double missing get_bound_client. The double now models submission-time bindings and checks sender endpoint propagation.
• Other: the three-stage endpoint test assumed the first bounded metadata query succeeded. Discovery tests now use bounded outer retries and verify producer port/generation. Exact ownership tests use synchronous control-message dispatch to avoid lost-reply claim-count ambiguity; real ZMQ discovery/completion coverage remains. Production timeout/ownership behavior is unchanged.

Local CPU-only validation with Python 3.12.3 / vLLM 0.29.0: 401 passed, 1 skipped, 3 subtests passed; repeated endpoint cases 100 passed; injected first-query misses 40 passed. The original one-shot assumption fails under the same injection. Ruff check/format and diff checks passed. This is not a remote CI or native RDMA rerun.

Remaining accelerator CI incompatibilities

• Intel 8081 explicitly uses vllm/vllm-openai-xpu:v0.28.0: 148 collection errors, including 147 direct missing-API/module errors and one separately tracked SenseNova import error.
• AMD 11683 and NPU 6715 also report vLLM 0.28.0 against this 0.29-based Omni tree. Missing APIs include compute_layout_strides, vllm.v1.kv_cache_layout, and the new entrypoint modules. NPU also reports a MiniCPM processor prompt= signature mismatch.

These failures require aligning the accelerator runtime revisions and rerunning CI; this patch does not claim those platforms pass or establish a baseline exemption. The PR diff against its merged main parent does not change the Buildkite/container/dependency configuration. No compatibility shim or broader CI-image change was added to this connector PR.

H3/#6264 E2E remains paused and incomplete; no GPU/native/E2E tests were launched for this fix.

[yuanwu2017]

Follow-up: the accelerator version mismatch fixes are already available on newer main. Updated this branch in signed merge 3e746e8 with pinned main 01a2f93, including AMD #7395, Intel/XPU #7441 and NPU #7433. Their hardware Dockerfiles and Intel pipeline match that main revision. No custom compatibility shim was needed.

Resolved the environment-inventory conflict preserving both NIXL controls, and adapted the new upstream terminal-chunk test mock to accept forwarded metadata. Both preceding CI test fixes remain intact.

Local CPU validation: 1,069 passed, 2 skipped, plus 3 passing subtests across combined connector/orchestrator and config/AMD checks. Three additional no-device/offline config failures reproduce with identical node IDs/messages on exact main in the same runtime. Focused inventory/AMD/chunk rerun: 155 passed. Ruff and diff checks passed. No GPU/native/E2E or accelerator-image builds were run locally; updated remote CI must verify the hardware results. H3 E2E remains paused.

[yuanwu2017]

NIXL native E2E validation completed; H3 integration results

The native two-process NIXL transport E2E matrix passed all 8 cases (236.34 seconds). These are real native transfers, not the CPU fake-agent tests. Separately, the H3 integration generated valid video/audio for all three tasks, with an outstanding model-request log-evidence check described below.

Tested stack / attribution

• This PR's tested revision: 3e746e8.
• Executed in the [Feature] Transfer AR-to-diffusion payloads over NIXL #6264 integration, now published as 7302d4c, not as an isolated checkout of [Core] Add NIXL omni connector #6093.
• Exact frozen/published tree: b99e97a3dafb0e7e9c1bd28d3a487b33bc418d24.
• Includes main 01a2f93, [feat] Extend disaggregated MiniMax-H3 stage with VAE encoders #6939 encoder snapshot 10ab470, and asukaqaq-s/vllm-omni#1 review fixes 90d24eb. The current [feat] Extend disaggregated MiniMax-H3 stage with VAE encoders #6939 merge head has equivalent dependency-tree content, but its merge commit is not an ancestor of this integration.

Native transport matrix — PASS

tests/distributed/omni_connectors/test_nixl_connector_native.py::test_native_two_process_structured_mixed_device_transfer

Payload	Queried metadata	Direct metadata
Generic structured mixed-device payload	PASS	PASS
H3 T2VA payload	PASS	PASS
H3 FL2VA payload	PASS	PASS
H3 Ref2VA payload	PASS	PASS

Native NIXL/UCX, separate producer and consumer processes, mixed CPU/CUDA structured tensors, including H3 text/visual/audio/layout fields and empty optional tensors. Logs record native transfer states, received payload values/types/shapes, source retention and final cleanup. This is same-host CUDA evidence, not cross-node or non-CUDA validation.

The matrix ran in validation e2e-latest6093-20260913-0606. CPU 547 passed and H3 config 21 passed in that validation. The subsequent 0625 model run explicitly reused those results only after verifying unchanged source/index/runtime; it did not rerun or double-count them.

Real H3 application generation — media PASS, transport log acceptance incomplete

Task	HTTP	Request time	MP4 bytes	Media validation
T2VA, FL2VA checkpoint	200	76.743 s	5,044,422	PASS
FL2VA image input, FL2VA checkpoint	200	65.350 s	4,744,712	PASS
Ref2VA video + embedded AAC, Ref2VA checkpoint	200	200.459 s	6,106,601	PASS

All outputs: H.264 1344×768, 107 frames at 24 fps, stereo 32-kHz AAC; full ffmpeg decode and preview extraction passed. Requests used 8 steps and seed42. Six physical GPUs0,1,3,4,5,6: Stage0 TP2/max_model_len32768; Stage1 TP4/Ulysses1/VAE4. Repository H3 YAML unchanged, with CLI overrides {"0":{"max_model_len":32768},"1":{"tensor_parallel_size":4,"ulysses_degree":1}}.

Important boundary: the application runner's request-correlated NIXL log check failed for all three requests: producer put records exist, but corresponding consumer get DEBUG records were absent. Consequently the overall application E2E runner exited 1, not 0. The missing log cause remains unresolved. Successful media output and the separate native matrix must not be presented as proof that this application-level transport check passed, or as verified absence of fallback.

Runtime: Python3.12.3 / vLLM0.29.0 / Torch2.13cu130 / NIXL1.3.2; immutable image sha256:bce11ef0dfb05cd4c9e18695bccdd751b7c200514b14ee67fa14bf06b3f981e4. Model snapshot 42ed227ee7df40d41602854ae760620d6eb651fe. All owned test containers were cleaned up and six test GPUs released; unrelated GPU2 workloads were untouched.

Detailed #6264 results and output hashes: #6264 (comment) . Local native logs/XML are retained under vllm-omni-pr6264-artifacts/e2e-latest6093-20260913-0606/; model logs, requests and videos under e2e-latest6093-20260913-0625/. These local paths are not public artifact attachments.

[yuanwu2017]

Final NIXL receiving acceptance PASS — correction and follow-up

Published fixes:

• [Feature] Transfer AR-to-diffusion payloads over NIXL #6264: a7ac1bddb8a48dd4835edf9a3671e4dba0964833, tree d6b868b42bd4805777a6f4cb728c55bf5524df5f. All tracked source hashes match the frozen full-model test source.
• [Core] Add NIXL omni connector #6093: ea34100772a6af18fe20bcd225a1cc5f594d2cf0, generic backport including the payload-only sender-info prerequisite, without H3-specific test dependencies.

Root cause: not merely absent DEBUG logs

The earlier 0625 run attempted to bind the receiver to the producer's occupied port. Receiver initialization failed and the application fell back to the inline prompt. Those videos were therefore not application-level NIXL receive evidence. The earlier matcher correctly failed; this supersedes the previous unresolved-log interpretation.

The fix removes shared incoming-edge zmq_port when the manager constructs a NIXL receiver. Request-scoped and explicit standalone sender endpoints remain supported. Sender/intermediate-stage local listener behavior is unchanged. No logger workaround or weakened acceptance matcher was used.

Fresh integration acceptance — all PASS, overall exit 0

Completed 2026-09-13 07:36:01 UTC. 670 CPU passed (1 skipped), 21 H3 config passed, 8 native UCX passed (237.44s). Native tests now exercise manager-created receivers against an already-bound producer endpoint and retain exact mixed-device payload/ownership checks.

Task	HTTP	Seconds	MP4 bytes	Put/get bytes	Media / receive match
T2VA	200	76.640	5,073,052	359,024 / 359,024	PASS / PASS
FL2VA image	200	65.479	4,765,076	11,158,136 / 11,158,136	PASS / PASS
Ref2VA video + embedded AAC	200	201.440	6,110,354	64,963,792 / 64,963,792	PASS / PASS

Exact keys recorded at both producer and consumer:

• video_sync-8837c02bfe1c75fb-bdf52d52_0_0
• video_sync-b89d4865e4016bb8-a63e69cd_0_0
• video_sync-a2bd7e40639d1149-b77a57f2_0_0

All outputs passed H.264 1344×768, 107 frames/24fps, stereo 32-kHz AAC, full ffmpeg decode and preview checks. Owned containers removed, six test GPUs released, unrelated GPU2 untouched.

Exact tested stack

Full-model tests used #6264 integration, based on #6093 3e746e8c691cc82fc0c4898d013983f1a8098db6 plus the receiver fix, main 01a2f93256975c7ff9565c5414b0fe0225bc4765, #6939 snapshot 10ab47041faa13172e40ccea055278f1cbdbdd78, and asukaqaq-s/vllm-omni#1 fixes 90d24eb108d889791ac935387ce1374b212db85a. It was not standalone #6093. The later generic backport commit is not an ancestor of this tested integration.

Standalone #6093 backport validation: 479 CPU passed, 1 skipped, 3 passing subtests; 2/2 generic native UCX cases passed. These two cases are separate from the integration's eight.

Runtime: Python3.12.3/vLLM0.29.0/Torch2.13cu130/NIXL1.3.2; image sha256:bce11ef0dfb05cd4c9e18695bccdd751b7c200514b14ee67fa14bf06b3f981e4. Same-host physical GPUs0,1,3,4,5,6: Stage0 TP2/maxlen32768/util0.6; Stage1 TP4/Ulysses1/VAE4. Stock H3 YAML and CLI overrides {"0":{"max_model_len":32768},"1":{"tensor_parallel_size":4,"ulysses_degree":1}}. 8 steps/seed42; model snapshot 42ed227ee7df40d41602854ae760620d6eb651fe, FL2VA and Ref2VA partitions.

Cross-node, non-CUDA and Turbo checkpoints remain untested; remote CI and approval are separate. #6264's inherited unsigned commit remains unresolved. Local evidence: vllm-omni-pr6264-artifacts/nixl-receive-final-20260913-071039/ and pr6093-review-artifacts-20260910/6093receiverfix/ (local paths, not public attachments).