Skip to content

feat(workflow): transfer remote tensors with NIXL - #13085

Closed
furionw wants to merge 65 commits into
qiwa/vllm-workflow-componentsfrom
qiwa/workflow-nixl-carrier
Closed

furionw wants to merge 65 commits into
qiwa/vllm-workflow-componentsfrom
qiwa/workflow-nixl-carrier

Conversation

@furionw

@furionw furionw commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Based on #13293.

Why

Most workflow values should use Dynamo's ordinary endpoint codec. A remote stage that produces a tensor needs a different transport and ownership lifecycle. This optional fork enables NIXL at binding time, selects it from the runtime value, and keeps the carrier opaque to graph authors.

Experimental: This workflow API is under active development and may evolve based on feedback. It is an optional orchestration layer over Dynamo's discoverable workers and endpoints; applications may compose those primitives directly when they prefer to own the orchestration logic.

What Change

  • Add tensor_carrier="nixl" as an opt-in remote binding capability.
  • Select NIXL only for a top-level runtime torch.Tensor; ordinary values keep the endpoint codec.
  • Derive consumers from WorkflowIR and bindings instead of adding physical edges to the plan.
  • Retain producer storage until graph-derived consumers complete and quarantine uncertain transfers.
  • Reject unsupported workflow-input, inline-boundary, workflow-output, and nested tensor cases.
classDiagram
    RemoteBinding --> NixlTensorCarrier : enables capability
    RemoteStageServer --> NixlTensorCarrier : exports runtime tensor
    NixlTensorCarrier *-- NixlLeaseRegistry : retains producer
    NixlTensorCarrier ..> NixlTensorFanout : creates consumers
    RemoteStageClient --> NixlCarriedValue : imports opaque carrier message

    note for RemoteBinding "WorkflowIR remains transport-neutral"
    note for NixlLeaseRegistry "Releases only after planned consumers finish"
    note for NixlCarriedValue "Internal; stage author receives tensor"
Loading

Test Plan

  • GPU-free tests cover value selection, ordinary fallback, fan-out, lifecycle completion, timeout quarantine, invalid topology, and direct mappings.
  • Applicable pre-commit hooks pass.

@datadog-official

This comment has been minimized.

@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from b84931d to e2378a7 Compare August 12, 2026 08:35
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch 3 times, most recently from f8ca899 to 3ed44f3 Compare August 12, 2026 08:56
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from 3fdc640 to 06196c1 Compare August 12, 2026 21:17
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 3ed44f3 to 6551a32 Compare August 12, 2026 21:19
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from 06196c1 to d97c40e Compare August 12, 2026 21:58
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 6551a32 to b0acc7f Compare August 12, 2026 21:58
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from d97c40e to 3c40733 Compare August 12, 2026 22:32
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from b0acc7f to 962c52b Compare August 12, 2026 22:33
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from 3c40733 to 7f8547f Compare August 13, 2026 00:20
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 962c52b to 6c45a91 Compare August 13, 2026 00:20
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from 7f8547f to 8de19c5 Compare August 13, 2026 22:22
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 6c45a91 to f389c11 Compare August 13, 2026 22:37
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from 8de19c5 to 97c5cd9 Compare August 13, 2026 23:15
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from f389c11 to 90405bd Compare August 13, 2026 23:15
@furionw
furionw force-pushed the qiwa/workflow-frontend-provider branch from 97c5cd9 to 2b18280 Compare August 13, 2026 23:33
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 90405bd to 6671ae0 Compare August 13, 2026 23:33
@furionw
furionw changed the base branch from qiwa/workflow-frontend-provider to qiwa/workflow-remote-execution August 14, 2026 05:14
@furionw
furionw force-pushed the qiwa/workflow-remote-execution branch from 5d2b429 to bba4aa7 Compare August 14, 2026 05:15
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 143daae to 3a3ee50 Compare August 14, 2026 05:15
@furionw
furionw force-pushed the qiwa/workflow-remote-execution branch from bba4aa7 to 291cb34 Compare August 14, 2026 21:58
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 3a3ee50 to bc42b4d Compare August 14, 2026 22:00
@furionw
furionw force-pushed the qiwa/workflow-remote-execution branch from 291cb34 to c7027ed Compare August 15, 2026 02:51
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from bc42b4d to 7a3898e Compare August 15, 2026 02:51
@furionw
furionw force-pushed the qiwa/vllm-workflow-components branch from f604836 to 225212a Compare August 25, 2026 21:27
@furionw
furionw force-pushed the qiwa/workflow-nixl-carrier branch from 03d778d to 9b83542 Compare August 25, 2026 21:27
@furionw
furionw force-pushed the qiwa/vllm-workflow-components branch 2 times, most recently from e7e4df9 to 426ece9 Compare August 26, 2026 07:26
@furionw furionw closed this Aug 28, 2026
@furionw
furionw deleted the qiwa/workflow-nixl-carrier branch August 29, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant