[WebGPU] Expose safe graph capture I/O in Python - #32074
Ananya Anand (4n4ny4) merged 21 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
This PR exposes a safe Python workflow for WebGPU graph capture/replay by introducing session-owned device OrtValue allocation, enforcing session provenance for session-scoped buffers, and restricting unsafe I/O patterns (raw pointers, DLPack export, cross-session use, and output rebinding) that can break captured replay semantics.
Changes:
- Add session-owned
OrtValuecreation/update APIs and a Python-facingrelease_captured_graph()workflow to manage captured resources safely. - Enforce provenance/lifetime rules for session-scoped and WebGPU
OrtValues acrossrun,run_async,run_with_ort_values,IOBinding, andOrtValueVectorusage patterns. - Add focused Python tests plus a reusable WebGPU graph-capture benchmark script demonstrating fixed device I/O and capture/replay.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/test/python/webgpu_graph_capture_benchmark.py | Adds a standalone benchmark demonstrating fixed WebGPU I/O binding, capture/replay, in-place updates, and explicit output readback. |
| onnxruntime/test/python/onnxruntime_test_python.py | Adds unit tests covering session-scoped OrtValue semantics and WebGPU graph-capture safety restrictions. |
| onnxruntime/python/onnxruntime_pybind_state.cc | Adds pybind methods for session-owned OrtValue allocation/update plus WebGPU graph-capture status/release plumbing. |
| onnxruntime/python/onnxruntime_pybind_ortvalue.cc | Blocks unsafe WebGPU operations (data_ptr, numpy, DLPack) and prevents putting WebGPU OrtValues into raw OrtValueVector. |
| onnxruntime/python/onnxruntime_pybind_iobinding.cc | Ensures WebGPU binding uses the correct allocator name and keeps the session alive for SessionIOBinding. |
| onnxruntime/python/onnxruntime_inference_collection.py | Adds Python APIs for session-owned OrtValues, graph-capture release, and provenance validation/enforcement in Session, IOBinding, and OrtValue. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Jiajia Qin (qjia7)
left a comment
Jiajia Qin (qjia7)
left a comment
There was a problem hiding this comment.
Review frame
- Review contract: Reviewed PR 32074 at exact head
a30855282ecb8da56d8d6cdd4cb3b74165f0575aagainst its recorded base3d3cfa6c004551ee31d3acff6d01fb572712ca91, including all six changed files, issue discussion, reviews, inline discussion, and head CI. GitHub reports only the passinglicense/clacheck. - Problem/feature validity: Validated. WebGPU replay retains the bind groups/buffer handles recorded during capture, while the pre-PR Python allocation path does not obtain the allocator from a particular session. Python therefore needs a session-scoped way to allocate and update fixed WebGPU inputs/outputs and to release captured resources.
- Risk/scope: Deep. This adds public Python APIs and changes allocator, data-transfer, I/O-binding, graph-ID, and object-lifetime behavior across Python and C++.
- Direction gate: Pass, with incomplete enforcement. Session allocator lookup plus Python-side session provenance is the right ownership direction and is consistent with
InferenceSession::GetAllocator,SessionIOBinding, and the WebGPU EP's per-session context. The owner solution must additionally associate each captured graph ID with immutable I/O buffer identities until release; C1 is the missing part of that safety boundary. The implementation also has the compatibility and transfer-routing gaps below.
Confirmed findings
C1 [P2]: Reject a different or mutated I/O binding while a graph ID is captured
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:534
Suggested comment:
[P2]Please associate each captured graph ID with theIOBindingand fixed OrtValues used for capture, and reject a different/rebound/cleared binding untilrelease_captured_graph(id)succeeds. This check currently establishes only that the binding belongs to the same session. Once graph 0 is captured,InferenceSession::RunImplreplays graph 0 by ID and WebGPU'sReplayGraphsubmits the previously captured bind groups; it never consults the buffers in the newly supplied binding. The new test demonstrates the resulting silent failure at lines 2069-2079: runningalternate_io_bindingupdates the original output and leaves the alternate output zero. Clearing the original binding before release can additionally return those buffers to WebGPU's cache while the captured bind group still identifies them. This contradicts the PR's claim that unsafe buffer-rebinding operations are rejected. Please enforce the lifecycle in the API (or expose a capture object that owns the fixed binding), and change this test to expect rejection rather than stale output.
Evidence and attribution: The PR exposes the previously unavailable session-owned buffers and advertises a safe Python workflow. Session.run_with_iobinding adds only a session-identity check. The PR's own test proves that another same-session binding is accepted and silently ignored by replay.
S1 [P2]: Preserve the documented per-run gpu_graph_id=-1 escape hatch
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:323
Suggested comment:
[P2]Please make this validation depend on the effectivegpu_graph_idinstead of rejecting every run whenever the session option enables capture. Core defineskGraphAnnotationSkip = -1, andCachedExecutionProviderForGraphReplay::AllowGraphCaptureOnRun()returns false for that ID; WebGPU'sOnRunStartlikewise does not create a per-graph manager or begin capture for-1. Such a run is therefore the supported non-capturing path and transient feeds are safe. Before this PR, Python callers could use it, butrun,run_async, andrun_with_ort_valuesnow reject it before C++ sees the run options, and the bind-time IOBinding checks also make a skipped run with ordinary bindings impossible. Please allowgpu_graph_id=-1, defer fixed-buffer validation untilrun_with_iobindingwhere the run options are known, and add regression coverage for both convenience-run and IOBinding skip paths.
Evidence and attribution: InferenceSession::CachedExecutionProviderForGraphReplay explicitly reserves -1 as kGraphAnnotationSkip, and the recursive capture/replay path is guarded by AllowGraphCaptureOnRun. The unconditional Python guard is added by this PR and removes that existing behavior.
C2 [P2]: Do not let the session-wide transfer search route CUDA buffers through WebGPU
Suggested inline location: onnxruntime/python/onnxruntime_pybind_state.cc:144
Suggested comment:
[P2]Please ensure this session-scoped copy selects the transfer implementation for the tensors' actual provider. These new creation/update APIs are generic (the CPU path is tested anddevice_type="cuda"is accepted), butDataTransferManager::CopyTensorchooses the first registered transfer whoseCanCopyreturns true. The built-in WebGPUDataTransfer::CanCopyaccepts every CPU/GPU or GPU/GPU pair without checking vendor ID. In a session registered as[WebGpuExecutionProvider, CUDAExecutionProvider], updating a session-created CUDA OrtValue therefore selects WebGPU first;DataTransferImpl::CopyTensorcasts the CUDA allocation toWGPUBuffer, andBufferManager::Uploadpasses it towgpuBufferGetMappedRange. The plugin WebGPU transfer already avoids this by requiring vendor ID 0. Please mirror that vendor/device check in the built-in transfer (or restrict this API to locations it can select unambiguously) and add a mixed-provider regression test.
Evidence and attribution: The PR's CopySessionOrtValue rejects only GPU-to-GPU pairs where exactly one memory-info name is WEBGPU_BUFFER; CPU-to-CUDA and CUDA-to-CUDA take the unrestricted session-wide search. The unsafe WebGPU cast is a direct consequence. The PR newly exposes this path for arbitrary session allocators and routes all session-scoped updates through it.
Clarifications
Q1: The reported 8.9% speedup is not reproducible from the added benchmark
webgpu_graph_capture_benchmark.py unconditionally enables graph capture at line 53 and has no baseline mode, so the same script cannot produce the stated 5.506 ms non-capture result. Please provide the exact baseline/capture commands and raw result JSON (including backend and input-copy mode), or add a capture toggle so the comparison uses identical fixed I/O, readback, warmup, and timing boundaries.
Test coverage
- C1: The WebGPU test exercises the wrong-binding case, but asserts the silent stale-output behavior instead of enforcing the advertised safety invariant.
- S1: No test covers
gpu_graph_id=-1on a graph-enabled WebGPU session. - C2: No test covers a session with WebGPU followed by another GPU provider.
- The positive WebGPU test is skipped when the EP or adapter is unavailable. At review time GitHub reports only
license/cla; there is no WebGPU-enabled build/test result attached to the PR. - Local static verification: both changed Python modules compile with Python's
compile()andgit diff --checkis clean. I did not claim runtime or C++ build success because the available local build does not contain a PR-compatible Python WebGPU extension.
Verdict
The feature is valid and the session-provenance design direction is appropriate. C1 and C2 block the advertised safe workflow because accepted calls can silently target old buffers or pass a foreign GPU handle to Dawn. S1 is an existing graph-control compatibility regression and should also be fixed before merge. Q1 remains a performance-evidence clarification. I found no cleanup-only items worth posting.
A WebGPU OrtValue can be created from an environment-registered shared
allocator instead of a session allocator. `_is_webgpu_buffer` is derived purely
from the tensor's allocator name (`Location().name == WEBGPU_BUFFER`) and
carries no session information, and the shared-allocator factories
(`OrtValue.ortvalue_from_shape_and_type` -> `GetSharedAllocator`, and the
`memory_info` overload -> `GetRegisteredSharedAllocator`) never attach a session
to the Python wrapper. Such a value therefore arrives with
`_is_webgpu_buffer=True` and `_session=None`, so treating "is a WebGPU buffer"
as "belongs to this session" is unsound by construction rather than merely
strict.
That state is also the default rather than an exotic one: the environment
registers a shared allocator for every EP device publishing a
`device_memory_info`, with no user call required. This change is what makes it
reachable, because it adds the `"webgpu"` alias to
`get_vendor_id_for_device_type`, which is exactly what makes
`OrtDevice.make("webgpu", 0)` resolve to the shared allocator's lookup key --
and then rejected every OrtValue produced that way.
Key off a *different* owning session instead, matching what the non-WebGPU
branch directly below already did:
- `_validate_ortvalue_ownership` rejects a foreign session first, then steers
WebGPU values to IOBinding regardless of ownership. Drops the `allow_webgpu`
parameter: all five call sites (and every call site on this branch) invoked it
without the argument, so it was always `False` and its `if not allow_webgpu:
raise` branch was unconditional in practice. No caller relied on it and the
observable semantics are unchanged -- WebGPU buffers are still always rejected
on the plain `run()` paths and must go through IOBinding.
- `bind_ortvalue_input`/`bind_ortvalue_output` collapse three checks into two;
the generic cross-session check subsumes the WebGPU-specific one.
- Graph capture still requires a WebGPU device buffer but no longer requires a
session to own it. Capture replays against the original buffers, so a bound
value must stay alive at a fixed address for the captured graph's lifetime --
and that is a property of the OrtValue rather than of its allocator. `Tensor`
owns an `AllocatorPtr` to the allocator that frees it and only frees in
`~Tensor`, and for tensor-backed buffers `BufferManager::Release` is reachable
only through `GpuBufferAllocator::Free` (its only other caller returns an
internal uniform buffer, never tensor memory), so while the OrtValue is alive
the handle cannot be released, recycled, or reassigned; `p_data_` is never
rewritten, so it cannot relocate; and `WebGpuContext::Replay` re-issues the
bind groups recorded at capture rather than re-resolving a buffer from an
allocator. Each of those is provenance-independent, and for the default device
both allocators resolve to the same context buffer manager anyway. What
neither provenance protects is releasing a bound value while a captured graph
still references it: `_session` pins the session, not the buffer, so the
strict check never provided the protection an asymmetry would have implied.
This deliberately extends past the reviewed guard into `update_inplace`, which
had the same defect and refused a session-less WebGPU destination outright.
Without that, a shared buffer could be bound but never filled and the
relaxation would be inert. It now routes through whichever session owns either
side and falls back to the shared-allocator data transfer path when neither
does.
Source contiguity normalization is also hoisted above the numpy dispatch. It
previously ran only in the session-owned branch, so the session-less path
handed a possibly non-contiguous array to a copy that reads it as contiguous --
silently wrong data, no error. That bug is pre-existing and already reachable
for CPU OrtValues; relaxing the WebGPU guards increases its reachability rather
than introducing it, so it is fixed here alongside.
Error messages drop the now-inaccurate "session-scoped" qualifier.
Tests: add `test_device_ortvalue_provenance_rules`, a hermetic CPU-only matrix
over (no owner / this session / another session) x (device / non-device) that
runs in CI without a GPU, since every WebGPU Python test currently skips there.
It asserts the distinct rejection messages in both directions, so a session-less
device buffer is provably rejected for needing IOBinding and never for
belonging to a session that did not create it. Flip the WebGPU test assertions
that required session ownership, and add `gpu_graph_id=-1` coverage showing that
an opted-out run tracks the current input rather than replaying.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
C1: pin the captured I/O binding per graph annotation ID. Replay re-issues the bind groups recorded at capture and never consults a newly supplied binding, so `run_with_iobinding` accepting a different or rebound binding silently targeted the original buffers. The first non-skipped run for a `gpu_graph_id` now pins the IOBinding and the exact OrtValues bound to it; a different IOBinding, a changed buffer set, a rebind, or a clear is rejected until `release_captured_graph(id)` succeeds, which unpins it. The pin holds strong references to the bound OrtValues, so a captured buffer also cannot be freed and recycled into the WebGPU buffer cache while the captured graph still names it. WebGPU buffer handles are opaque and deliberately do not expose `data_ptr`, so buffer identity is the OrtValue object itself. S1: honor the documented per-run `gpu_graph_id=-1` escape hatch. Core reserves -1 as `InferenceSession::kGraphAnnotationSkip`, `AllowGraphCaptureOnRun()` returns false for it, and the WebGPU EP neither creates a per-graph buffer manager nor begins capture, so such a run is the supported non-capturing path and transient feeds are safe. `run`, `run_async` and `run_with_ort_values` rejected every run whenever the session option enabled capture, removing graph control Python callers had before. Validation now depends on the effective ID. The bind-time fixed buffer checks are likewise deferred to `run_with_iobinding`, where the run options are known, so a skipped run works with ordinary bindings; a capturing run still requires fixed WebGPU device OrtValues. C2: do not let the session-wide transfer search route foreign GPU buffers through WebGPU. `DataTransferManager::CopyTensor` picks the first registered transfer whose `CanCopy` accepts the pair, and the built-in WebGPU transfer accepted every CPU/GPU and GPU/GPU pair without checking the vendor. In a session registered as [WebGpuExecutionProvider, CUDAExecutionProvider] a CUDA copy therefore selected WebGPU, which reinterpreted the CUDA allocation as a `WGPUBuffer` and handed that foreign handle to Dawn. WebGPU allocations always carry `VendorIds::NONE`, so refuse GPU devices with any other vendor ID, matching the guard the plugin data transfer already applies. Q1: make the benchmark able to produce its own baseline. `webgpu_graph_capture_benchmark.py` hard-enabled graph capture, so the script could not produce the non-capture arm it was being compared against. Add `--graph-capture` / `--no-graph-capture` (capture stays the default) so both arms come from identical fixed I/O, readback, warmup and timing boundaries, and record the mode in the result JSON. Tests: `test_graph_annotation_id_run_option` covers the effective-ID parsing hermetically on CPU, which matters because every WebGPU Python test skips on current CI agents; it also caught that `RunOptions.get_run_config_entry` raises rather than returning a default for an unset key, which would have broken every run that passed a RunOptions without `gpu_graph_id`. The WebGPU test now asserts that a different binding is rejected instead of asserting the silent stale-output behavior, and covers the skip path through both `run_with_iobinding` and the convenience `run` API. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
_GRAPH_ANNOTATION_SKIP mirrors InferenceSession::kGraphAnnotationSkip. Nothing enforces that the two stay in sync, so record the linkage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review frame
- Review contract: Re-audited PR 32074 at exact head
48a2b3c66ae1385424f2c08fd8ea2d8f13d407d2. There is no new code range beyond the prior re-review againsta30855282ecb8da56d8d6cdd4cb3b74165f0575a. Current PR text, discussion, reviews, inline comments, and CI were fetched again. All visible required jobs have completed successfully, including Linux WebGPU, external-Dawn, static/plugin, and macOS WebGPU builds/tests. - Problem/feature validity: Validated. WebGPU replay retains the bind groups and buffer handles recorded during capture. Python needs fixed WebGPU buffers whose lifetime spans capture/replay, in-place updates, explicit readback, and captured-graph release.
- Risk/scope: Deep. The PR adds public Python APIs and graph-ID state, allocator/data-transfer routing, IOBinding lifetime enforcement, provider-reset interactions, and support for WebGPU allocator-backed values.
- Direction gate: Pass for graph-ID/IOBinding pinning, but fail for the additional session-owned OrtValue abstraction. The existing
OrtValuefactories andcopy_tensorsAPI already map to the environment-registered WebGPU shared allocator and transfer. Without a custom external device, those and the session allocator all resolve to WebGPU context 0, so graph capture does not require parallel session-specific allocation/copy APIs or Python ownership rules. Supporting shared allocators with a custom external device requires a separate C++ change and is deferred as an explicit TODO.
Prior issue resolution
- Prior C1 (different/rebound IOBinding): Resolved for a stable session. The first successful capturing run records the IOBinding/signature, holds the bound OrtValues, rejects rebinding/clearing and different bindings, and unpins only after native release succeeds.
- Prior S1 (
gpu_graph_id=-1): Resolved. All convenience run APIs now inspect the effective graph ID, and IOBinding validation is deferred untilrun_with_iobinding, so skipped runs can use transient bindings. - Prior C2 (CUDA handle routed through WebGPU): The unsafe path is fixed: built-in WebGPU
CanCopynow rejects non-NONEGPU vendors beforeDataTransferManagercan select it. T1 records the remaining regression-test gap. - Prior Q1 (benchmark baseline): Resolved. The benchmark now supports matched
--graph-captureand--no-graph-capturearms and records the selected mode. - Shared-allocator review comment: The existing shared allocator is the appropriate default-context path. Shared allocation for custom external devices remains a C++ follow-up because the shared allocator currently always resolves through
DefaultContext().
Confirmed findings
C1 [P2]: Clear or reject captured state before replacing the session
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:307
Suggested comment:
[P2]Please integrate_captured_graph_bindingswithset_providers()before replacing_sess. After graph 0 is captured, this method destroys/replaces the owning Python session without removing graph 0 from_captured_graph_bindingsor from the old IOBinding's_pinned_graph_ids. A new IOBinding from the replacement session is then rejected as "captured with a different IOBinding", even though the new native session has no captured graph. Callingrelease_captured_graph(0)afterward releases the new session (where graph 0 does not exist) and unpins the old binding while its oldSessionIOBindingcan still keep the old native session alive. Please either rejectset_providers()until every captured graph is explicitly released, or release each graph through the old session and clear both sides of the Python bookkeeping before_reset_session(). Add a capture ->set_providers()-> new capture regression test.
Evidence and attribution: set_providers() directly calls _reset_session(). The new _captured_graph_bindings map is initialized only in Session.__init__, is populated by run_with_iobinding(), and is otherwise removed only by release_captured_graph(). SessionIOBinding uses py::keep_alive<1, 2>, so an old IOBinding can retain the old native session after the Python session wrapper is reset. This stale cross-session state is introduced by the new pinning implementation.
D1 [P2]: Reuse the existing OrtValue allocation and copy APIs
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:489
Suggested comment:
[P2]Please use the existingOrtValue.ortvalue_from_shape_and_type/ortvalue_from_numpyfactories andcopy_tensorspath instead of adding parallel session-specific allocation and copy APIs. CUDA graph capture already uses those APIs. For WebGPU without a custom external device, the environment-registered shared allocator, its data transfer, and session allocators all resolve throughWebGpuContextFactorycontext 0, so theirWGPUBuffers belong to the same Dawn device even when the allocator instances differ. The missing work is to recognize"webgpu"as GPU/vendorNONEwith allocator nameWEBGPU_BUFFER, not to addSession.create_ortvalue_*,InferenceSession.update_ortvalue_inplace, Python_sessionprovenance, and blanket restrictions oncopy_tensorsand vector APIs. Please keep the existing allocation/update surface, add only the WebGPU mapping and capture-lifetime enforcement, and adaptonnxruntime/test/python/onnxruntime_test_python_cudagraph.pyinto a shared CUDA/WebGPU graph-capture test, renaming it toonnxruntime_test_python_graphcapture.pyand updatingtools/ci_build/build.pyto run it for both CUDA and WebGPU builds. Custom-external-device shared allocation should remain an explicit C++ TODO: the shared allocator/data transfer must retain and use that external device instead ofDefaultContext()before that configuration is supported.
Evidence and attribution: OrtApi::GetSharedAllocator, CreateTensorAsOrtValue, and CopyTensors are the existing public native contracts, and the Python OrtValue factories/update methods already expose that workflow. The WebGPU factory registers a shared allocator and matching transfer backed by DefaultContext(). The new session methods duplicate those operations and directly expose InferenceSession::GetDataTransferManager(), for which there is no corresponding public session-copy API. Their Python ownership model then requires new restrictions throughout run APIs, IOBinding, OrtValueVector, and copy_tensors.
C2 [P2]: Do not base safety checks on a mutable cached device marker
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:1259
Suggested comment:
[P2]Please do not cache_is_webgpu_bufferas a mutable Python attribute or add a separate_is_webgpu_bufferpybind API for it. Capture and copy validation treat this bit as authoritative, but callers can overwrite it, and the new CPU-only provenance test does exactly that; a CPU OrtValue can therefore be made to pass the fixed-WebGPU-buffer check while a real WebGPU value can be made to bypass WebGPU-specific guards. The C++device_name()change already derives"webgpu"from the tensor's actualOrtMemoryInfo, so Python can query that read-only native result when needed. Keep the private C++IsWebGpuBufferhelper for nativedata_ptr/NumPy/DLPack validation, but remove the redundant mutable Python marker/binding.
Evidence and attribution: OrtValue.__init__ copies _ortvalue._is_webgpu_buffer() into an ordinary assignable field, and _validate_ortvalue_ownership, _validate_capture_bindings, update_inplace, and copy_tensors trust that field. test_device_ortvalue_provenance_rules assigns it on CPU-backed values, demonstrating that it is not tied to the underlying tensor location.
S1 [P2]: Do not disable existing vector and copy APIs for WebGPU
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:618
Suggested comment:
[P2]Please remove the blanket WebGPU restrictions added tocopy_tensors,run_with_ortvaluevector, andOrtValueVector.push_back. The default-context WebGPU values should come from the environment-registered shared allocator and are intentionally supported by the environment-level transfer; a non-capturing WebGPU session with CPU-only vectors is also unrelated to opaque WebGPU storage. These APIs worked before this PR and are being disabled only to support the new session-owned Python abstraction. Reusing the existing shared-allocation model removes that ownership problem and preserves compatibility. Add coverage for CPU<->WebGPU and WebGPU<->WebGPUcopy_tensors, shared WebGPU vectors, and CPU-only vector execution with WebGPU registered.
Evidence and attribution: The Python guard checks only "WebGpuExecutionProvider" in self._providers; it does not inspect capture state, graph assignment, or the supplied vectors. The C++ OrtValueVector.push_back guard checks only WEBGPU_BUFFER and cannot distinguish shared from session-owned storage. Both therefore reject safe pre-existing uses in order to compensate for provenance tracked only in the Python wrapper.
T1 [P2]: Add a regression test for the foreign-GPU transfer guard
Suggested inline location: onnxruntime/core/providers/webgpu/data_transfer.cc:40
Suggested comment:
[P2]Please add focused regression coverage for the built-inDataTransfer::CanCopypredicate covering CPU<->WebGPU, WebGPU<->WebGPU, and rejection of NVIDIA/AMD GPU endpoints. This guard prevents a foreign GPU pointer from being reinterpreted and passed to Dawn as aWGPUBuffer, but the new tests do not exercise this predicate or a mixed WebGPU/CUDA session. If constructingDataTransferwithout a liveBufferManagermakes a device-free unit test impractical, please extract the device-compatibility predicate into a testable helper or cover it in the WebGPU-enabled provider tests.
Evidence and attribution: The eight-line vendor guard is the complete fix for the prior unsafe transfer selection. No current-head C++ or Python test references the built-in WebGPU DataTransfer::CanCopy behavior or registers WebGPU with another GPU provider.
Clarifications
None.
Test coverage
- C1: Current tests cover rebinding and release within one session, but not provider/session replacement while a graph is pinned.
- D1: Current tests exercise the newly added session allocation/copy surface instead of proving that the existing shared allocator,
OrtValuefactories, andcopy_tensorspath support default-context WebGPU graph capture. The existing CUDA graph test is not extended to cover WebGPU. - C2: The CPU-only provenance test mutates
_is_webgpu_bufferrather than exercising native memory metadata, which demonstrates the marker is mutable but does not validate real WebGPU provenance. - S1: Current tests assert rejection from a capture-enabled WebGPU session, but do not cover the previously supported non-capturing CPU-vector case or shared default-context WebGPU vectors.
- T1: No test exercises the new built-in WebGPU
CanCopyvendor guard. - The graph-capture behavior test still skips when WebGPU or a Dawn adapter is unavailable. The PR body reports a local WebGPU run, and all visible CI jobs now pass, including Linux WebGPU, external-Dawn, static/plugin, and macOS WebGPU builds/tests. Those results do not establish coverage for C1's provider-reset sequence, D1's existing copy path, S1's CPU-vector compatibility case, or T1's built-in transfer predicate.
- Static evidence: the three changed Python files compile, and the complete current-head diff passes
git diff --check. Runtime tests were unavailable because this managed worktree has no built Python extension or test binary.
Verdict
The feature is valid and the graph-ID/IOBinding pinning direction is appropriate, but the session-owned OrtValue direction should be removed. C1 blocks provider-reset correctness. D1 reuses the existing shared allocator, OrtValue factories, and native CopyTensors contract instead of adding Python-only session allocation/copy semantics. C2 removes a mutable duplicate of native memory metadata from safety decisions. S1 restores existing vector and copy APIs that the session-provenance design unnecessarily disables. Custom-external-device shared allocation is explicitly deferred to a C++ follow-up that makes the shared allocator retain and use the external device. T1 requests durable coverage for the safety-critical transfer fix. There are no remaining clarification requests or cleanup-only items.
CanCopy() gained a vendor guard so a foreign GPU handle (e.g. a CUDA pointer in a multi-EP build) is never reinterpreted as a WGPUBuffer, but nothing tested it. DataTransfer holds a BufferManager reference, so constructing one needs a live device. Extract the device-compatibility decision into a static IsSupportedDevicePair() helper that touches no state, and test it directly. WebGPU EP tests build into onnxruntime_provider_test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…session Session._reset_session() cleared eleven fields and recreated _sess but left _captured_graph_bindings pointing at the replaced C.InferenceSession. A capture followed by set_providers() therefore left cross-session state: the stale entry rejected a fresh IOBinding from the replacement session as "captured with a different IOBinding", the old IOBinding stayed pinned and could no longer be rebound or cleared, and a later release_captured_graph(0) released the graph on the replacement session while unpinning a binding that belonged to the old one. Release each captured graph through the session that captured it, before any teardown, so a failure leaves the session intact rather than half torn down. The CPU regression test reproduces the bookkeeping run_with_iobinding() records on capture; capture itself needs a device, but the state it leaves behind does not. The WebGPU test covers the real capture -> set_providers() -> capture again sequence end to end. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
OrtValue.__init__ cached ortvalue._is_webgpu_buffer() into an ordinary assignable attribute, and four sites trusted it: the run-path device check in Session._validate_ortvalue_ownership, IOBinding._validate_capture_bindings, the update_inplace mismatch check, and copy_tensors. Two other sites already called the native method instead, so the same question had two answers depending on where it was asked. Assigning the attribute was enough to make a CPU tensor pass graph-capture validation, which requires device-resident values; test_device_ortvalue_provenance_rules did exactly that. Delegate to the native method through a read-only property so every site resolves it the same way and provenance cannot be spoofed. The provenance test loses the half it was only simulating and keeps the session-ownership rules it genuinely exercises. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Both were hard-rejected with an ORT_ENFORCE, but both already worked and were useful before this feature branch. numpy() falls through to CreateDataTransferMemCpy(), which resolves the environment-registered WebGPU data transfer and returns correct host data, and data_ptr() returns the WGPUBuffer handle, which is exactly what a WebGPU interop caller needs. Rejecting them removed working behaviour without replacing it, so drop both guards. The DLPack guards stay: those really are broken for an opaque buffer handle, because DLPack promises a dereferenceable address. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…identity
run_with_ortvaluevector was the only run API that decided by provider name:
if "WebGpuExecutionProvider" in self._providers:
raise RuntimeError(...)
run(), run_with_ort_values() and run_async() all gate on
_validate_graph_capture_run_api() instead. So this refused a CPU-only
OrtValue vector on a WebGPU session that was not capturing at all, and it
ignored capture state, graph assignment, and whether the supplied vectors
were even device-backed. That use worked before this feature branch.
Call the same validation the other three run APIs already call. This is a
consistency fix reusing already-tested logic, not a new policy: a raw vector
is unsafe only while capture is armed, because replay re-issues the buffers
recorded at capture and would silently ignore the values passed here. The
gpu_graph_id=-1 opt-out now works here exactly as it does everywhere else.
The stated reason for the old guard -- that raw vectors "cannot retain
session ownership" -- was also untrue; see the following commit.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The guard refused every WebGPU output with "it cannot retain session
provenance". That reason is not true of this code: the vector returned by
SessionIOBinding.get_outputs is a pybind reference_internal, which is defined
as a reference plus keep_alive<0,1>, so it already keeps the IOBinding alive,
and the IOBinding already keeps the session alive via the keep_alive<1,2>
added earlier on this branch. Together with keep_alive<0,1> on
OrtValueVector.__getitem__ the chain is complete:
OrtValue -> OrtValueVector -> SessionIOBinding -> InferenceSession
So the lifetime the guard was compensating for is structurally guaranteed,
and refusing the call only removed access.
The new test pins that chain rather than the guard: it reaches a WebGPU
output through the raw vector, drops the IOBinding and the session, and then
releases the buffer. Without the full chain that release would call
GpuBufferAllocator::Free() through a buffer-manager getter capturing a dead
EP, so the test fails loudly if a future change breaks a link.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Neither message described its real hazard, and push_back's cited a property of the code that does not hold. push_back: the hazard is lifetime, and it is specific to a standalone vector. The vector returned by SessionIOBinding::get_outputs is reference_internal and so keeps the session alive, but a user-built OrtValueVector has no parent and cannot. A WebGPU buffer allocated by a session's EP is released through GpuBufferAllocator, whose buffer-manager getter captures that EP, so freeing it after the session is gone is a use-after-free. C++ cannot currently tell a session-allocated WebGPU value from one backed by a shared allocator, since their OrtMemoryInfo is identical, so the guard has to stay; the TODO ties its removal to giving GpuBufferAllocator a reference to its BufferManager's owner. push_back_batch: nothing to do with lifetime. It wraps foreign storage by raw address, and a WebGPU allocation is an opaque WGPUBuffer handle rather than an addressable pointer, so no torch tensor can back one. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The guard rejected a value if it was session-scoped OR a WebGPU buffer, and both halves were wrong. Being session-scoped is not a hazard here. copy_tensors is synchronous and the caller holds the Python wrappers, and therefore the session, for its whole duration, so there is no window in which the allocator can outlive its EP. Being a WebGPU buffer is not a hazard either. CPU-to-WebGPU, WebGPU-to-WebGPU and WebGPU-to-CPU copies all work through the environment-registered WebGPU data transfer, and worked before this feature branch. The real hazard is narrower: that data transfer resolves the *default* WebGPU context, so a buffer belonging to a caller-supplied context would be copied on the wrong wgpu::Device -- silently, because Dawn reports it on the async device error callback rather than at the call site, if validation is enabled at all. Reject exactly that, and say so. This is a partial mitigation, and the TODO in the code says so. A value can only be attributed to a context when it carries session provenance, because every WebGPU allocation reports OrtDevice(GPU, VendorIds::NONE, 0) whatever its context. Closing the hole properly needs WebGPU allocations to carry their real context id, which would also let the data transfer refuse the copy itself instead of relying on a Python-side guard. numpy() has the same exposure and no session to consult, so it is unaffected either way. WebGpuExecutionProvider::GetDeviceId() already returns the context id, so the new binding is a plain getter returning -1 when the session has no WebGPU EP. A real second context needs caller-supplied Dawn instance and device handles, which Python cannot produce, so the rejection branch is covered by unit-testing the predicate with an injected context id. That test runs on CPU-only machines, which matters because no CI leg runs this file on a WebGPU host. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Jiajia Qin (qjia7)
left a comment
Jiajia Qin (qjia7)
left a comment
There was a problem hiding this comment.
Review frame
- Review contract: Re-reviewed PR 32074 at exact head
364fad2d9d7516371704099af29ba256b58a6047, with base
3d3cfa6c004551ee31d3acff6d01fb572712ca91, and specifically audited the four-commit range
48a2b3c66ae1385424f2c08fd8ea2d8f13d407d2..364fad2d9d7516371704099af29ba256b58a6047.
Current metadata, discussion, reviews, inline comments, CI, the incremental diff, and the complete PR diff were fetched
read-only. - Problem/feature validity: Validated. WebGPU graph replay retains recorded bind groups and buffer handles, so Python
needs fixed device buffers whose lifetime spans capture/replay, in-place input updates, explicit readback, and graph
release. - Risk/scope: Deep. The PR adds public Python allocation/copy APIs, graph-ID and IOBinding lifetime state,
allocator/data-transfer routing, provider-reset behavior, and restrictions on existing OrtValue APIs. - Direction gate: Pass for graph-ID/IOBinding pinning and lifecycle enforcement, but fail for the parallel
session-owned OrtValue abstraction. For WebGPU without a custom external device, the environment shared allocator,
its transfer, and session allocators all useWebGpuContextFactorycontext 0 and therefore the same Dawn device.
ExistingOrtValuefactories andcopy_tensorsalready expose the required native shared-allocation/copy contracts.
Custom-external-device shared allocation remains a native C++ TODO: shared allocators/transfers must retain and use
the external device instead ofDefaultContext()before that configuration is supported.
Prior issue resolution
- Captured graph/IOBinding identity and rebinding: Resolved. A successful capture pins its IOBinding and buffer
signature until native graph release succeeds. gpu_graph_id=-1capture opt-out: Resolved. Convenience run APIs preserve the native per-run skip behavior.- Foreign GPU handles selected by WebGPU
DataTransfer: Resolved. WebGPU rejects vendor-tagged GPU endpoints. - Reproducible benchmark baseline: Resolved. Capture and no-capture modes are both available and reported.
- Captured state surviving
set_providers(): Resolved in0f42878a4b._reset_session()releases every captured
graph through the old native session before replacing it, and new synthetic and real WebGPU tests cover reset and
recapture. - Mutable
_is_webgpu_buffersafety marker: Resolved in9588dadf6b. Python now exposes a read-only property that
queries native tensor memory metadata on every access; the spoofing test was removed. - Missing
CanCopyregression test: Resolved in53d701e238. The device-pair predicate is directly tested for
CPU/WebGPU endpoints and NVIDIA, AMD, Intel, and Microsoft GPU rejection; the new source is included by the WebGPU
provider-test source glob. - WebGPU
numpy()anddata_ptr(): No new finding.364fad2d9drestores behavior that predated this branch.
numpy()uses the environment-registered transfer for normal context-0 buffers, whiledata_ptr()preserves the
established opaqueWGPUBuffer-handle interop behavior. DLPack remains correctly rejected because it requires a
dereferenceable address.
Confirmed findings
D1 [P2]: Reuse the existing OrtValue allocation and copy APIs
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:489
Suggested comment:
[P2]This direction remains unresolved at the new head. Please use the existing
OrtValue.ortvalue_from_shape_and_type/ortvalue_from_numpyfactories andcopy_tensorspath instead of adding
parallel session-specific allocation and copy APIs. CUDA graph capture already uses those APIs. For WebGPU without
a custom external device, the environment-registered shared allocator, its data transfer, and session allocators all
resolve throughWebGpuContextFactorycontext 0, so theirWGPUBuffers belong to the same Dawn device even when
allocator instances differ. The missing work is to recognize"webgpu"as GPU/vendorNONEwith allocator name
WEBGPU_BUFFER, not to addSession.create_ortvalue_*,InferenceSession.update_ortvalue_inplace, Python
_sessionprovenance, and the restrictions that follow from that ownership model. Please keep the existing
allocation/update surface, add only the WebGPU mapping and capture-lifetime enforcement, and adapt
onnxruntime/test/python/onnxruntime_test_python_cudagraph.pyinto a shared CUDA/WebGPU graph-capture test, renaming
it toonnxruntime_test_python_graphcapture.pyand updatingtools/ci_build/build.pyto run it for both CUDA and
WebGPU builds. Custom-external-device shared allocation should remain an explicit C++ TODO: the shared
allocator/data transfer must retain and use that external device instead ofDefaultContext()before that
configuration is supported.
Evidence and attribution: OrtApi::GetSharedAllocator, CreateTensorAsOrtValue, and CopyTensors are the existing
public native contracts, and the Python OrtValue factories/update methods already expose that workflow. The WebGPU
factory registers a shared allocator and matching transfer backed by DefaultContext(). The new session methods
duplicate those operations and directly expose InferenceSession::GetDataTransferManager(), for which there is no
corresponding public session-copy API. The four new commits do not change these APIs or generalize the existing CUDA
graph test; tools/ci_build/build.py:1839-1841 still invokes
onnxruntime_test_python_cudagraph.py only when args.use_cuda.
S1 [P2]: Do not disable existing vector and copy APIs for WebGPU
Suggested inline location: onnxruntime/python/onnxruntime_inference_collection.py:618
Suggested comment:
[P2]Please remove the blanket WebGPU restrictions added tocopy_tensors,run_with_ortvaluevector, and
OrtValueVector.push_back. A non-capturing WebGPU session with CPU-only vectors is unrelated to opaque WebGPU
storage, and default-context WebGPU values should come from the environment shared allocator and are intentionally
supported by the environment-level transfer. These APIs worked before this PR and are disabled only to compensate
for the new session-owned Python abstraction. Reusing the existing shared-allocation model removes that ownership
problem and preserves compatibility. Add coverage for CPU<->WebGPU and WebGPU<->WebGPUcopy_tensors, shared
WebGPU vectors, and CPU-only vector execution with WebGPU registered.
Evidence and attribution: Session.run_with_ortvaluevector() rejects solely because WebGPU is registered, without
inspecting capture state or any supplied vector. OrtValueVector.push_back() rejects every native
WEBGPU_BUFFER, including shared context-0 values. Python copy_tensors() likewise rejects every WebGPU value before
calling the existing native C.copy_tensors. The new commits leave all three restrictions unchanged.
Clarifications
None.
Test and CI evidence
- The new provider-reset tests directly cover stale capture bookkeeping and real WebGPU capture ->
set_providers()
-> recapture. - The read-only marker test confirms assignment is rejected and native memory metadata remains authoritative.
WebGpuDataTransferTestdirectly covers the safety-critical device predicate and builds through
onnxruntime_provider_test.- Coverage still exercises the new session allocation/copy surface rather than proving graph capture with the existing
sharedOrtValuefactories andcopy_tensors. The existing CUDA graph test has not been generalized or added to the
WebGPU runner. - All visible WebGPU jobs passed, including Linux x64, macOS arm64/x64, static/plugin, and external-Dawn builds/tests.
The only failed check is the unrelated Linux TensorRT test job:
onnxruntime_global_thread_pools_testhitcudaErrorMemoryAllocation: out of memory. - The complete current-head diff passes
git diff --check, and the changed Python files compile. Runtime tests were
unavailable locally because this managed worktree has no built Python extension or test binary.
Verdict
The four new commits correctly resolve all three outstanding correctness/test gaps they target, and restoring
numpy()/data_ptr() preserves prior WebGPU behavior without weakening the DLPack guard. No new regression was found.
The review remains blocked on one architectural root cause and its compatibility fallout: D1 should remove the
Python-only session allocation/copy layer in favor of the existing native shared allocator and CopyTensors
contracts, and S1 should restore the existing copy/vector APIs. Doing so would make the implementation substantially
clearer by removing _session provenance, duplicate creation/update bindings, and the blanket restrictions and tests
needed to maintain that parallel model.
BufferManager::MemCpy reports misuse with ORT_ENFORCE, which throws: an aliased src/dst pair, an undersized destination and a still-mapped buffer all throw rather than returning a Status. The shared data transfer reaches MemCpy through WebGpuDataTransferImpl::CopyTensorsImpl, which is a noexcept C ABI callback and only handled a returned non-OK Status. The exception therefore escaped a noexcept frame and std::terminate killed the process with no message and no stack. Wrap the callback body so any throw becomes an OrtStatus, which is what the surrounding C API already expects. This was unreachable from Python while copy_tensors rejected every WebGPU OrtValue outright. Allowing the supported copies exposes it, so it is fixed and pinned here rather than left for the first caller to hit: copy_tensors([a], [a]) aliased -> was a silent 0xC0000409, now RuntimeError copy_tensors([64x64], [2x2]) undersized -> was a silent 0xC0000409, now RuntimeError Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Thanks for the review. The three S1 restrictions are gone as of 91541a5: run_with_ortvaluevector now gates on capture state rather than provider name, the get_outputs_as_ortvaluevector guard is dropped, and copy_tensors only rejects values attributable to a non-default WebGPU context, so CPU to WebGPU and WebGPU to WebGPU copies all work now, with tests. Your review was right that the earlier commits left these unchanged. They were written but unpushed at that head. Removing the copy_tensors restriction also surfaced a crash worth flagging: copy_tensors with aliased src/dst, or an undersized destination, silently killed the process. BufferManager::MemCpy reports misuse via ORT_ENFORCE, which throws, but it is reached through CopyTensorsImpl, a noexcept C-ABI callback, so the throw hit std::terminate. Pre-existing, but only reachable once WebGPU values are allowed through, so I fixed it at the boundary in the same push. Two guards stayed: push_back, because a standalone vector cannot keep the session alive the way the IOBinding one does (which is why that could be unblocked and this could not), and push_back_batch, because it wraps raw pointers and a WebGPU allocation is an opaque handle. On D1 you are right about the mechanism, and I checked rather than assumed: without a custom external device everything does resolve through context 0. The problem is that path is not reachable from the Python wheel build. cmake/onnxruntime_providers_webgpu.cmake:22-32 excludes ep/ when onnxruntime_USE_EP_API_ADAPTERS is off, and ep/factory.cc:107-108 holds the only EpDevice_AddAllocatorInfo calls, so the OrtEpDevice registers with no allocator and ortvalue_from_shape_and_type(..., "webgpu") throws while the session-owned one works. That is also why the shared graph-capture test is circular today: both the cudagraph and dmlgraph tests are built on those same global factories, so a WebGPU arm can only be written right now using the API you are asking me to remove. I would rather land the cmake fix first and move this layer onto it, but happy to file that follow-up, or do it in the other order if you prefer. |
Jiajia Qin (qjia7)
left a comment
Jiajia Qin (qjia7)
left a comment
There was a problem hiding this comment.
I think we can limit this PR to basic WebGPU graph-capture support for the default context:
- Allocate fixed GPU input/output tensors through the session allocator using
Session.create_ortvalue_from_shape_and_type. - Upload CPU inputs with the environment-level
onnxruntime.copy_tensors. - Bind those GPU tensors with IOBinding and capture/replay the graph.
- Copy the GPU output into a CPU OrtValue with
onnxruntime.copy_tensorsand verify the result.
The native session allocator is already a public C/C++ API (OrtApi::CreateAllocator), so exposing the minimal Python allocation method is reasonable. For the default WebGPU context, the session allocator and environment data transfer both use context 0, so this workflow does not require an environment shared allocator.
Please keep the Python surface minimal: retain Session.create_ortvalue_from_shape_and_type and copy_tensors, but use allocation-plus-copy instead of adding Session.create_ortvalue_from_numpy and the session-specific update_ortvalue_inplace path. Shared-allocator support and adapting onnxruntime/test/python/onnxruntime_test_python_cudagraph.py into a common CUDA/WebGPU graph-capture test can be handled in a follow-up after static WebGPU shared-allocator registration is available.
Custom external WebGPU contexts should remain unsupported here with a native TODO.
…pdate path A device tensor is allocated from the session allocator and populated with the environment-level copy_tensors, rather than through a session-specific upload API. For the default WebGPU context the session allocator and the environment data transfer both resolve context 0, so the two compose without an environment shared allocator. Removes Session.create_ortvalue_from_numpy and the session-routed update_ortvalue_inplace path, which duplicated allocate-plus-copy behind a second surface. OrtValue.update_inplace keeps its pre-existing non-session behaviour; only the session-routed branch is gone, along with the pybind overloads and the CopySessionOrtValue/UpdateSessionOrtValue helpers. Session.create_ortvalue_from_shape_and_type and copy_tensors are retained: the native session allocator is already public as OrtApi::CreateAllocator. Tests follow the supported flow end to end: allocate fixed device tensors, upload with copy_tensors, bind with IOBinding and capture/replay, then copy the device output back to a CPU OrtValue and verify. The CPU-ownership test now covers allocation and provenance only, since a CPU-to-CPU copy depends on a data transfer that is not registered in every build. Custom external WebGPU contexts remain unsupported, with the native TODO stating that shared allocators and transfers must retain and use the external device instead of resolving DefaultContext(). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ability test_webgpu_graph_capture_session_ortvalues builds a session with disable_cpu_ep_fallback and requests the WebGPU EP, so on a build without that EP the nodes have nowhere to go and session creation fails with "graph nodes that are assigned to the default CPU EP, but fallback to CPU EP has been explicitly disabled". The test lost its @unittest.skipIf when a later test was inserted directly beneath the decorator, which rebound the decorator to the new function. Restore it so the test skips on CPU-only builds, as the other WebGPU tests do. The failure is invisible to a runtime check on a machine where the WebGPU EP is compiled in, because the session simply succeeds there. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…wnership Extract GetDeviceAllocatorName next to GetDeviceName and use it from both CreateSessionOrtValue and the IOBinding paths, replacing the duplicated GPU + VendorIds::NONE -> WEBGPU_BUFFER selection. Drop the unconditional 'WebGPU OrtValues must be used with IOBinding' rejection. It also fired on a WebGPU session that never enabled graph capture, where a device OrtValue feed is an ordinary run. Accept a WebGPU buffer owned by another session on the same WebGPU context. Move the data-transfer predicate and the copy_tensors misuse fix out of this PR; they are covered by microsoft#32315 and a separate branch. Drop the graph-capture benchmark script. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
bind_ortvalue_input and bind_ortvalue_output still used the strict same-session test, so a WebGPU buffer accepted by run() was rejected by the binding path, which is the path device values are most likely to take. Move the rule into _validate_ortvalue_session_compatibility and call it from all three sites. The verb is a parameter so the run and bind messages stay distinct. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
update_inplace still required the exact same session, so a WebGPU buffer that bind_ortvalue_input accepted could not be copied into a value owned by the target session. Both are device-to-device operations on one context, so split the decision into _is_ortvalue_session_compatible and have the raising helper and update_inplace share it. The predicate treats a null target session as compatible, which keeps a sessionless destination working and preserves the previous behaviour. update_inplace keeps its own message because it compares two values rather than a value against a session. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
#32315) ### Description `BufferManager::MemCpy` reports misuse with `ORT_ENFORCE`, which throws: an aliased src/dst pair, an undersized destination and a still-mapped buffer all throw rather than returning a Status. The shared data transfer reaches it through `WebGpuDataTransferImpl::CopyTensorsImpl`, which is a `noexcept` C ABI callback and only handled a returned non-OK Status. The exception escaped a `noexcept` frame, so `std::terminate` took the process down with no message and no stack. This wraps the callback body so any throw becomes an `OrtStatus`, which is what the surrounding C API already expects. The two sibling callbacks are deliberately left alone: `CanCopyImpl` only calls C ABI function pointers, and `ReleaseImpl` returns void, so converting a throw there would mean swallowing it rather than reporting it, which is a separate decision from this one. ### Motivation and Context Not currently reachable from Python, because `copy_tensors` rejects every WebGPU OrtValue outright. It becomes reachable as soon as the supported copies are allowed, which is what #32074 does, so it is worth fixing at the boundary on its own rather than landing inside a larger change. The regression test for it uses the session-scoped allocation APIs under discussion in #32074 and is held there rather than duplicated here. Co-authored-by: Ananya Anand <4n4ny4@users.noreply.github.com>
Description
Adds a safe way to use WebGPU graph capture from Python. Replay reuses the exact buffers recorded at capture, and Python had no way to allocate those buffers, update them, or release a captured graph so anything you tried either failed or quietly returned stale results. This adds session-owned WebGPU
OrtValues (plus support for ones from a shared allocator, which have no session), in-place updates, andrelease_captured_graph(id), and the first capturing run now pins itsIOBindingso rebinding is rejected instead of silently writing to the original buffers. There's also one C++ change: WebGPU'sCanCopyaccepted any GPU device pair without checking vendor ID, so in a session with WebGPU and CUDA registered a CUDA copy would get routed to WebGPU and handed to Dawn as aWGPUBuffer. WebGPU now refuses GPU buffers from other vendorsMotivation and Context
Graph capture already exists in the WebGPU EP, but there was no supported way to reach it from Python. You could turn the session option on, but without fixed device buffers the results were wrong without raising an error — replay just kept using whatever was bound the first time.
On an NVIDIA TITAN V, capture takes YOLO26n from 5.885 ms to 5.475 ms p50, about 7%. Both numbers come from the script in this PR using its defaults, 5 fresh processes per arm with the arms alternating.
Tested locally against a WebGPU build on the TITAN V: 80 passed, 2 skipped (no
onnxinstalled), 3 failures that are pre-existing and unrelated (missing LoRA test data, custom-ops library not built).