Skip to content

Enable graph capture for WebGPU models and DML continuous decoding tests - #2099

Merged
baijumeswani merged 17 commits into
mainfrom
enable-graph-capture-webgpu-ci
May 19, 2026
Merged

Enable graph capture for WebGPU models and DML continuous decoding tests#2099
baijumeswani merged 17 commits into
mainfrom
enable-graph-capture-webgpu-ci

Conversation

@qjia7

@qjia7 qjia7 commented Apr 24, 2026

Copy link
Copy Markdown
Contributor
  • Add enable_graph_capture flag to model download config in _test_utils.py
  • When enable_graph_capture=True and device=webgpu, pass enable_webgpu_graph=true to the model builder so generated models have enableGraphCapture=1
  • Enable graph capture for qwen-2.5-0.5b model (used by guidance tests)
  • Remove #if !USE_DML guard from multi-turn guidance tests since PR [WebGPU] Support continuous decoding (RewindTo) with graph capture #2083 fixed RewindTo for DML (DML always uses graph capture/static mask)

qjia7 added 3 commits April 24, 2026 14:40
@qjia7
Enable graph capture for WebGPU models and DML continuous decoding tests
1defb8e 
- Add enable_graph_capture flag to model download config in _test_utils.py
- When enable_graph_capture=True and device=webgpu, pass enable_webgpu_graph=true
  to the model builder so generated models have enableGraphCapture=1
- Enable graph capture for qwen-2.5-0.5b model (used by guidance tests)
- Remove #if !USE_DML guard from multi-turn guidance tests since PR #2083
  fixed RewindTo for DML (DML always uses graph capture/static mask)
@qjia7
Fix clang-format whitespace in guidance_tests.cpp
0889c74
@qjia7
Add RewindQwen25 C++ test for rewind with qwen-2.5 model
d12b57e 
Test RewindTo with the qwen-2.5 model. Exercises the static mask rewind
path if the EP supports it (DML by default, WebGPU with enableGraphCapture=1
set by _test_utils.py), otherwise falls back to the dynamic mask path.

Guarded by TEST_QWEN_2_5, skipped when the model is not available.
Verified: RewindQwen25CAPI passes (full rewind + partial rewind).
@qjia7
qjia7 marked this pull request as ready for review April 27, 2026 02:58
Copilot AI review requested due to automatic review settings April 27, 2026 02:58
Copilot AI reviewed Apr 27, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates test infrastructure to generate WebGPU models with graph capture enabled (via model-builder extra options) and expands continuous decoding (RewindTo) test coverage to include the qwen-2.5 test model, aligning with recent backend fixes.

Changes:

  • Extend test/python/_test_utils.py model download configuration to carry a per-model “graph capture” flag and pass enable_webgpu_graph=true when building WebGPU models.
  • Enable WebGPU graph capture for the qwen-2.5-0.5b model used by guidance-related tests.
  • Add a new C API test that validates RewindTo behavior on the qwen-2.5 model, and adjust DML-related guards in guidance tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
test/python/_test_utils.py Adds a per-model graph-capture flag and wires it to the model builder via --extra_options enable_webgpu_graph=true for WebGPU builds.
test/guidance_tests.cpp Removes the DML continuous-decoding guard from the (currently disabled) multi-turn guidance tests.
test/c_api_tests.cpp Adds a qwen-2.5-based RewindTo regression test intended to exercise static-mask rewind when supported.

Comment thread test/python/_test_utils.py Outdated
Comment thread test/c_api_tests.cpp Outdated
qjia7 added 2 commits April 27, 2026 13:49
@qjia7
Address PR #2099 review: fix GetModelPath cache bug and clarify comments
b579c33 
- Fix GetModelPath to cache per model_type using unordered_map instead of
  a single static string. Previously, whichever model resolved first would
  be returned for all subsequent calls regardless of model_type.
- Add comment clarifying enable_graph_capture currently only affects WebGPU EP.
- Disable RewindQwen25CAPI for CUDA builds (USE_CUDA guard) pending investigation.
- Use value check (#if !USE_CUDA) instead of #if !defined(USE_CUDA) since
  CMake always defines USE_CUDA as 0 or 1.

Verified: All 29 CAPITests pass (20 passed, 9 skipped).
@qjia7
Upgrade OnnxRuntime Foundry package to 1.26.0-dev-20260410-0804-ce913…
9371428 
…76bdf
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/python/_test_utils.py Outdated
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/python/_test_utils.py
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/python/_test_utils.py Outdated
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/python/_test_utils.py Outdated
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/c_api_tests.cpp
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/guidance_tests.cpp
kunal-vaishnavi
kunal-vaishnavi reviewed Apr 28, 2026
View reviewed changes
Comment thread test/test_utils.h
qjia7 added 4 commits April 29, 2026 13:30
@qjia7
webgpu: resolve PR 2099 review comments
f23b7eb 
Address reviewer feedback in test infra and coverage updates.

- Keep enable_graph_capture generic, remove default arg, and pass graph_capture explicitly from callers.
- Move WebGPU-specific behavior note to implementation site where enable_webgpu_graph is applied.
- Clarify CUDA exclusion comment for RewindQwen25CAPI: known pre-existing dynamic mask issue, to be fixed in a separate PR.
- Restore DML continuous-decoding guard in guidance tests.
- Clarify model path probing rationale and keep per-model path cache fix in test utils.
@qjia7
test: pass graph capture flag in whisper e2e model build
89696ac 
Fix CodeQL too-few-arguments warning after download_model signature update.

- Add enable_graph_capture=False to whisper-tiny download_model call in Python E2E test.
- Keep behavior unchanged for whisper model generation.

Verified: python -m py_compile test/python/_test_utils.py test/python/test_onnxruntime_genai_e2e.py
@qjia7
Re-enable RewindQwen25CAPI test for CUDA EP
1868ebf 
Remove the !USE_CUDA guard so the test runs on all EPs in CI.
@qjia7
webgpu: keep CUDA gate for qwen rewind test
34c2330 
Keep RewindQwen25CAPI excluded on CUDA to avoid CI regression.

Document that the CUDA failure is a pre-existing issue exposed by this PR and should be fixed in a separate follow-up PR.
@qjia7
qjia7 requested a review from kunal-vaishnavi May 8, 2026 02:05
@qjia7

qjia7 commented May 8, 2026

Copy link
Copy Markdown
Contributor Author

@kunal-vaishnavi Please take another look. The windows CUDA CI seems not stable.

qjia7 added 2 commits May 11, 2026 10:19
@qjia7
Merge branch 'main' into enable-graph-capture-webgpu-ci
81d2727
@qjia7
test: enable CUDA graph-capture path for Qwen2.5 rewind
35d2e81 
Enable CUDA graph-capture model generation in test/python/_test_utils.py by mapping enable_graph_capture to enable_cuda_graph for CUDA builds.

Remove the CUDA exclusion from CAPITests.RewindQwen25CAPI and keep the test body provider-agnostic by relying on generated model config.

Update position input comments to reflect CUDA static-mask handling when enable_cuda_graph is active.
@qjia7
qjia7 requested a review from a team as a code owner May 11, 2026 02:38
qjia7 added 3 commits May 13, 2026 09:18
@qjia7
Merge branch 'main' into enable-graph-capture-webgpu-ci
265061d
@qjia7
test: split Qwen2.5 into baseline and graph-capture test models
6ca698c 
- Keep qwen-2.5-0.5b as baseline (no graph capture) for general API tests
  (hidden_states, multi-token prefill, batch>1)
- Add qwen-2.5-0.5b-graph with graph capture enabled for graph-capture EPs
  (WebGPU, CUDA, DML, NvTensorRtRtx)
- RewindQwen25CAPI prefers graph model, falls back to baseline
- Skip graph-capture model generation on unsupported devices (e.g. CPU)
@qjia7
test: disable RewindQwen25CAPI on CUDA and add TODO for investigation
cfa23c7 
Disable RewindQwen25CAPI test on CUDA using preprocessor guard (#if TEST_QWEN_2_5 && !USE_CUDA)
since the rewind operation fails on CUDA for qwen-2.5 model with both static mask (graph-capture
variant) and non-static mask (baseline variant).

Added TODO comment to track the need for investigating root cause of this failure.
Test continues to run on WebGPU and DML where it works correctly.
@qjia7

qjia7 commented May 13, 2026

Copy link
Copy Markdown
Contributor Author

@kunal-vaishnavi @baijumeswani It seems that the CUDA CIs still fail.

qjia7 added 2 commits May 15, 2026 13:35
@qjia7
Merge branch 'main' into enable-graph-capture-webgpu-ci
6989829
@qjia7
Skip graph-capture model download on CUDA CI
6bef16f 
Remove "cuda" from _GRAPH_CAPTURE_DEVICES to avoid HF model
download failures in CUDA CI pipeline.
baijumeswani
baijumeswani reviewed May 19, 2026
View reviewed changes
Comment thread test/python/_test_utils.py
@baijumeswani
baijumeswani enabled auto-merge (squash) May 19, 2026 20:32
@baijumeswani
baijumeswani merged commit ca9ecd2 into main May 19, 2026
16 of 17 checks passed
@baijumeswani
baijumeswani deleted the enable-graph-capture-webgpu-ci branch May 19, 2026 21:20
@dependabot dependabot Bot mentioned this pull request May 30, 2026
Closed
@dependabot dependabot Bot mentioned this pull request Jun 7, 2026
Merged
This was referenced Aug 1, 2026
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@baijumeswani baijumeswani baijumeswani approved these changes
@kunal-vaishnavi kunal-vaishnavi kunal-vaishnavi approved these changes

Assignees

No one assigned

Labels

0.14.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@qjia7 @baijumeswani @kunal-vaishnavi