Fix webgpu shared buffer issue - #2191
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a WebGPU shared-buffer issue in RecurrentState by introducing a dual-mode buffer strategy: alias input/output to the same buffer when graph capture is active (TRT-RTX), or use separate past/present buffers with swap on each step for general EPs (e.g., WebGPU, which prohibits aliasing the same buffer as read-only and read-write). The implementation mirrors the established pattern in DefaultKeyValueCache.
Changes:
- Add
share_buffers_flag (derived fromparams_->use_graph_capture) andinput_index_/output_index_tracking toRecurrentState. - Conditionally allocate/zero a separate
pasts_vector in non-shared mode;Add()binds input topasts_vs.presents_accordingly. - Implement swap-and-rebind in
Update()and zero+rebind inRewindTo(0)for the non-shared path; shared path is unchanged.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/models/recurrent_state.h | Adds pasts_, share_buffers_, and index members for non-shared buffer tracking. |
| src/models/recurrent_state.cpp | Allocates separate pasts when not in graph-capture mode; swaps/rebinds buffers in Update() and RewindTo(). |
apsonawane
enabled auto-merge (squash)
June 1, 2026 04:41
baijumeswani
reviewed
Jun 1, 2026
baijumeswani
reviewed
Jun 1, 2026
baijumeswani
reviewed
Jun 1, 2026
baijumeswani
approved these changes
Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request refactors the management of recurrent state buffers in
RecurrentStateto better support both graph capture (e.g., TRT-RTX) and general execution providers. The main change is to conditionally use shared or separate buffers for past/present state, improving compatibility and correctness across execution environments.Recurrent state buffer management:
share_buffers_flag toRecurrentStateto determine whether to use the same buffer for both input and output (for graph capture) or separate buffers with swapping (for general EPs like WebGPU). [1] [2]pasts_andpresents_vectors conditionally based onshare_buffers_, ensuring correct buffer usage and initialization.AddandUpdatemethods to handle buffer aliasing and swapping according to the execution mode, maintaining stable buffer addresses for graph capture and proper buffer updates otherwise.RewindTomethod to zero and rebind buffers as needed, supporting both shared and separate buffer modes.Internal state tracking:
input_index_andoutput_index_members to track the range of recurrent state buffers within the global input/output vectors, enabling efficient buffer rebinding after rewinding or updating. [1] [2]