Repository navigation
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Qwen-Image-2.1's first VAE decode can spend over 13 minutes benchmarking MIOpen convolution solvers on Radeon 8060S, while the next decode takes about two seconds. Cache isolation traced the delay to solver search: retaining the solver database removes it, but retaining only compiled kernels does not.
Fix
Set
MIOPEN_SEARCH_CUTOFF=1during Studio's existing lazy ROCm hardware detection, before model convolutions. MIOpen can skip excessively slow search candidates while retaining its naive fallback solver.Enable the default only when every visible Torch GPU reports gfx1151. Preserve explicit environment values, including
0; mixed, unknown and other architectures keep their existing defaults. Users on other ROCm GPUs can opt in with the same environment variable. The setting is process-wide and inherited by workers.Tests
Live A/B against main at
4e8503e437on Radeon 8060S / gfx1151, Windows and Linux, PyTorch 2.11 / ROCm 7.13. Each OS runs stock and the actual patch on the same runner with separate empty MIOpen caches. Qwen-Image-2.1 Q4_K_M, Auto memory, 1024x1024, 40 steps, two prompts, seed 42, guidance 1.0 and a 19 GiB Torch allocation cap.All four paired images and decoder input tensors are bit-identical. Cold peak Torch GPU allocation stays 14.92 GiB; warm decode stays around 1.5-2 seconds. The probes confirm that startup sets the flag automatically, with no external override in the patched cases. Load/download time is excluded. Denoiser timing also varied, so the VAE phase is the direct measure of this fix.
Live actual-patch A/B
Separate solver validation passed 468 CPU-reference forward/gradient comparisons across FP32, FP16, BF16, grouped/depthwise/pointwise/3D convolutions and multiple layouts. Solver validation
Rebased onto main at
b1c522a4ea; 308 related hardware and startup tests passed, 36 skipped. The live A/B above was run on the stated earlier base and was not repeated after this rebase. Coverage includes explicit overrides, mixed and unknown architectures, probe failures, ROCm SDK version detection, CUDA exclusion and lazy startup. Ruff, formatting and diff checks passed.Scope
Test hardware is Radeon 8060S / gfx1151 with shared memory. The allocation cap does not emulate a discrete GPU's transfer costs. RX 7900 XT remains untested. This changes cold solver-search behavior, not Auto placement, model-weight memory use or vision-attention correctness.