gpui_windows: Recover renderers after DirectX device loss

[railapex]

Objective

Closes #52085

Prevent a recoverable DirectX device removal or reset from aborting the process inside the Windows window procedure.

We reproduced the same fatal callback path without changing the installed driver: a debug-only fault injector enters the real vsync and WM_GPUI_GPU_DEVICE_LOST path, then forces window recovery to fail. The existing handler panics after its recovery attempts; Windows terminates the process with 0xC0000409.

Solution

• Give every Windows renderer one atomic state: a complete active bundle or suspended. No partially rebuilt device, swapchain, pipeline, target, or atlas becomes visible.
• Suspend every live window before publishing replacement global DirectX and DirectWrite state. Windows created during that generation start suspended.
• Build each window candidate off to the side, then commit it only if the recovery generation, drawable size, and lifetime still match.
• Retry from normal vsync deadlines (0, 100, 250, 500, 1000, 2000, 4000, 8000 ms) without sleeping the vsync thread. Exhaustion leaves the window suspended and the process alive.
• Make renderer re-entry explicit. Recovery messages use try_borrow_mut and return deferred; the vsync-owned coordinator retries the pending window after the active draw, present, or resize releases its borrow.
• Force the first recovered frame before publishing the active bundle, so it cannot replay atlas tiles from the removed device. Offscreen rendering is likewise rejected while recovery is suspended.
• Add generation, window, attempt, size, compositor mode, adapter, and chained Windows-error diagnostics under the stable gpui_device_loss prefix.
• Replace the fixed 350 ms post-loss sleep with immediate global recovery and non-blocking retry deadlines. Global creation keeps retrying with an 8 s cap; each window gets eight failed candidate attempts, while stale resize races do not spend that budget.

This follows Microsoft's lost-device contract rather than inventing a new renderer lifecycle. Direct3D 11 requires replacing the device and every device-dependent resource, including swapchains. DirectComposition likewise requires a new DXGI device, DirectComposition device, and content; its pending commands are committed atomically. Win2D exposes the same transition as DeviceLost followed by CreateResources(NewDevice). GPUI's extra machinery handles its multi-window and re-entrant Win32 shape: unpublished candidates, generation/size/lifetime validation, and a required fresh first frame keep old device state from crossing the recovery boundary.

Sources: Microsoft's Direct3D 11 device-loss guidance, DirectComposition device-state contract, atomic DirectComposition commits, and Win2D device-loss handling.

Deterministic test seam

Debug builds accept:

• GPUI_TEST_DEVICE_LOSS_AT_VSYNCS=<n>[,<n>...]
• GPUI_TEST_DEVICE_RECOVERY_FAILURE=<stage>:<count>

The seam is inert outside debug builds. It injects at the real vsync and recovery boundaries; it does not install, reset, disable, or otherwise touch a display driver.

Unit tests cover trigger parsing, all seven candidate stages, a fresh failure budget per generation, deferred attempts, retry deadlines, exact eight-attempt exhaustion, active/destroyed terminal states, and recycled HWND registration. They are ordinary inline Rust tests next to the private recovery types and run in Zed's Windows cargo nextest job.

script/test-gpui-windows-device-loss.ps1 is the native integration test. It builds two existing GPUI examples, launches one owned process per scenario, waits for structured gpui_device_loss_test records, asserts that the process is still alive at the expected terminal result, requires every recovered window's forced fresh frame to complete a successful Present(), and stops that exact child through its process handle. It is intentionally separate from nextest because it needs an interactive Windows desktop and a real DirectX/DirectComposition window.

Related work and scope boundaries

• #34374 introduced the DirectX 11 backend and its device-loss recovery. This PR keeps that recovery entry point but makes publication atomic and failure nonfatal.
• #42114 removed unsound ManuallyDrop bookkeeping and deliberately panicked when recovery could not complete. This PR preserves the sound ownership model while replacing the panic with an explicit suspended state and bounded retries.
• #55878 fixed stale atlas replay after a successful recovery by forcing a full first frame. This PR keeps that invariant, moves the forced-frame flag before active-state publication, and also prevents atlas access while recovery is incomplete.
• #50898 made the Linux wgpu renderer recover from device loss, and #52389 forced its first recovered scene to rebuild. The forced-scene invariant is shared here. The recovery implementation is not: Windows owns DirectX/DirectWrite globals, per-HWND swapchains, and synchronous window messages that the wgpu path does not have.
• #36798 records the same DXGI_ERROR_DEVICE_HUNG/device-recovery family and a later atlas index panic. The atlas symptom was addressed by Fix DirectX atlas panic after GPU device recovery #55878; this PR addresses the remaining partial-recovery and fatal-exhaustion path.
• #49956 is primarily a remote-host report, so this PR must not claim to fix it. Its attached log is useful corroboration: failed resize, missing render target, repeated device recreation, and a later hang are the inconsistent intermediate state this PR prevents.
• #50972 proposes WARP for machines that cannot acquire a hardware adapter. That is an initial adapter-selection/fallback problem, not recovery of existing windows after a removed device. WARP remains compatible with this state machine but is not needed to fix the reproduced failure.
• #61469 is now addressed by #63489, which adds bounded outer-loop paint and input fairness. It does not enter device loss or rebuild DirectX state. The exact submitted gpui_windows: Prevent paint and input starvation under sustained messages #63489 head cherry-picks cleanly over this branch and passes this PR's full 12-scenario recovery harness. The fixes remain separate because their triggers and state are unrelated.
• #63182 makes Windows frame scheduling demand-driven and adds lifecycle-safe per-window request state. It touches device-recovery wakeup and overlapping Windows files, but solves idle scheduling rather than rebuilding removed DirectX state. Neither PR should absorb the other; whichever lands second should rebase and rerun its Windows gates.
• #60295 is already in this branch's base and prevents re-entrant draw corruption. This recovery code respects that invariant by deferring renderer borrows; it does not replace or extend the arena/draw-coordinator fix.

Testing

Before, against public injector-only baseline commit 16ce921398:

$env:GPUI_TEST_DEVICE_LOSS_AT_VSYNCS = '30'
$env:GPUI_TEST_DEVICE_RECOVERY_FAILURE = 'resources:8'
./target/debug/examples/hello_world.exe

Three consecutive runs entered the existing fatal handler and exited 0xC0000409 with:

Device lost: DirectXRenderer failed to recover from lost device after multiple attempts

After:

• resources:8: eight failures, final result=exhausted; process remained alive with the window suspended.
• resources:7: attempt eight reached result=active.
• vsyncs 30,180 plus resources:1: both generations failed once and recovered on attempt two, proving the failure budget resets.

The submitted native integration test reproduces those checks with one command:

./script/test-gpui-windows-device-loss.ps1

The reviewed implementation passed all twelve scenarios twice. After the final upstream rebase and squash, the exact final head passed all twelve again. Every recovered window completed a successful post-recovery Present().

Native recovery scenario coverage

• Each of the seven candidate-construction stages failed once, then became active on attempt two.
• Two distinct HWNDs recovered in one generation, on attempts one and two.
• DirectComposition, resources:7: active on attempt eight.
• DirectComposition, resources:8: exhausted on attempt eight and suspended.
• DirectComposition, vsyncs 30,180 plus resources:1: generations one and two independently active on attempt two.
• DirectComposition disabled, resources:1: active on attempt two.

Tested Zed base ef075910c99ce2c8fd07da4174e7c3bb71513f35, fork head ec6123c7e9:

• cargo fmt -p gpui_windows --check: pass
• cargo test -p gpui_windows --features test-support -- --skip test_clipboard: 18 passed, 1 clipboard test filtered
• script/clippy.ps1 -p gpui_windows: pass
• cargo build -p gpui --example hello_world --example on_window_close_quit: pass
• script/test-gpui-windows-device-loss.ps1: 12 scenarios passed at the final squashed head, after two earlier full passes of the same implementation; every recovered window completed a successful post-recovery Present(), every owned process was alive at its expected result, and each was terminated through its process handle afterward
• injector-only baseline 16ce921398, resources:8: three consecutive 0xC0000409 exits with the original fatal-handler message
• combined with exact submitted #63489 head cf61a7492e over the final squashed head: clean cherry-pick; present_starvation check, formatting, 18 focused tests, canonical release Clippy, and all 12 native recovery scenarios passed

Coverage boundary: the Rust tests run in normal Windows CI. The native script requires an interactive Windows desktop, so it is submitted and repeatable but not part of the parallel nextest job. It exercises the real vsync, window-message, DirectX, and DirectComposition paths, but injects the loss rather than servicing a physical display driver. No driver was installed, reset, disabled, or otherwise changed. The seam proves the software recovery state machine; it does not claim coverage of every adapter, driver, or Windows build.

Self-Review Checklist:

• I've reviewed my own diff for quality, security, and reliability.
• Unsafe blocks added or changed by this PR have justifying comments.
• This PR does not change UI or icons.
• Tests cover the new and changed behavior.
• Performance impact is acceptable: parsing is debug-only; recovery work runs only after device removal; normal vsync does not sleep or allocate recovery candidates.

---

Release Notes:

• Fixed a Windows crash when a DirectX graphics device is removed or reset.

[zed-industries-bot]

	Messages
📖	This PR includes links to the following GitHub Issues: #52085, #36798, #49956, #61469 If this PR aims to close an issue, please include a Closes #ISSUE line at the top of the PR body.

Generated by 🚫 dangerJS against ec6123c