Skip to content

Bench app context phase 2 - #58202

Merged
Anthony-Eid merged 30 commits into
mainfrom
bench-app-context-phase-2
Jun 10, 2026
Merged

Bench app context phase 2#58202
Anthony-Eid merged 30 commits into
mainfrom
bench-app-context-phase-2

Conversation

@Anthony-Eid

@Anthony-Eid Anthony-Eid commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

This PR builds out GPUI's benchmark harness so render benchmarks measure realistic frame costs using GPUI-owned, runtime-gated instrumentation. It adds measurements for frame draw time, dirty-to-draw latency, invalidation coalescing, and frame-budget overruns, and runs benchmark workloads with production-like concurrency.

How it works

Frame timings flow through the GPUI profiler. Window::draw emits a FrameTiming { window_id, dirty_at, invalidations, draw_start, draw_end } event into a global ring buffer in gpui::profiler, mirroring the existing task-timing channel. Collection is runtime-gated by profiler::set_frame_trace_enabled (one relaxed atomic load when disabled — no Instant::now calls in production). BenchReport is a pure listener: it drains events through a cursor-based FrameTimingCollector and builds histograms in the bench layer. Window carries no bench-only cfg fields, and the same event channel can later feed the miniprofiler UI or an in-app frame-time HUD.

BenchDispatcher is a multithreaded PlatformDispatcher for benchmarks: background tasks run on a worker pool (same priority queue as LinuxDispatcher, with task-profiler hooks), timers fire in real time on a dedicated thread, and foreground tasks queue until the bench thread drains them with a blocking run_until_idle(). Unlike TestDispatcher, work executes in parallel in real time, so wall-clock measurements reflect production concurrency. In-flight accounting is panic-safe via drop guards.

gpui::bench_platform() returns a per-process TestPlatform backed by the BenchDispatcher, cached in a thread-local so worker threads persist across Criterion calibration passes. This replaces the earlier approach of constructing a real platform per invocation, which had process-global singleton issues, never ran foreground tasks (no run loop pumped the main queue), and couldn't open windows on headless CI.

Text shaping uses NoopTextSystem: deterministic across machines/font installations and CI-portable. Measured cost of this trade: ~10% of editor draw time vs MacTextSystem (Noop still emits one glyph per character at fixed advances, so downstream layout/paint structure is preserved).

GPU coverage (macOS only for now). PlatformHeadlessRenderer gained render_scene, which encodes and submits the scene to Metal against a cached offscreen target without blocking on completion or reading pixels back — matching production present() CPU cost (render_scene_to_image would overstate it: it waits for the GPU and copies pixels back). TestWindow::draw forwards scenes to the renderer, the real MetalAtlas means glyph/SVG rasterization happens during paint, and bench_renderer presents after each measured update. Platforms without a headless renderer degrade to discarding the scene.

Example

#[gpui::bench]
fn editor_render(cx: &mut BenchAppContext) {
    init_context(cx);

    let buffer = cx.update(|cx| { /* build a MultiBuffer */ });

    let mut window = cx.add_empty_window();
    let editor = window.update(|window, cx| {
        let editor = window.replace_root(cx, |window, cx| {
            let mut editor = Editor::new(EditorMode::full(), buffer, None, window, cx);
            editor.set_style(editor::EditorStyle::default(), window, cx);
            editor
        });
        window.focus(&editor.focus_handle(cx), cx);
        editor
    });

    let mut move_down = true;
    cx.bench_renderer(editor, move |editor, window, cx| {
        if move_down {
            editor.move_down(&MoveDown, window, cx);
        } else {
            editor.move_up(&MoveUp, window, cx);
        }
        move_down = !move_down;
    });
}

Example output (release, M-series)

editor_render           time:   [329.75 µs 330.17 µs 330.69 µs]

GPUI bench report (all observed iterations): editor_render
  note: includes Criterion warmup/calibration
  window dirty-to-draw:
    samples: 31533
    mean: 0.321ms
    p50: 0.322ms
    p90: 0.336ms
    p95: 0.342ms
    p99: 0.360ms
    max: 0.504ms
    frame budget overruns total: 0
    frame budget overruns max: 0
  window draw:
    samples: 31533
    mean: 0.295ms
    p50: 0.295ms
    p90: 0.307ms
    p95: 0.313ms
    p99: 0.330ms
    max: 0.455ms
    frame budget overruns total: 0
    frame budget overruns max: 0
  invalidations per frame: mean 5.00, max 5

(invalidations per frame: mean 5.00 is real signal: each move_down notifies the window five times before the draw.)

Known limitations

  • Draw-per-flush: the harness draws synchronously when effects flush rather than coalescing invalidations to a vsync tick, so dirty-to-draw excludes queueing delay, and frame budget overruns is a draw-time budget proxy rather than actual missed presents. A frame-paced mode is natural follow-up work.
  • GPU submission is measured on macOS only; other platforms have no headless renderer yet.
  • The GPUI report includes Criterion warmup/calibration samples (noted in the output); Criterion's time is the regression-gating number.
  • run_until_idle waits for queued, running, and already-due work, but not for timers that haven't reached their due time — the dispatcher runs in real time and can't skip ahead like TestDispatcher's virtual clock.

Future work

  • A vsync-like frame-pacing mode (suppress draw-on-flush; tick-driven draw + present) so dirty-to-draw captures queueing delay
  • Record present duration in FrameTiming so the report can split draw vs present
  • Benches that scroll through novel content (cold layout caches) and an agent-panel render bench
  • Headless renderers for Windows/Linux
  • Move benches into a dedicated crate

Self-Review Checklist:

  • I've reviewed my own diff for quality, security, and reliability
  • Unsafe blocks (if any) have justifying comments
  • The content is consistent with the UI/UX checklist
  • Tests cover the new/changed behavior
  • Performance impact has been considered and is acceptable

Release Notes:

  • N/A

@cla-bot cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Jun 1, 2026
@zed-community-bot zed-community-bot Bot added the staff Pull requests authored by a current member of Zed staff label Jun 1, 2026
Base automatically changed from bench-app-context to main June 1, 2026 22:31
@Anthony-Eid

Copy link
Copy Markdown
Contributor Author

@Veykril @osiewicz I would appreciate your guys feedback on the direction I'm going with this PR because you guys are some of the performance wizards on the team.

I'm aiming to make a benchmark suite for Zed so we can have something to prevent performance regressions and make some performance issues a lot easier to reproduce

Comment thread crates/editor/Cargo.toml Outdated
@Anthony-Eid
Anthony-Eid enabled auto-merge June 10, 2026 09:21
@Anthony-Eid
Anthony-Eid added this pull request to the merge queue Jun 10, 2026
Merged via the queue into main with commit 297c4a4 Jun 10, 2026
31 checks passed
@Anthony-Eid
Anthony-Eid deleted the bench-app-context-phase-2 branch June 10, 2026 09:30
This was referenced Jun 18, 2026
jonx pushed a commit to jonx/zed-aros that referenced this pull request Jul 17, 2026
This PR builds out GPUI's benchmark harness so render benchmarks measure
realistic frame costs using GPUI-owned, runtime-gated instrumentation.
It adds measurements for frame draw time, dirty-to-draw latency,
invalidation coalescing, and frame-budget overruns, and runs benchmark
workloads with production-like concurrency.

## How it works

**Frame timings flow through the GPUI profiler.** `Window::draw` emits a
`FrameTiming { window_id, dirty_at, invalidations, draw_start, draw_end
}` event into a global ring buffer in `gpui::profiler`, mirroring the
existing task-timing channel. Collection is runtime-gated by
`profiler::set_frame_trace_enabled` (one relaxed atomic load when
disabled — no `Instant::now` calls in production). `BenchReport` is a
pure listener: it drains events through a cursor-based
`FrameTimingCollector` and builds histograms in the bench layer.
`Window` carries no bench-only cfg fields, and the same event channel
can later feed the miniprofiler UI or an in-app frame-time HUD.

**`BenchDispatcher`** is a multithreaded `PlatformDispatcher` for
benchmarks: background tasks run on a worker pool (same priority queue
as `LinuxDispatcher`, with task-profiler hooks), timers fire in real
time on a dedicated thread, and foreground tasks queue until the bench
thread drains them with a blocking `run_until_idle()`. Unlike
`TestDispatcher`, work executes in parallel in real time, so wall-clock
measurements reflect production concurrency. In-flight accounting is
panic-safe via drop guards.

**`gpui::bench_platform()`** returns a per-process `TestPlatform` backed
by the `BenchDispatcher`, cached in a thread-local so worker threads
persist across Criterion calibration passes. This replaces the earlier
approach of constructing a real platform per invocation, which had
process-global singleton issues, never ran foreground tasks (no run loop
pumped the main queue), and couldn't open windows on headless CI.

**Text shaping** uses `NoopTextSystem`: deterministic across
machines/font installations and CI-portable. Measured cost of this
trade: ~10% of editor draw time vs `MacTextSystem` (Noop still emits one
glyph per character at fixed advances, so downstream layout/paint
structure is preserved).

**GPU coverage (macOS only for now).** `PlatformHeadlessRenderer` gained
`render_scene`, which encodes and submits the scene to Metal against a
cached offscreen target without blocking on completion or reading pixels
back — matching production `present()` CPU cost (`render_scene_to_image`
would overstate it: it waits for the GPU and copies pixels back).
`TestWindow::draw` forwards scenes to the renderer, the real
`MetalAtlas` means glyph/SVG rasterization happens during paint, and
`bench_renderer` presents after each measured update. Platforms without
a headless renderer degrade to discarding the scene.

## Example

```rust
#[gpui::bench]
fn editor_render(cx: &mut BenchAppContext) {
    init_context(cx);

    let buffer = cx.update(|cx| { /* build a MultiBuffer */ });

    let mut window = cx.add_empty_window();
    let editor = window.update(|window, cx| {
        let editor = window.replace_root(cx, |window, cx| {
            let mut editor = Editor::new(EditorMode::full(), buffer, None, window, cx);
            editor.set_style(editor::EditorStyle::default(), window, cx);
            editor
        });
        window.focus(&editor.focus_handle(cx), cx);
        editor
    });

    let mut move_down = true;
    cx.bench_renderer(editor, move |editor, window, cx| {
        if move_down {
            editor.move_down(&MoveDown, window, cx);
        } else {
            editor.move_up(&MoveUp, window, cx);
        }
        move_down = !move_down;
    });
}
```

## Example output (release, M-series)

```
editor_render           time:   [329.75 µs 330.17 µs 330.69 µs]

GPUI bench report (all observed iterations): editor_render
  note: includes Criterion warmup/calibration
  window dirty-to-draw:
    samples: 31533
    mean: 0.321ms
    p50: 0.322ms
    p90: 0.336ms
    p95: 0.342ms
    p99: 0.360ms
    max: 0.504ms
    frame budget overruns total: 0
    frame budget overruns max: 0
  window draw:
    samples: 31533
    mean: 0.295ms
    p50: 0.295ms
    p90: 0.307ms
    p95: 0.313ms
    p99: 0.330ms
    max: 0.455ms
    frame budget overruns total: 0
    frame budget overruns max: 0
  invalidations per frame: mean 5.00, max 5
```

(`invalidations per frame: mean 5.00` is real signal: each `move_down`
notifies the window five times before the draw.)

## Known limitations

- **Draw-per-flush**: the harness draws synchronously when effects flush
rather than coalescing invalidations to a vsync tick, so `dirty-to-draw`
excludes queueing delay, and `frame budget overruns` is a draw-time
budget proxy rather than actual missed presents. A frame-paced mode is
natural follow-up work.
- **GPU submission is measured on macOS only**; other platforms have no
headless renderer yet.
- The GPUI report includes Criterion warmup/calibration samples (noted
in the output); Criterion's `time` is the regression-gating number.
- `run_until_idle` waits for queued, running, and already-due work, but
not for timers that haven't reached their due time — the dispatcher runs
in real time and can't skip ahead like `TestDispatcher`'s virtual clock.

## Future work

- A vsync-like frame-pacing mode (suppress draw-on-flush; tick-driven
draw + present) so dirty-to-draw captures queueing delay
- Record present duration in `FrameTiming` so the report can split draw
vs present
- Benches that scroll through novel content (cold layout caches) and an
agent-panel render bench
- Headless renderers for Windows/Linux
- Move benches into a dedicated crate

Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the UI/UX checklist
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Release Notes:

- N/A
jolutz pushed a commit to jolutz/zed that referenced this pull request Aug 8, 2026
This PR builds out GPUI's benchmark harness so render benchmarks measure
realistic frame costs using GPUI-owned, runtime-gated instrumentation.
It adds measurements for frame draw time, dirty-to-draw latency,
invalidation coalescing, and frame-budget overruns, and runs benchmark
workloads with production-like concurrency.

## How it works

**Frame timings flow through the GPUI profiler.** `Window::draw` emits a
`FrameTiming { window_id, dirty_at, invalidations, draw_start, draw_end
}` event into a global ring buffer in `gpui::profiler`, mirroring the
existing task-timing channel. Collection is runtime-gated by
`profiler::set_frame_trace_enabled` (one relaxed atomic load when
disabled — no `Instant::now` calls in production). `BenchReport` is a
pure listener: it drains events through a cursor-based
`FrameTimingCollector` and builds histograms in the bench layer.
`Window` carries no bench-only cfg fields, and the same event channel
can later feed the miniprofiler UI or an in-app frame-time HUD.

**`BenchDispatcher`** is a multithreaded `PlatformDispatcher` for
benchmarks: background tasks run on a worker pool (same priority queue
as `LinuxDispatcher`, with task-profiler hooks), timers fire in real
time on a dedicated thread, and foreground tasks queue until the bench
thread drains them with a blocking `run_until_idle()`. Unlike
`TestDispatcher`, work executes in parallel in real time, so wall-clock
measurements reflect production concurrency. In-flight accounting is
panic-safe via drop guards.

**`gpui::bench_platform()`** returns a per-process `TestPlatform` backed
by the `BenchDispatcher`, cached in a thread-local so worker threads
persist across Criterion calibration passes. This replaces the earlier
approach of constructing a real platform per invocation, which had
process-global singleton issues, never ran foreground tasks (no run loop
pumped the main queue), and couldn't open windows on headless CI.

**Text shaping** uses `NoopTextSystem`: deterministic across
machines/font installations and CI-portable. Measured cost of this
trade: ~10% of editor draw time vs `MacTextSystem` (Noop still emits one
glyph per character at fixed advances, so downstream layout/paint
structure is preserved).

**GPU coverage (macOS only for now).** `PlatformHeadlessRenderer` gained
`render_scene`, which encodes and submits the scene to Metal against a
cached offscreen target without blocking on completion or reading pixels
back — matching production `present()` CPU cost (`render_scene_to_image`
would overstate it: it waits for the GPU and copies pixels back).
`TestWindow::draw` forwards scenes to the renderer, the real
`MetalAtlas` means glyph/SVG rasterization happens during paint, and
`bench_renderer` presents after each measured update. Platforms without
a headless renderer degrade to discarding the scene.

## Example

```rust
#[gpui::bench]
fn editor_render(cx: &mut BenchAppContext) {
    init_context(cx);

    let buffer = cx.update(|cx| { /* build a MultiBuffer */ });

    let mut window = cx.add_empty_window();
    let editor = window.update(|window, cx| {
        let editor = window.replace_root(cx, |window, cx| {
            let mut editor = Editor::new(EditorMode::full(), buffer, None, window, cx);
            editor.set_style(editor::EditorStyle::default(), window, cx);
            editor
        });
        window.focus(&editor.focus_handle(cx), cx);
        editor
    });

    let mut move_down = true;
    cx.bench_renderer(editor, move |editor, window, cx| {
        if move_down {
            editor.move_down(&MoveDown, window, cx);
        } else {
            editor.move_up(&MoveUp, window, cx);
        }
        move_down = !move_down;
    });
}
```

## Example output (release, M-series)

```
editor_render           time:   [329.75 µs 330.17 µs 330.69 µs]

GPUI bench report (all observed iterations): editor_render
  note: includes Criterion warmup/calibration
  window dirty-to-draw:
    samples: 31533
    mean: 0.321ms
    p50: 0.322ms
    p90: 0.336ms
    p95: 0.342ms
    p99: 0.360ms
    max: 0.504ms
    frame budget overruns total: 0
    frame budget overruns max: 0
  window draw:
    samples: 31533
    mean: 0.295ms
    p50: 0.295ms
    p90: 0.307ms
    p95: 0.313ms
    p99: 0.330ms
    max: 0.455ms
    frame budget overruns total: 0
    frame budget overruns max: 0
  invalidations per frame: mean 5.00, max 5
```

(`invalidations per frame: mean 5.00` is real signal: each `move_down`
notifies the window five times before the draw.)

## Known limitations

- **Draw-per-flush**: the harness draws synchronously when effects flush
rather than coalescing invalidations to a vsync tick, so `dirty-to-draw`
excludes queueing delay, and `frame budget overruns` is a draw-time
budget proxy rather than actual missed presents. A frame-paced mode is
natural follow-up work.
- **GPU submission is measured on macOS only**; other platforms have no
headless renderer yet.
- The GPUI report includes Criterion warmup/calibration samples (noted
in the output); Criterion's `time` is the regression-gating number.
- `run_until_idle` waits for queued, running, and already-due work, but
not for timers that haven't reached their due time — the dispatcher runs
in real time and can't skip ahead like `TestDispatcher`'s virtual clock.

## Future work

- A vsync-like frame-pacing mode (suppress draw-on-flush; tick-driven
draw + present) so dirty-to-draw captures queueing delay
- Record present duration in `FrameTiming` so the report can split draw
vs present
- Benches that scroll through novel content (cold layout caches) and an
agent-panel render bench
- Headless renderers for Windows/Linux
- Move benches into a dedicated crate

Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the UI/UX checklist
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Release Notes:

- N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement staff Pull requests authored by a current member of Zed staff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants