Debug and Validation Layers (#416)

We've been talking about various ways to perform CPU-side
validation/testing over the outputs of Vello pipeline stages. It's also
generally useful to be able to visualize the contents of some of these
intermediate data structures (such as bounding boxes, the line soup,
etc) and to be able to visually interpret any errors that are surfaced
from the validation tests.

I implemented the beginnings of this in a new `debug_layers` feature.
I'm putting this up as a Draft PR as there are a few unresolved aspects
that I'd like to get feedback on first (more on this below).

## Running Instructions

To try this out run the `with_winit` example with `--features
debug_layers` and use the number keys (1-4) to toggle the individual
layers.

## Summary
This PR introduces the concept of "debug layers" that are rendered
directly to the surface texture over the fine rasterizer output. There
are currently 4 different layers:

- `BOUNDING_BOXES`: Visualizes the edges of path bounding boxes as green
lines
- `LINESOUP_SEGMENTS`: Visualizes LineSoup segments as orange lines
- `LINESOUP_POINTS`: Visualizes the LineSoup endpoints as cyan circles.
- `VALIDATION`: Runs a collection of validation tests on intermediate
Vello GPU data structures. This is currently limited to a watertightness
test on line segments. Following the test run, the layer visualizes the
positions of detected errors as red circles.

These layers can be individually toggled using a new `DebugLayers` field
in `vello::RenderParams`. The following is an example output with all 4
layers enabled:

<img width="906" alt="Screenshot 2023-12-12 at 3 13 51 PM"
src="https://github.com/linebender/vello/assets/6933700/658760c1-ed95-41b8-8444-3a6dfa9ddee7">

Each layer is implemented as an individual render pass. The first 3
layers are simple visualizations of intermediate GPU buffers. The
`VALIDATION` layer is special since it runs CPU-side validation steps
(currently a watertightness test over the LineSoup buffer), which
requires read-back. The general idea is that `VALIDATION` can grow to
encompass additional sanity checks.

### Overview of Changes
- Engine support for render pipeline creation and draw commands. In
particular, the existing `blit` pipeline can now be expressed as a
`Recording`. The debug layer render passes get recorded to this
`Recording`. All render passes share the same render encoder and target
the same surface texture.
- A simple mechanism to extend the lifetime of GPU buffers beyond their
original `Recording` to allow them to be used in a subsequent render
pass. Currently this separation into multiple recordings is necessary
since the visualizations require GPU->CPU read-back. This is partially
encapsulated by the new `CapturedBuffers` data structure.
- The `debug` module and the `DebugLayers` and `DebugRenderer` data
structures. `DebugRenderer` is an encapsulation of the various render
pipelines used to visualize the layers that are requested via
`DebugLayers`. `DebugRenderer` is currently also responsible for
execution the validation tests when the `VALIDATION` layer is enabled.
- The `with_winit` example now has key bindings (the number keys 1-4) to
toggle the individual layers.

## Open Questions
1. It probably makes sense to have a better separation between running
validation tests and visualizing their output. Currently both are
performed by `DebugRenderer::render`.

2. `CapturedBuffers` doesn't handle buffer clean up well. The current
`engine` abstractions require that a buffer be returned to the
underlying engine's pool via a `Recording` command. This creates and
submits a command buffer to simply free a buffer, which is a bit too
heavy-weight. This whole mechanism could use some rethinking.

Currently, these buffers get conditionally freed in various places in
the code and it would be nice to consolidate that logic.

3. The `VALIDATION` layer currently doesn't work with the `--use-cpu`
flag since the buffer download command isn't supported for CPU-only
buffers. Currently, it's the job of `src/render.rs` to know which
buffers need to get downloaded for validation purposes. It currently
simply records a download command. It would be nice for the engine to
make the download command seamless across both CPU and GPU buffers
rather than having the `src/render.rs` code do something different
across the CPU vs GPU modalities.

4. Currently all layers require read-back. The debug layers
(`BOUNDING_BOXES`, `LINESOUP_SEGMENTS`, `LINESOUP_POINTS`) read back the
`BumpAllocators` buffers to obtain instance counts used in their draw
commands. This read-back could be avoided by instead issuing indirect
draws for the debug layers. I think this could be implemented with a
relatively simple scheme: a new compute pipeline stage is introduced
(gated by `#[cfg(feature = "debug_layers")]`, which can inspect any
number of the vello intermediate buffers (such as the `bump` buffer) and
populate an indirect draw buffer. The indirect draw buffer would be laid
out with multiple
[`DrawIndirect`](https://docs.rs/wgpu/latest/wgpu/util/struct.DrawIndirect.html)
entries, each assigned to a different pre-defined instance type (the
`DebugRenderer` only issues instanced draws). `DebugRenderer` would then
issue an indirect draw with the appropriate indirect buffer offset for
each render pipeline.

The read-back would still be necessary for CPU-side validation stages
and their visualization can't really take advantage of the indirect
draw. Then again, the exact ordering of the draw submission and the
read-backs implemented in this PR is likely to change following the
proposal in #366.

---------

Co-authored-by: Daniel McNab <[email protected]>
Co-authored-by: Bruce Mitchener <[email protected]>

examples/headless/src/main.rs

@@ -139,6 +139,7 @@ async fn render(mut scenes: SceneSet, index: usize, args: &Args) -> Result<()> {
         width,
         height,
         antialiasing_method: vello::AaConfig::Area,
+        debug: vello::DebugLayers::none(),
     };
     let mut scene = Scene::new();
     scene.append(&fragment, Some(transform));

examples/simple/src/main.rs

@@ -7,7 +7,7 @@ use std::sync::Arc;
 use vello::kurbo::{Affine, Circle, Ellipse, Line, RoundedRect, Stroke};
 use vello::peniko::Color;
 use vello::util::{RenderContext, RenderSurface};
-use vello::{AaConfig, Renderer, RendererOptions, Scene};
+use vello::{AaConfig, DebugLayers, Renderer, RendererOptions, Scene};
 use winit::application::ApplicationHandler;
 use winit::dpi::LogicalSize;
 use winit::event::*;
@@ -151,6 +151,7 @@ impl<'s> ApplicationHandler for SimpleVelloApp<'s> {
                             width,
                             height,
                             antialiasing_method: AaConfig::Msaa16,
+                            debug: DebugLayers::none(),
                         },
                     )
                     .expect("failed to render to surface");

examples/with_winit/Cargo.toml

@@ -25,8 +25,9 @@ workspace = true
 name = "with_winit_bin"
 path = "src/main.rs"
 
+
 [dependencies]
-vello = { workspace = true, features = ["buffer_labels"] }
+vello = { workspace = true, features = ["buffer_labels", "debug_layers"] }
 scenes = { workspace = true }
 
 anyhow = { workspace = true }

examples/with_winit/src/lib.rs

@@ -162,6 +162,8 @@ struct VelloApp<'s> {
 
     prev_scene_ix: i32,
     modifiers: ModifiersState,
+
+    debug: vello::DebugLayers,
 }
 
 impl<'s> ApplicationHandler<UserEvent> for VelloApp<'s> {
@@ -329,6 +331,27 @@ impl<'s> ApplicationHandler<UserEvent> for VelloApp<'s> {
                                         },
                                     );
                                 }
+                                debug_layer @ ("1" | "2" | "3" | "4") => {
+                                    match debug_layer {
+                                        "1" => {
+                                            self.debug.toggle(vello::DebugLayers::BOUNDING_BOXES);
+                                        }
+                                        "2" => {
+                                            self.debug
+                                                .toggle(vello::DebugLayers::LINESOUP_SEGMENTS);
+                                        }
+                                        "3" => {
+                                            self.debug.toggle(vello::DebugLayers::LINESOUP_POINTS);
+                                        }
+                                        "4" => {
+                                            self.debug.toggle(vello::DebugLayers::VALIDATION);
+                                        }
+                                        _ => unreachable!(),
+                                    }
+                                    if !self.debug.is_empty() && !self.async_pipeline {
+                                        log::warn!("Debug Layers won't work without using `--async-pipeline`. Requested {:?}", self.debug);
+                                    }
+                                }
                                 _ => {}
                             }
                         }
@@ -464,6 +487,7 @@ impl<'s> ApplicationHandler<UserEvent> for VelloApp<'s> {
                     width,
                     height,
                     antialiasing_method,
+                    debug: self.debug,
                 };
                 self.scene.reset();
                 let mut transform = self.transform;
@@ -674,6 +698,8 @@ fn run(
         Some(render_state)
     };
 
+    let debug = vello::DebugLayers::none();
+
     let mut app = VelloApp {
         context: render_cx,
         renderers,
@@ -718,6 +744,7 @@ fn run(
         complexity: 0,
         prev_scene_ix: 0,
         modifiers: ModifiersState::default(),
+        debug,
     };
 
     event_loop.run_app(&mut app).expect("run to completion");
@@ -786,6 +813,7 @@ pub fn main() -> anyhow::Result<()> {
     #[cfg(not(target_arch = "wasm32"))]
     env_logger::builder()
         .format_timestamp(Some(env_logger::TimestampPrecision::Millis))
+        .filter_level(log::LevelFilter::Warn)
         .init();
     let args = parse_arguments();
     let scenes = args.args.select_scene_set()?;

vello/Cargo.toml

@@ -18,6 +18,7 @@ default = ["wgpu"]
 bump_estimate = ["vello_encoding/bump_estimate"]
 hot_reload = ["vello_shaders/compile"]
 buffer_labels = []
+debug_layers = []
 wgpu = ["dep:wgpu"]
 wgpu-profiler = ["dep:wgpu-profiler"]
 

vello/src/debug.rs

@@ -0,0 +1,119 @@
+// Copyright 2023 the Vello Authors
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#[cfg(all(feature = "debug_layers", feature = "wgpu"))]
+mod renderer;
+#[cfg(all(feature = "debug_layers", feature = "wgpu"))]
+mod validate;
+
+use std::fmt::Debug;
+
+#[cfg(all(feature = "debug_layers", feature = "wgpu"))]
+pub(crate) use renderer::*;
+
+/// Bitflags for enabled debug operations.
+///
+/// Currently, all layers additionally require the `debug_layers` feature.
+#[derive(Copy, Clone)]
+pub struct DebugLayers(u8);
+
+impl Debug for DebugLayers {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        let mut tuple = f.debug_tuple("DebugLayers");
+        if self.contains(Self::BOUNDING_BOXES) {
+            tuple.field(&"BOUNDING_BOXES");
+        }
+        if self.contains(Self::LINESOUP_SEGMENTS) {
+            tuple.field(&"LINESOUP_SEGMENTS");
+        }
+        if self.contains(Self::LINESOUP_POINTS) {
+            tuple.field(&"LINESOUP_POINTS");
+        }
+        if self.contains(Self::VALIDATION) {
+            tuple.field(&"VALIDATION");
+        }
+
+        tuple.finish()
+    }
+}
+
+// TODO: Currently all layers require read-back of the BumpAllocators buffer. This isn't strictly
+// necessary for layers other than `VALIDATION`. The debug visualizations use the bump buffer only
+// to obtain various instance counts for draws and these could instead get written out to an
+// indirect draw buffer. OTOH `VALIDATION` should always require readback since we want to be able
+// to run the same CPU-side tests for both CPU and GPU shaders.
+impl DebugLayers {
+    /// Visualize the bounding box of every path.
+    /// Requires the `debug_layers` feature.
+    pub const BOUNDING_BOXES: DebugLayers = DebugLayers(1 << 0);
+
+    /// Visualize the post-flattening line segments using line primitives.
+    /// Requires the `debug_layers` feature.
+    pub const LINESOUP_SEGMENTS: DebugLayers = DebugLayers(1 << 1);
+
+    /// Visualize the post-flattening line endpoints.
+    /// Requires the `debug_layers` feature.
+    pub const LINESOUP_POINTS: DebugLayers = DebugLayers(1 << 2);
+
+    /// Enable validation of internal buffer contents and visualize errors. Validation tests are
+    /// run on the CPU and require buffer contents to be read-back.
+    ///
+    /// Supported validation tests:
+    ///
+    ///    - Watertightness: validate that every line segment within a path is connected without
+    ///      any gaps. Line endpoints that don't precisely overlap another endpoint get visualized
+    ///      as red circles and logged to stderr.
+    ///
+    /// Requires the `debug_layers` feature.
+    pub const VALIDATION: DebugLayers = DebugLayers(1 << 3);
+
+    /// Construct a `DebugLayers` from the raw bits.
+    pub const fn from_bits(bits: u8) -> Self {
+        Self(bits)
+    }
+
+    /// Get the raw representation of this value.
+    pub const fn bits(self) -> u8 {
+        self.0
+    }
+
+    /// A `DebugLayers` with no layers enabled.
+    pub const fn none() -> Self {
+        Self(0)
+    }
+
+    /// A `DebugLayers` with all layers enabled.
+    pub const fn all() -> Self {
+        // Custom BitOr is not const, so need to manipulate the inner value here
+        Self(
+            Self::BOUNDING_BOXES.0
+                | Self::LINESOUP_SEGMENTS.0
+                | Self::LINESOUP_POINTS.0
+                | Self::VALIDATION.0,
+        )
+    }
+
+    /// True if this `DebugLayers` has no layers enabled.
+    pub const fn is_empty(self) -> bool {
+        self.0 == 0
+    }
+
+    /// Determine whether `self` is a superset of `mask`.
+    pub const fn contains(self, mask: DebugLayers) -> bool {
+        self.0 & mask.0 == mask.0
+    }
+
+    /// Toggle the value of the layers specified in mask.
+    pub fn toggle(&mut self, mask: DebugLayers) {
+        self.0 ^= mask.0;
+    }
+}
+
+/// Returns the union of the two input `DebugLayers`.
+impl std::ops::BitOr for DebugLayers {
+    type Output = Self;
+
+    fn bitor(self, rhs: Self) -> Self {
+        Self(self.0 | rhs.0)
+    }
+}