Proposal: Bring back vector-based tiled brushes

[verelpode]

Proposal: Bring back vector-based tiled brushes

Summary

Could you please consider bringing back the ability and documentation for how to make vector-based tiled brushes such as checkerboard and gridline brushes? This already existed previously but disappeared in UWP. Prior to UWP, TileBrush had a derived class called DrawingBrush and you can see examples of gridline brushes and checkerboard brushes in the second image in "Painting with Images, Drawings, and Visuals", under the heading "Paint an Area with a Drawing". But how can this be done in UWP? Please note these were vector-based brushes not bitmaps. How can we make such vector-based tiled brushes in UWP?

Rationale

Currently UWP has TileBrush with subclass ImageBrush but this only supports raster/bitmap tiled brushes, but in today's GUI world, everyone expects display resolution independence and support for very high resolution displays etc, thus the need for vector tiled brushes.

A tiled brush that displays gridlines is an example of something that looks terrible when implemented using ImageBrush. A bitmap image of gridlines does not scale well. In addition, it is inefficient to implement it as an ImageBrush. As a vector tile brush, it looks better and renders faster.

Important Notes

The following example XAML demonstrates how to make a checkerboard brush in WPF, but this stopped working in UWP because UWP doesn't have DrawingBrush. How can we do the equivalent in UWP?

<DrawingBrush Viewport="0,0,20,20" ViewportUnits="Absolute" Stretch="None" TileMode="Tile">
   <DrawingBrush.Drawing>
     <DrawingGroup>
       <GeometryDrawing Brush="#18FFFFFF">
         <GeometryDrawing.Geometry>
           <GeometryGroup>
             <RectangleGeometry Rect="0,0,10,10"/>
             <RectangleGeometry Rect="10,10,10,10"/>
           </GeometryGroup>
         </GeometryDrawing.Geometry>
       </GeometryDrawing>
       <GeometryDrawing Brush="#34FFFFFF">
         <GeometryDrawing.Geometry>
           <GeometryGroup>
             <RectangleGeometry Rect="10,0,10,10"/>
             <RectangleGeometry Rect="0,10,10,10"/>
           </GeometryGroup>
         </GeometryDrawing.Geometry>
       </GeometryDrawing>
     </DrawingGroup>
   </DrawingBrush.Drawing>
 </DrawingBrush> 

Open Questions

• Should this feature use the same API as it did in WPF-Desktop or should it be redesigned?
• Should this feature be made in connection with Compositor?

[jesbis]

@verelpode would Win2D work for your scenarios?

You can see an example of how it's used to create a repeated checkerboard in the "drawimage enumerations" example in the Win2D Gallery sample app by tiling the result of CreateFromColors() using CanvasEdgeBehavior.Wrap.

For more complex usages there's also TileEffect.

[verelpode]

@jesbis Thanks for the idea. Correct me if I'm wrong, but that Win2D object cannot be converted to Windows.UI.Xaml.Media.Brush, therefore it would be difficult to integrate. I was expecting vector tiled brushes to be usable anywhere that Windows.UI.Xaml.Media.Brush is usable (like how it was in WPF-Desktop), including assigning a vector tiled brush to these properties:

• Windows.UI.Xaml.Controls.Control.Background
• Windows.UI.Xaml.Controls.Panel.Background
• Windows.UI.Xaml.Shapes.Shape.Fill
• etc.

Win2D seems like a case of switching to a completely different graphics engine whereas I was thinking that vector tiled brushes would be a part of Windows.UI.Xaml, just like they were in WPF-Desktop.

[jesbis]

Thanks for clarifying! Xaml, Composition and Win2D are mostly built on the same graphics engine and just provide different levels of abstraction, but I can see the value of having it supported as part of the Xaml API.

Right now you have a couple options for Brushes:

Win2D has a CanvasImageSource which can be used as the source for a Xaml brush.

Another option might be to create your own Xaml brush that subclasses XamlCompositionBrushBase and uses the Windows.UI.Composition APIs to tile. It could use the same approach as in Win2D (e.g. CanvasEdgeBehavior.Wrap).

[verelpode]

Aha. Although I've not yet used the new Composition API, it looks like the idea of subclassing XamlCompositionBrushBase is the best solution, and it is a "modern" solution in the sense that it is based on the latest API (the Composition API). This solution seems to be a modern UWP equivalent to the System.Windows.Media.VisualBrush in WPF-Desktop, a subclass of System.Windows.Media.TileBrush.

If I understand correctly, it would be implemented like the following ?

public class VisualTileBrush : Windows.UI.Xaml.Media.XamlCompositionBrushBase
{

    public Windows.UI.Composition.Visual Visual
    {
        get { ... }
        set {
            ...
            base.CompositionBrush = new CompositionSurfaceBrush(...); 
            // CompositionSurfaceBrush or a different subclass of CompositionBrush?
            // Somehow link up this.Visual with base.CompositionBrush.
            // Somehow enable the tiling option.
        }
    }

}

If I understand correctly, the checkerboard or gridline pattern or whatever pattern would be defined in the Visual object that is assigned to VisualTileBrush.Visual. Then we somehow create an instance of a CompositionBrush that uses that Visual, and then assign the CompositionBrush instance to the inherited XamlCompositionBrushBase.CompositionBrush property. Is that correct?

[jevansaks]

I would recommend using a CompositionEffectBrush. You can create an effect graph using Win2D. One of the effects in there is BorderEffect which lets you tile its source.

You might be able to glean some info from the AcrylicBrush that we have implemented in this repo (https://github.com/Microsoft/microsoft-ui-xaml/blob/master/dev/Materials/Acrylic/AcrylicBrush.cpp#L845). In that scenario we tile an image wrapped in CompositionSurfaceBrush, which can be given to the BorderEffect.

If you wanted it to be an SVG-based tile then I think you could use Win2D to render the base tile and feed that through the effect brush.

[jesbis]

You shouldn't need to use anything to do with Visuals for a brush.

For reference here's a basic C# example of a tiling brush based on XamlCompositionBrushBase:
https://gist.github.com/jesbis/774eb0294132a7137957cb9b39b44717

Disclaimer: just an example, and not 100% tested or guaranteed.
Also note that you need to add the Win2D.uwp NuGet package to access Microsoft.Graphics.Canvas.Effects.

[verelpode]

hmm, if it doesn't use a Visual object to represent the checkerboard or other vector-based pattern to tile, then what other object should be used to represent the pattern? The link to ImageTileBrush.cs is helpful except one thing needs to be replaced: Instead of using a raster image (the ImageTileBrush.ImageSourceUri property), I guess that BorderEffect.Source should be somehow linked to a Visual or other vector-based object that represents the pattern or retained-mode-vector-image to be tiled.

[jesbis]

All content is eventually rasterized, so it's just a matter of what stage you do it at. Using pre-generated image files can often yield better performance if it works for your scenario.

If you do want to generate the content at runtime then instead of this line in the gist above:

var surface = LoadedImageSurface.StartLoadFromUri(ImageSourceUri);

you could draw your own content, e.g.:

var graphicsDevice = CanvasComposition.CreateCompositionGraphicsDevice(Window.Current.Compositor, CanvasDevice.GetSharedDevice());
var surface = graphicsDevice.CreateDrawingSurface(new Size(10,10), DirectXPixelFormat.B8G8R8A8UIntNormalized, DirectXAlphaMode.Premultiplied);
using (var session = CanvasComposition.CreateDrawingSession(surface))
{
    session.FillRectangle(new Rect(0, 0, 5, 5), Colors.Blue);
    session.FillRectangle(new Rect(5, 5, 10, 10), Colors.Yellow);
}

(note this is just an example for testing: for a real implementation you'd likely want to additionally cache the CompositionGraphiceDevice, handle the RenderingDeviceReplaced event appropriately, and dispose the device in OnDisconnected).

[verelpode]

Correct me if I'm wrong: If the content is loaded from a raster image file OR drawn at runtime using CreateDrawingSession, either way, this pre-rasterized solution would be incompatible with ScrollViewer.ZoomFactor and Viewbox etc? Actually the tiled brush would still appear on-screen but it would suffer raster scaling artifacts/distortions, unlike WPF's DrawingBrush and VisualBrush which are fully scalable / resolution-independent.

Another disadvantage is that if this tiled brush implementation is publicly added to Windows.UI.Xaml.Media, then it's actually unusable by a typical xaml app unless the Win2D nuget package is added to the app. It'd be like saying: "We created a new class Windows.UI.Xaml.Media.SomeNewTiledBrush for everyone to use but actually you cannot use it unless you add the Win2D nuget package to your app."
(The tiled brush content is drawn by the app by using Win2D outside of Windows.UI.Xaml.Media.SomeNewTiledBrush, thus the requirement to add the nuget package to the app.)

UWP/Windows.UI.Xaml is supposed to be better than its predecessor WPF, but for this pre-rasterized tiled brush implementation, it seems like we're falling back to an inferior solution in comparison to WPF. Loss of a preexisting capability isn't a happy feeling.

How difficult would it be to create a UWP/Xaml version of System.Windows.Media.VisualBrush ? Note VisualBrush also supports tiling. Obviously the UWP Windows.UI.Xaml.Media.VisualBrush would use an instance of Windows.UI.Composition.Visual instead of System.Windows.Media.Visual.

Although WPF has VisualBrush AND DrawingBrush, I believe DrawingBrush becomes unnecessary if Windows.UI.Xaml.Media.VisualBrush is created. No need to try to create UWP versions of both of them, but currently we have neither of them.

[jesbis]

If you need to support dynamic scaling at runtime then you do need something Visual-based after all 😊

In the latest insider SDKs you can use Windows.UI.Composition.CompositionVisualSurface, which is similar to the WPF VisualBrush but a) faster and b) more flexible. It will definitely be more expensive than pre-rasterized content though.

Here's an example that uses high-performance composition shapes and supports scaling (with ViewBox, ScrollViewer, etc.):
https://gist.github.com/jesbis/e95fd936b581639969ab6f57fab36c4a

You could also refactor it to accept arbitrary sets of shapes or Visuals if needed.

[verelpode]

Looks good, thank you! That example could also be used as the basis for creating Windows.UI.Xaml.Media.VisualBrush. Although WPF VisualBrush inherits System.Windows.Media.TileBrush, I don't think it's a problem in practice if UWP VisualBrush inherits XamlCompositionBrushBase instead of the existing Windows.UI.Xaml.Media.TileBrush (or at least it wouldn't bother me personally).

Alternatively, if desired, the relevant parts of the code inside XamlCompositionBrushBase could be incorporated into Windows.UI.Xaml.Media.VisualBrush instead of inheriting XamlCompositionBrushBase, in order to prevent VisualBrush's internal implementation details spilling into the public API contract. i.e. if VisualBrush inherits XamlCompositionBrushBase, then this inheritance becomes part of the public API contract despite the fact that it's actually an internal implementation detail.

A related issue that could be considered is the adding of the tiled brush capability to the Composition API. As I understand it, currently the Composition API -- if used alone -- is incapable of making a tiled CompositionBrush, hence the need to get the capability via BorderEffect in the Win2D nuget package. It could be worthwhile to consider making the Composition API directly support a tiled CompositionBrush in normal UWP/Xaml apps that don't have the Win2D package added.

Therefore would it make sense to create a class named Windows.UI.Composition.CompositionSurfaceTileBrush that inherits CompositionBrush? CompositionSurfaceTileBrush would be similar to the existing CompositionSurfaceBrush except with the difference that CompositionSurfaceBrush has properties to scale and transform the ICompositionSurface whereas CompositionSurfaceTileBrush would just tile the Surface.

Alternatively, a "TileMode" property could be added to CompositionSurfaceBrush, similar to System.Windows.Media.TileBrush.TileMode, with the default value being TileMode.None.

I think that a graphics API is generally expected to have a tiling capability, so I think it would make good sense if the Composition API supports a CompositionBrush that tiles a Composition Surface. When this capability is added to the Composition API, then of course it could also be used to create Xaml tiled brush that is internally implemented as a wrapper of the same capability within the Composition API.

[michael-hawker]

Little late to the thread, we do have a Tile Control in the Toolkit, but it's a bit limited and a wrapper.

I agree this should just be a brush based thing which can be applied as a background anywhere.

[verelpode]

In issue #719, @bschoepke requested that WPF's VisualBrush be supported in UWP/WinUI:

VisualBrush is a good one. You can achieve mostly the same outcome using Windows.UI.Composition.RedirectVisual but it's less convenient than a brush, and there's no way to "Freeze" it.

The RedirectVisual workaround is interesting.

In a previous message here, @jesbis mentioned Windows.UI.Composition.CompositionVisualSurface and a link to an example, but I misunderstood half of the message because I thought CompositionVisualSurface and the example were the same topic, but actually the example demonstrated something else: XamlCompositionBrushBase. So it's worthwhile to again mention CompositionVisualSurface.

In any event, CompositionVisualSurface is still not an implementation of Windows.UI.Xaml.Media.Brush, thus a UWP equivalent of WPF VisualBrush is still desirable.

[liquidboy]

I was looking for a similar brush too, i vote for bringing this to winui

[Sergio0694]

Not sure if this is a perfect workaround for what @verelpode is trying to do, but we have at least partial support for such a feature in the various brushes and effects in the Windows Community Toolkit, specifically in the Microsoft.Toolkit.Uwp.UI.Media package.

In particular, there's the PipelineBuilder class (and accompanying PipelineBrush), which basically allows you to define a custom Win2D/Composition effects pipeline that's then rendered to a brush, eg. you can use it to recreate everything from a custom acrylic brush built from scratch to literally any other pipeline you might want to add. The PipelineBuilder exposes a number of APIs to create a brush also from a direct input CompositionBrush or IGraphicsEffectSource instance.

I'm thinking you might be able to achieve a tiled vectorized image brush by creating a pipeline like so (general idea):

• Create a CompositionGraphicsDevice instance, and get a CompositionDrawingSurface
• Create a CanvasDrawingSession and draw your vector geometry on that surface
• Create a CompositionSurfaceBrush from the Compositor instance and the drawing surface above
• Pass this CompositionBrush to PipelineBuilder.FromBrush()
• Add a Win2D BorderEffect to the same pipeline
• Call the AsBrush() method on the pipeline to finally get a XamlCompositionBrush instance

You should then be able to just use this brush in your XAML controls as usual, and it'll render an infinitely-repeating tiles surface with your original geometry drawn to that surface. If you try this out, let us know how it goes! 😄

Also for completeness, even though it's not exactly the same, you can also use the TilesBrush from the same package, and set the DPI scaling to preserve the ones from the initial image - this will still give you a 1:1 rendering of your initial image, endlessly repeated on your target area. I'm using this in a pipeline for my custom acrylic brush and I can confirm that the image is rendered with the correct scaling on different devices, so you shouldn't have to worry about your source image being pixelated on other devices, as the image is just tiled and not actually stretched, with the correct DPI scaling depending on your monitor/DPI in use 🙂