[Feature] Integrate CompositionProToolkit Rendering Surfaces

[ratishphilip]

Describe the problem this feature would solve

CompositionProToolkit library provides five types of rendering surface interfaces.

• IRenderSurface - This interface acts as the base interface for interfaces which render to the ICompositionSurface. It mainly contains references to an ICompositionGenerator object and an ICompositionSurface object which are the core objects required for rendering any geometry or image onto a ICompositionSurface.
• IMaskSurface - This interface is used for rendering custom shaped geometries onto ICompositionSurface so that they can be useds as masks on Composition Visuals.
• IGaussianMaskSurface - This interface derives from IMaskSurface and is used for rendering custom shaped geometries onto ICompositionSurface so that they can be useds as masks on Composition Visuals. You can apply a Gaussian Blur to the mask.
• IGeometrySurface - This interface is used for rendering custom shaped geometries onto ICompositionSurface.
• IImageSurface - This interface is used for rendering images onto ICompositionSurface.
• IImageMaskSurface - This interface is used for creating a mask using the alpha values of the image pixels.

Here is the interface hierarchy

These surfaces can then be used to create brushes which can be used for rendering custom shapes and images.

These surfaces can also be used for creating masks from geometric shapes or images.

Using Win2d, these surfaces can be updated at 60fps to create an animation.

This can also be used to create custom shaped backdrop brush too.

Describe the solution

I would like to port the above to WindowsCommunityToolkit. This would exist in the Microsoft.Toolkit.Uwp.UI.Media project under the Microsoft.Toolkit.Uwp.UI.Media.Surface namespace.

In addition to the render surfaces, there would be Brushes (deriving from XamlCompositionBrushBase) corresponding to each of the rendering surfaces, so that it would be easy to use them directly in XAML.

Hello, 'ratishphilip! Thanks for submitting a new feature request. I've automatically added a vote 👍 reaction to help get things started. Other community members can vote to help us prioritize this feature in the future!

[michael-hawker]

Thanks @ratishphilip could you provide a bit more detail on where this would live in the toolkit and what developers will be able to do with this platform? I'm assuming it's about creating more complex effects, but it'd be good to know what that looks like in terms of other technologies this applies to.

@Sergio0694 I think you understand this space more than I do. These could also be integrated with the Pipeline brushes, right?

[ratishphilip]

@michael-hawker I have updated the details. Kindly let me know if that would suffice.
(Thanks for updating the title 😊 I missed that out)

[ratishphilip]

@Sergio0694 It would be awesome if it is integrated with Pipeline Brushes.

[Kyaa-dost]

@Sergio0694 thoughts on this feature?

[Sergio0694]

Oh, wow, how did I miss these notifications? 🤦
Thanks @Kyaa-dost for the new ping!

@ratishphilip These surface APIs look super cool, and I think it would definitely make sense to have them in the Toolkit as well, especially considering you've already ported the Win2D mini path language and other geometry APIs already in #3430. Have you thought about what kind of integration with the pipeline APIs you'd like to add? Would that be a second step once this feature is merged, or were you planning to alter the API surface from the start for better integration? And if so, how?

Just curious to know more details in case you've already done a preliminary investigation of the possible changes related to this 🙂

[michael-hawker]

Yeah, since we have revamped the animations as well to work with the Effects library, it'd be cool if we could extend or build on top of all these bits so they all play nice together.

Am I correct that these interfaces get used and tied into the composition APIs in your existing library here via extensions?

Seems a little similar to the methods Sergio's exposed on the PipelineBuilder API set, so maybe we just bring those over as new capabilities, and then add new effects to expose to XAML and animations?

[ratishphilip]

Thanks @Sergio0694. I have completed the porting the rendering surfaces from CompositionProToolkit to Windows Community Toolkit. However, I felt it might still be a little complicated for the users to define every thing in code.
So I started creating brushes (derived from XamlCompositionBrushBase) for each of the surfaces. This way it would be easy to define them in XAML.
Creation of the brushes is 80% done and the documentation is pending. (It should be done in a couple of weeks. Sorry for the delay, my day job is keeping me quite busy)
Once I complete the above, I am planning to submit a PR.
Then, once it is merged, we can plan about integrating it with the PipelineBrushes - would like to know your opinion, too, whether we need to alter the API surface.

[Sergio0694]

That sounds great to me, and I'm really curious to see the changes in the PR and the documentation! 🙂

"Sorry for the delay, my day job is keeping me quite busy"

There's absolutely nothing to be sorry for! You're doing these contributions on your own initiative and to help the community, and we appreciate your time working on this! You don't have to justify yourself if life keeps you busy outside of this project! 😄

[ratishphilip]

Hi @Sergio0694 @michael-hawker ,
I have submitted the draft PR for the render surfaces and their corresponding XAML brushes.
Kindly review it.

Still working on the documentation. Will update PR once done.