SkiaScene availability for Android and Windows

[juliusdeblaaij]

Hello,
I was delighted to see that you ported SkiaScene to MAUI, as you alluded to here: OndrejKunc/SkiaScene#27.
After having written some example code, I noticed that your implementation is only available on iOS. Is this a bug, or simply because you haven't gotten around to the implementation?
Thanks for all the hard work!

I am running .NET 7.0 with the following workloads:

Installed Workload Id | Manifest Version | Installation Source

maui-windows | 7.0.92/7.0.100
maui-maccatalyst | 7.0.92/7.0.100
maccatalyst | 16.4.7089/7.0.100
maui-ios | 7.0.92/7.0.100
ios | 16.4.7089/7.0.100
maui-android | 7.0.92/7.0.100
android | 33.0.68/7.0.100

[FreakyAli]

I thought I added Android, did I miss it? Damn! I Haven't used it yet the same reason why it's still left it in pre-release. Windows I'm not sure if I would wanna add but if you could use it. I guess I could do it over the weekend. Tell me what part seems to be missing I'll try and check it out!

[juliusdeblaaij]

Hey, woah quick reply.
TLDR:

• A working sample which listens to gestures (even if it is iOS only) will allow people to continue developing while your looking at Android
• Getting your implementation to work on iOS and Android would be amazing.
• Is TouchTracking required, or can regular gesture recognizers serve as a stopgap?

In more detail now; I tried to implement this sample:

https://github.com/OndrejKunc/SkiaScene/blob/master/samples/FormsSample/SkiaScene.FormsSample/MainPage.xaml.cs
https://github.com/OndrejKunc/SkiaScene/blob/master/samples/FormsSample/SkiaScene.FormsSample/MainPage.xaml

But i.e. tapping the canvas or otherwise triggers no _touchGestureRecognizer.OnTap event.

Furthermore, creating a touch handler instead of using TouchTracking as a XAML effect is impossible.

touchHandler = new TouchHandler() { UseTouchSlop = true };
touchHandler.RegisterEvents(canvasView); //Pass View to the touch handler
touchHandler.TouchAction += OnTouch; //Listen to the touch gestures

(Cannot convert SKCanvas to UIKit UIView) (probably related to there only being an iOS implementation).

Currently, while the scene renders a simple shape, it doesn't respond to any gestures.

To even get my code-behind to compile, I had to be generaous with IF statements:

using SkiaSharp.Views.Maui;
using SkiaSharp;

#if IOS
using Maui.FreakyEffects.SkiaScene;
using Maui.FreakyEffects.TouchTracking;
using Maui.FreakyEffects.SkiaScene.TouchManipulation;
using Maui.FreakyEffects.Platforms.iOS;
#endif

namespace PrismMauiBlank.Views;

public partial class SkiaScene : ContentPage
{
#if IOS
    private ISKScene _scene;
    private ITouchGestureRecognizer _touchGestureRecognizer;
    private ISceneGestureResponder _sceneGestureResponder;
#endif

    public SkiaScene()
	{
		InitializeComponent();
        this.SizeChanged += OnSizeChanged;
    }

    private void OnSizeChanged(object sender, EventArgs eventArgs)
    {
        SetSceneCenter();
    }

    private void SetSceneCenter()
    {
#if IOS
        if (_scene == null)
        {
            return;
        }
        var centerPoint = new SKPoint(canvasView.CanvasSize.Width / 2, canvasView.CanvasSize.Height / 2);
        _scene.ScreenCenter = centerPoint;
#endif
    }

    private void InitSceneObjects()
    {
#if IOS
        _scene = new SKScene(new MyRenderer())
        {
            MaxScale = 10,
            MinScale = 0.3f,
        };
        SetSceneCenter();
        _touchGestureRecognizer = new TouchGestureRecognizer();
        _touchGestureRecognizer.OnTap += _touchGestureRecognizer_OnTap;
        _sceneGestureResponder = new SceneGestureRenderingResponder(() => canvasView.InvalidateSurface(), _scene, _touchGestureRecognizer)
        {
            TouchManipulationMode = TouchManipulationMode.IsotropicScale,
            MaxFramesPerSecond = 100,
        };
        _sceneGestureResponder.StartResponding();
#endif
    }

    private void _touchGestureRecognizer_OnTap(object sender, TapEventArgs args)
    {
        throw new NotImplementedException();
    }

#if IOS
    private void OnTouchEffectAction(object sender, TouchActionEventArgs args)
    {
        var viewPoint = args.Location;
        SKPoint point =
            new SKPoint((float)(canvasView.CanvasSize.Width * viewPoint.X / canvasView.Width),
                        (float)(canvasView.CanvasSize.Height * viewPoint.Y / canvasView.Height));

        var actionType = args.Type;
        _touchGestureRecognizer.ProcessTouchEvent(args.Id, actionType, point);
    }
#endif

    private void OnPaint(object sender, SKPaintSurfaceEventArgs args)
    {
#if IOS
        if (_scene == null)
        {
            InitSceneObjects();

        }
        SKImageInfo info = args.Info;
        SKSurface surface = args.Surface;
        SKCanvas canvas = surface.Canvas;
        _scene.Render(canvas);
#endif
    }
}

#if IOS
public class MyRenderer : ISKSceneRenderer
{
    public void Render(SKCanvas canvas, float angleInRadians, SKPoint center, float scale)
    {
        var fillPaint = new SKPaint()
        {
            Color = Colors.Red.ToSKColor(),
            StrokeWidth = 5,
            IsStroke = true,
        };

        canvas.DrawCircle(center, 100, fillPaint);
    }
}
#endif

A lot to unpack, sorry for the bad write-up.

[juliusdeblaaij]

I tried manual movements, but scene.MoveByVector() doesn't do anything, neither in the page constructor by itself or as part of a gesture:

    double lastTotalX, lastTotalY;

    private void PanGestureRecognizer_PanUpdated(object sender, PanUpdatedEventArgs e)
    {
#if IOS
        if (e.StatusType == GestureStatus.Started)
        {
            lastTotalX = 0;
            lastTotalY = 0;
        }

        if(e.StatusType == GestureStatus.Running)
        {
            float xDelta = (float)(e.TotalX - lastTotalX);
            float yDelta = (float)(e.TotalY - lastTotalY);

            _scene.MoveByVector(new SKPoint(xDelta, yDelta)); //Move by 10 units to the right independently from current rotation and zoom

            lastTotalX = e.TotalX;
            lastTotalY = e.TotalY;
        }

        if(e.StatusType == GestureStatus.Completed || e.StatusType == GestureStatus.Canceled)
        {
            lastTotalX = 0;
            lastTotalY = 0;
        }

        canvasView.InvalidateSurface();

#endif
    }

That is all the info I can give for now, I hope that we can work this out and that people can benefit from your implementation. 😬

[FreakyAli]

Have you tried the Sample project once maybe? Because I remember porting the whole thing to Maui...
I will try and check this over the weekend 🤞
Anyway in the meantime can you make the above changes in the sample project in the repo and see if it misbehaves as well? If it does I'll see what can be done!

[juliusdeblaaij]

I will get back to you to see how the (modified) sample behaves.

[FreakyAli]

Sure will be waiting for your response

[juliusdeblaaij]

using Maui.FreakyEffects.SkiaScene;

Actually is allowed on Android too, but I cannot decern the difference between my production environment and the sample project in regards to Maui.FreakyEffects.SkiaScene. My production env still shows:

using Maui.FreakyEffects.SkiaScene;
using Maui.FreakyEffects.SkiaScene.TouchManipulation;
using Maui.FreakyEffects.SkiaScene.TouchTracking;

(Hover): (net-7.0-android33.0) Not Available

2. In the sample project, OnPaint is never called on iOS, that seems to be a bug with SkiaSharp as it does get called on Android. Because of this InitSceneObjects never gets called, TouchGestureRecognizer = null, and tapping the screen causes a null reference exception and crash. This however doesn't happen on the production code, there PaintSurfaces calls normally the OnPaint method.

3. In the sample project (Android build), the SceneGestureRenderingResponder fires on Android a couple times after I swipe/pan/tap on the screen (so I guess it handles all touch statusses, that is good). But then no actually transformation takes place in the scene.

private void InitSceneObjects()
{
    _scene = new SKScene(new TestScenereRenderer())
    {
        MaxScale = 10,
        MinScale = 0.3f,
    };
    SetSceneCenter();
    _touchGestureRecognizer = new TouchGestureRecognizer();
    _sceneGestureResponder = new SceneGestureRenderingResponder(() => canvasView.InvalidateSurface(), _scene, _touchGestureRecognizer)
    {
        TouchManipulationMode = TouchManipulationMode.IsotropicScale,
        MaxFramesPerSecond = 100,
    };
    _sceneGestureResponder.StartResponding();
}

[FreakyAli]

@juliusdeblaaij Interesting indeed, why don't you make a branch with your changes and push it, i would like to take a look!

[juliusdeblaaij]

@FreakyAli excuse the delay, I created a PR which should allow SKCanvasView to draw on iOS. I made no other changes to the sample, the scene gesture recognizer gets called on both platforms. Simply, no scene transformation becomes visible. I would like to hear whether you had scene transformations with gestures working in the past, and that they broke, or have simply not been fully working up until now.

[FreakyAli]

@juliusdeblaaij this is either a bug with Maui Skia or InvalidateSurface isn't getting called for some reason? That's what I can think of, of the top of my head. I will check your PR and later on try and check if the transformations are erroneous or not

[juliusdeblaaij]

@FreakyAli yeah indeed it should be a bug with Skia, which then sneaked into FreakyEffects until I updated the Kkisa version that FreakyEffects used. Truth be told, I don't know how the FreakyEffects Skia version then leads to the SkiaCanvas not painitng on iOS in an axternal project (like my production env).

[FreakyAli]

@juliusdeblaaij The reason Skia updates on your project is probably because FreakyEffects updates it to the latest... its a guess though

[juliusdeblaaij]

Hi Ali, have you had the time to see if you can make FreakyEffects (touch tracking & SkiaScene) work for Android yet? @FreakyAli

[FreakyAli]

@juliusdeblaaij I was waiting for you to guide me to a branch where you were making changes!

[juliusdeblaaij]

I made no other changes to the sample,

Apologies, I thought I already indicated I hadn't made any changes (except bumping the SkiaSharp version).
Also I have good news, once I tried to manually import all freaky effect touch tracking files, the project compiled. Sadly, the third party control I'm working around still absorbs all touch events on top of it on Android... so it wasn't a success for that.
But we can be glad that the touch tracking code seems to build to Android, I have no idea why the NUGET version doesn't want to work on Android though :)

[FreakyAli]

@juliusdeblaaij Its really weird ngl, what i would recommend you to do is create a reproducible branch in the this project here itself so i can take a look, just tag me with the branch which has the issue, add some comments if you wanna explain what something is doing

[juliusdeblaaij]

@FreakyAli Good morning. There is a PR available at PR #7 .

Issues
The major cause of the problems discussed in this thread:

1. the SKCanvas doesn't show on iOS,
2. touch manipulations don't work on Android and iOS
   seems to be that the Matrices were not properly assigned to, causing them to stay in the default Identity state.

Proposed solution
Updating the Matrices, also causes the canvas to render on iOS. This might be because the Matrix width (if there even is such a property) is 0 to start with on iOS.
Please have a look at PR #7, and let me know if it works for you also.

PR #7 might make PR #6 unneccesary, as it was not a fix for the problems.

[FreakyAli]

@juliusdeblaaij Perfect i will check this out in my free time hopefully i will have a solution for you soon

[FreakyAli]

@juliusdeblaaij I checked the PR and it seems you have already made all the changes that you needed, what do you need me to do here 😕

[juliusdeblaaij]

@FreakyAli I took the liberty of writing a fix, but it might not fit your needs or standards.
If you could test the code on your end and also see whether it follows your programming style for the rest of the library. 😀

[FreakyAli]

@juliusdeblaaij Checkout this PR #8 once it's merged you will have your NuGet, I will be doing it on Sunday, so if you want any more changes do let me know, I am also going ahead with your code it looks great, Thanks for all the help brother!

[FreakyAli]

@juliusdeblaaij Assuming that you don't need anything else i will be releasing this today!