Performance issues

[giaur500]

I noticed performance when zooming/moving is actually poor in compare to native solutions like Google Maps or Mapbox official api. On low end devices map is almost unusable.

From other side, raster tiles from flutter map plugin are still very fast. Are you aware performance issues and is there anything possible to make it working faster?

[FaFre]

If you are interested, there are some past conversations in the closed issues where you can have a read and might add some ideas. The tile rendering (drawing on canvas) is currently only possible in the flutter main isolate (flutter/flutter#75755), when I'm not mistaken this is still the bottleneck.

You can improve performance by tuning your style. Feel free to do your own experiments and share it with us :)

[greensopinion]

We've done a lot of work to address performance issues, but it's still not as good as raster or native maps. Poor performance is most noticeable with tiles having a lot of data (e.g. in LA or London).
You can find previous discussions by searching for other issues

The theme can make a big difference as noted by @FaFre (thanks!) but you may find that raster tiles are needed to get the performance that you want, depending on your use-case.

I've added some examples that you may find helpful here: https://github.com/greensopinion/flutter-vector-map-tiles-examples
There are two custom themes in there that may help with performance.

[giaur500]

As far as I understand, currently is not possible to optimize it becuse of Flutter limitations? Ok, I will try themes, Indeed, a lot of data causes poor performance.

[FaFre]

@greensopinion I pushed the flutter issue and got a response that flutter_isolate plugin might improve performance and gain access to dart ui: flutter/flutter#75755 (comment)

However, this plugin is only Android/iOS compatible, do you see any chance this would improve the current performance?

[greensopinion]

Yes, there's a good chance that it could be used to offload some CPU-intensive tasks off the UI thread. We are already offloading some work to isolates, but there are many things that can't be offloaded due to use of dart:ui.

I imagine that the easiest and most beneficial thing to do would be to render a raster image on an isolate. This could either be integrated directly with the existing flutter_map raster tile capabilities, or perhaps in vector_map_tiles. The idea would be to render vector tiles as images and supply those to flutter_map. The benefit over using raster tiles directly would be choice of themes client-side, and rendering tiles at higher zoom levels that aren't normally available. At 2x (retina) size raster tiles look pretty sharp, but we could even render tiles at incremental zoom levels (e.g. a tile at 14.25 zoom instead of 14 then scaled to 14.25)

I'm a little reluctant to invest in functionality that is using officially unsupported flutter/dart capabilities, such as dart:ui on non-UI isolates. As noted on flutter/issues/75755 things may already break with Impeller.

If we consider moving forward with this idea, we should first evaluate how Impeller affects performance. For example, a comparison running the example library in the profiler with Impeller and without would be really helpful.

[giaur500]

@greensopinion do you think it's possible to disable Impeller and use isolates on Android only?

[greensopinion]

At least for now it's possible - but that will likely change once Flutter fully adopts Impeller since they won't want to maintain two separate rendering frameworks for Android indefinitely.

[giaur500]

I don't know how much work is it, but I think better to use isolates on Android, even as temporary solution. Still more than nothing.

[greensopinion]

However, this plugin is only Android/iOS compatible, do you see any chance this would improve the current performance?

I've pushed an experiment to https://github.com/greensopinion/flutter-vector-map-tiles-raster-experiment
Feel free to try it out!

[giaur500]

It works pretty fast, almost the same performance as native Google Maps. Very good result in my opinion. Not tried on iOS though

[aamirki]

Hey I just tried out that experiment on iOS and the performance is really good!

[giaur500]

After more testing. It seems to be better, but still not as good as rester map or native maps like Google Maps. I will test without any style applied to see how does it work, possible only style applied causes better performance

[greensopinion]

I've created a branch in this repository for experimentation: flutter-vector-map-tiles/tree/raster-experiment

There are a few things needed to take it further:

1. there may be a memory leak
2. the API is unnatural and needs revisiting
3. there is no way to plug in rendering in a separate isolate
4. tuning

[Pepslee]

However, this plugin is only Android/iOS compatible, do you see any chance this would improve the current performance?

I've pushed an experiment to https://github.com/greensopinion/flutter-vector-map-tiles-raster-experiment Feel free to try it out!

It works really cool. But there is some problem in my case. When I'm panning for a long distance it seems that a huge number of isolates creates for all panned tiles. Seems like tile provider doesn't drop deprecated isolates for tiles which was rapidly panned. For example, I have 100 tiles on the map and I`m trying to pan from 1 tile to 99 tile by one swipe. Tile provider starts to create 99 isolates and tries to process them all but not just skips unnecessary tiles and process only several last tiles.

[greensopinion]

The experiment is only intended as a proof-of-concept. There is a lot of work needed to make it ready for real usage.

I've been working on #129 to introduce raster images as an alternative to vectors. Raster images have the advantage of improved frame rate (less jank) and they only need to be rendered the first time since they can be stored in the cache for future use. The result is quite good. Of course, raster images aren't as sharp at fractional zoom levels. #129 is now available as version 3.3.0 of this library.

[Pepslee]

The experiment is only intended as a proof-of-concept. There is a lot of work needed to make it ready for real usage.

I've been working on #129 to introduce raster images as an alternative to vectors. Raster images have the advantage of improved frame rate (less jank) and they only need to be rendered the first time since they can be stored in the cache for future use. The result is quite good. Of course, raster images aren't as sharp at fractional zoom levels. #129 is now available as version 3.3.0 of this library.

Thank you, I have already tried this update, it work really faster, but I still have freezes ((. In any case any calculation in main thread entail freezes of the UI. Even if they won't be visible for the human eye, but they will still affect any timer or animation in app.
I hope this problem will be solved with impeller, and there will be a mechanism for rendering in non main thread.

[JaffaKetchup]

@greensopinion Please note https://pub.dev/packages/flutter_map/versions/4.0.0-dev.1/changelog

[greensopinion]

Thanks Luke. Is the update to let me know about API changes or does it specifically relate to performance?

…

On Tue, Apr 4, 2023 at 5:15 AM Luka S ***@***.***> wrote: @greensopinion <https://github.com/greensopinion> Please note https://pub.dev/packages/flutter_map/versions/4.0.0-dev.1/changelog — Reply to this email directly, view it on GitHub <#120 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEKDQ7WZYRQ3S72XYZHLMNDW7QGGPANCNFSM6AAAAAAU4Q2O2E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[JaffaKetchup]

@greensopinion Both. Probably not a massive performance improvement, but there is a chance it will help. There are major API changes - but I'm not sure if any of them will affect you. Probably the most major for plugin maintainers is the renaming of Coords to TileCoordinates.

[greensopinion]

Thanks @JaffaKetchup - created #136 in preparation for flutter_map major version change.

[mdmm13]

Hey @greensopinion, was trying to follow up on your discussion here about using flutter_isolate to offload processing yet still able to call dart.ui functions outside the main isolate. It's not the full solution I understand from your comment here, but part of it. The other part being the complexity of the theme, which you're trying to solve by selectively delaying the rendering if I understood correctly.

I'm not sure I got it right, but per nmfisher's (flutter_isolate maintainer) last comment here, the status on this PR being merged into dart's main branch and the comment here, it appears that the functionality that Impeller previously blocked on iOS should now be back again with Dart 3?

Does this mean, flutter_isolate + selective delays COULD solve many of our performance issues? What am I missing?

[DavidDohmen]

Hey, @greensopinion - have you considered drawVerticesRaw for rendering? It seems to be orders of magnitude faster than using widgets or canvas.
This video explains it very thoroughly and the bookmarked section shows a comparison between the different methods;
https://youtu.be/pD38Yyz7N2E?t=742

[mdmm13]

Hey, @greensopinion - have you considered drawVerticesRaw for rendering? It seems to be orders of magnitude faster than using widgets or canvas. This video explains it very thoroughly and the bookmarked section shows a comparison between the different methods; https://youtu.be/pD38Yyz7N2E?t=742

Would probably solve all problems with vector maps without going GL?

[JaffaKetchup]

Just for context, we're looking into potentially using drawVertices to draw polygons in FM, but that's still in mostly private discussion.
You may also be interested in dart_earcut (self-promo :D), which will perform the triangulation needed for drawVertices extremely quickly (it doesn't work well on self-intersecting polys, they'll need to be simplified into multiple simple polys).

[greensopinion]

Looks promising! Feel free to prototype something.

[DavidDohmen]

Unfortunately I have neither the time nor skillset to do so. We would however contribute financially with a bounty, if a certain speed level is reached, because then flutter_map would become feasible as an alternative to mapbox for us.
So if anyone with the right skillset wants to pick this up, let's talk!

[j-bbr]

I might be able to spend some time on this in the near to medium term. So I was wondering, first of all, whether somebody is already working on this or on similar things with drawVertices in FM as @JaffaKetchup mentioned?

[JaffaKetchup]

@j-bbr Right now we are not implementing drawVertices, and if we do, it'll be for Polygons only: we have no plans to make a vector layer for now.

[JaffaKetchup]

I have just had a little experiment with a custom Mapbox style, and noticed that setting VectorTileLayerMode.vector actually gave considerably better performance on my computer than the raster mode. It may be worth revising the documentation.

[greensopinion]

That's cool @JaffaKetchup - can you quantify it?

[j-bbr]

I've put a profiling / integration test for vector maps based on this example from the documentation in this repository here. The test simulates map interaction (moving the map, zooming in, flinging the map again, zooming out etc) and records performance related data in a summary (examples attached) and an event trace .json file. The trace file can be viewed with chrome://tracing (see image)
It should be straightforward to change the map vector style, map starting location, vector vs. raster mode, package version (released or from git branch / revision ) and get reproducible, comparable results.

As an example I've attached 2 summaries, profiled on a Samsung S20, using the Stadia Maps OSM Bright style once in vector and once in raster mode and moving / zooming over Tokio.
tokio_map_timeline.timeline_summary_raster.json
tokio_map_timeline.timeline_summary_vector.json

[greensopinion]

@j-bbr that's really cool! Having a repeatable way to produce profiling data could help to drive improvements. It's always a challenge to know if a change makes a difference. Do you plan to improve it further?

[j-bbr]

Yes, I would like to and I'm open to input for next steps as well. At the moment I am still wondering which metrics are most insightful to compare. For vector mode it's fairly straightforward to see that the frame rasterizer times are above the 16 ms budget for smooth 60 fps but in the raster mode its more difficult to tell from the timelines since the delay happens of the UI-thread.
I think it would be great too if this could ultimately be done in CI (Firebase Test Lab or similar) but unfortunately it doesn't seem very straightforward.

[JaffaKetchup]

@j-bbr It would be awesome if you could make a version without the vector tiles stuff, and it would be much appreciated as a test in FM core!

[mdmm13]

Just FYI, interesting debouncing support being added to the core flutter_map, to drive performance for lower-end devices and web. Great video showing impact here.

Might be worth checking out for vector also, since the best performance optimization is doing less computations in the first place.

[Hellomik2002]

I am not sure but maybe it can be helpful? https://pub.dev/packages/dart_ui_isolate
@chipweinberger Released new plugin

[greensopinion]

Just FYI, interesting debouncing support being added to the core flutter_map, to drive performance for lower-end devices and web. Great video showing impact here.

Thanks @mdmm13 that's helpful! I'm curious as to why the default debounce is 0ms.

Would you be interested in submitting a PR? A good place to start: https://github.com/greensopinion/flutter-vector-map-tiles/blob/main/lib/src/grid/grid_layer.dart#L144

[JaffaKetchup]

I'm curious as to why the default debounce is 0ms.

We decided 0 was best as it the feature does impact functionality a fair amount. New tiles aren't loaded at all until the debounce is complete: a long fling will not load any tiles, for examples.

[greensopinion]

I am not sure but maybe it can be helpful? https://pub.dev/packages/dart_ui_isolate @chipweinberger Released new plugin

@Hellomik2002 cool to see more options come up. Do you have any idea how this would be an improvement over flutter_isolate?

We've prototyped something with flutter_isolate which makes interactions glassy smooth - but it needs work before it could be useful in a real app. Feel free to try it out replacing flutter_isolate with dart_ui_isolate.

[ignatz]

Since dart_ui_isolates came up, I'm going to share my vector tile layer: https://github.com/ignatz/vector_tile_it/tree/main/example .

It contains all the optimizations mentioned before and is just generally a bit less layered. That said, it's pretty untested and I'm sure that my wild experiments introduced some regressions. I'm not using it productively myself since I'm still contemplating switching to a rust based rasterizer (at least for the tiles).

Enough disclaimer, I played with flutter_ui_isolate to see how much of an impact it has. That said, it's important to note that flutter_ui_isolates are only supported on android, ios and osx. So if you wanna play around with it, it can be easily disabled.

Do you have any idea how this would be an improvement over flutter_isolate?

The two main differences are that they have less memory overhead and you can use dart:ui (flutter/flutter#10647). Based on the issue I would have naively expected you to encounter issues but maybe things have improved since the bug was filed in 2017 🤷‍♀️

[mdmm13]

Thanks @mdmm13 that's helpful! I'm curious as to why the default debounce is 0ms.

Would you be interested in submitting a PR? A good place to start: https://github.com/greensopinion/flutter-vector-map-tiles/blob/main/lib/src/grid/grid_layer.dart#L144

Sure, but won't it automatically be added as soon as flutter_map gets released as v6.1 or v7? It's already been merged into master there?

[Soap-141]

Someone know when it will be available?

[greensopinion]

Since dart_ui_isolates came up, I'm going to share my vector tile layer: https://github.com/ignatz/vector_tile_it/tree/main/example .

@ignatz very cool to see how you were able to integrate the renderer with very little code. vector_map_tiles suffers from code complexity due to the many iterations it took to evolve and my many hacks. If you've learned anything that you would like to see make it back into vector_map_tiles let me know.

[ignatz]

I'm not quite sure what you have in mind. Don't get me wrong, I'm super appreciative of the work that you've done especially for vector_tile_renderer but I wrote my own tile layer on top because I didn't think it was possible for me to make the changes I wanted to make to vector_map_tiles.

You can certainly pull in my changes to vector_tile_renderer, they should benefit any user and were some -70% reduction for Mapbox terrain maps (I don't even remember anymore).

As for the tile layers, I'm not sure what "I would like to see make it back" or how that would work. My layer takes the hybrid approach: rasterized terrain with crisp vector labels and symbols, has pretty much all I'd expect from a tile layer, and is ~1/3 the code. At the expense of sounding accidentally presumptuous, if you're not happy with the state of vector_map_tiles, I'd love for you to take ownership in vector_tile_it (the name is negotiable :) )

Also to be clear, my initial excitement has worn off a bit. I'm not sure that even with all these improvements and changes, flutter-canvas-based vector tiles would meet my personal performance goals. As I've mentioned before, I've toyed with the idea of pushing the rasterization to a mapboxlibre-rs renderer 🤷‍♀️. But even then it wouldn't be true vector maps but rather faster raster maps with vector labels. I certainly don't get the point of rasterizing everything, why wouldn't I just use raster map tiles? (I guess if you really wanted to change the styling on the client but that sounds rather niche)

[mdmm13]

Tessellation or not, the issue you linked sounds very relevant and plausible: flutter/flutter#131345 . If that was the case, we're not talking bugs but rather crappy implementation. According to the issue, a solution is on halt for "higher priority" work. (tangential rant: gotta love flutter government, hope they'll eventually hand it over to a foundation). Meanwhile, playing with the blending might provide some insights 🤷‍♀️

Just FYI, Flutter 3.22 appears to improve tessellation issues drastically. Might help.

[ignatz]

I just quickly ran 3.19 and 3.22 side-by-side. I wasn't sure if Linux was also using the new vulkan-based impeller renderer, looking at the output I'm leaning yes:

Maybe worth waiting a few fixes :hide:

Also worth highlighting from the article you linked:

While pleased with these improvements, there is still more work to do. Among other opportunities, we are aware that polyline generation remains prominent in CPU profiles, and we intend to investigate shifting this work to the GPU, as well.

[mdmm13]

I just quickly ran 3.19 and 3.22 side-by-side. I wasn't sure if Linux was also using the new vulkan-based impeller renderer, looking at the output I'm leaning yes:

Thank you for the quick check - looks like a 20-30% performance improvement on average.

Also worth highlighting from the article you linked:

While pleased with these improvements, there is still more work to do. Among other opportunities, we are aware that polyline generation remains prominent in CPU profiles, and we intend to investigate shifting this work to the GPU, as well.

The fact that they're stating that in the official announcement means that they're probably working on it :-)

[Hellomik2002]

I'm not quite sure what you have in mind. Don't get me wrong, I'm super appreciative of the work that you've done especially for vector_tile_renderer but I wrote my own tile layer on top because I didn't think it was possible for me to make the changes I wanted to make to vector_map_tiles.

You can certainly pull in my changes to vector_tile_renderer, they should benefit any user and were some -70% reduction for Mapbox terrain maps (I don't even remember anymore).

As for the tile layers, I'm not sure what "I would like to see make it back" or how that would work. My layer takes the hybrid approach: rasterized terrain with crisp vector labels and symbols, has pretty much all I'd expect from a tile layer, and is ~1/3 the code. At the expense of sounding accidentally presumptuous, if you're not happy with the state of vector_map_tiles, I'd love for you to take ownership in vector_tile_it (the name is negotiable :) )

Also to be clear, my initial excitement has worn off a bit. I'm not sure that even with all these improvements and changes, flutter-canvas-based vector tiles would meet my personal performance goals. As I've mentioned before, I've toyed with the idea of pushing the rasterization to a mapboxlibre-rs renderer 🤷‍♀️. But even then it wouldn't be true vector maps but rather faster raster maps with vector labels. I certainly don't get the point of rasterizing everything, why wouldn't I just use raster map tiles? (I guess if you really wanted to change the styling on the client but that sounds rather niche)

publish it, let's see it

[JaffaKetchup]

I was doing some experiments recently with custom shaders in flutter_map to render raster tiles with opacity more efficiently. Unfortunately it wasn't useful in the end (barely any noticable performance improvement & flutter/flutter#133944), but I did learn a tiny bit about shaders, so not all was wasted!

(top secret behind the scenes chat ;))

However, I was wondering whether this would even be remotely possible here. I did some searching and found: http://kunzhou.net/zjugaps/pathrendering/ (https://web.archive.org/web/20240316141847/http://kunzhou.net/zjugaps/pathrendering/). And they've seeming extensively tested it with OSM data! They provide all the code and a paper, and seem to have both vertex and fragment shaders. Unfortunately it doesn't look like Flutter supports vertex shaders (yet; see below), but maybe this is worth something.

Also note https://github.com/flutter/flutter/blob/master/docs/engine/impeller/Flutter-GPU.md: flutter/flutter#130921, https://docs.google.com/document/d/1Sh1BAC5c_kkuMVreo7ymBzPoMzb7lamZRPsI7GBXv5M/edit?resourcekey=0-5w8u2V-LS41tCHeoE8bDTQ & https://github.com/orgs/flutter/projects/134/views/1?pane=issue&itemId=54689466. This could really enhance GPU capabilities (and also open the door to vertex shaders).
I think this could be a really promising development. See existing examples with vertex and fragment shaders: https://github.com/bdero/flutter-gpu-examples.

[kekland]

I feel like the way to go would be to wait until https://github.com/flutter/flutter/blob/master/docs/engine/impeller/Flutter-GPU.md drops (so that there's no need to hack your way around trying to add OpenGL/Vulkan/Metal bindings on different platforms) to be able to use vertex shaders.

For now it seems like you can use it only on macOS (and it's highly experimental)

[JaffaKetchup]

Yep, agreed, wasn't going to suggest implementing the bindings alone! (Hey, long time no see @kekland :D)

Of course, there's probably more that can be done here with Flutter. I haven't had a proper look through https://github.com/greensopinion/dart-vector-tile-renderer though. FM's polygon renderer is extremely complex now, and we've squeezed out everything possible, to the point where some of the biggest drains come from Flutter trying to render strokes (disabling caps and joins improves it somewhat, and using lines of 1 device pixels helps).
In addition to the drawVertices mode (which sometimes helps, sometimes hinders), we use batching to join paths together, and minimize draw calls. This has a hugely positive effect on performance - although of course, it might be more complex to do this properly here, to preserve stacking order whilst keeping the order in such a way that batching is effective (we have to draw and prune the buffer whenever some characteristics between the last and current polygon change, and labels and opacity also need special expensive handling).

In summary, there's probably 3 things to try:

1. Improve the renderer using more advanced techniques - not very easy, but fully Dart & Flutter
2. Offload onto a UI-capable isolate/thread - easiest, but liable to break and introducing native code
3. Start experimenting with/wait for Flutter GPU support to use custom shaders

[kekland]

@JaffaKetchup Hi, yeah, great to see you :P

I'll try to play around with the existing GPU support whenever I'll have some spare time to see at what state it's currently at and whether we can benefit from parallelizing the process of converting the tile data to a vector.

So far the performance seems to be alright when using the raster mode, but it would be great to have a full Google Maps-like experience. Haven't tested on lower-end devices yet though

[T-moz]

Wow, did you see the new announcement about flutter GPU preview ?
https://medium.com/flutter/getting-started-with-flutter-gpu-f33d497b7c11

[JaffaKetchup]

That's quite an announcement! I have no idea where to even begin implementing the algorithm I mentioned above, but it should at least now be possible!

(Strange, I was just checking the progress of flutter_gpu yesterday, wondering what was going on :D)

[mvarendorff2]

Hey thread!

I am currently looking into lags occurring when moving the map while tiles are loading / rendering. One particular comment stood out to me in this thread:

We've #120 (comment) which makes interactions glassy smooth - but it needs work before it could be useful in a real app.

@greensopinion what work were you referring to there? Glassy smooth interactions sound exactly like what I am looking for at the moment and I'd love to contribute to bring using isolates along to a "production-ready" state!

I am also curious about thoughts from anyone here about flutter_isolate vs dart_ui_isolate. My understanding is that dart_ui_isolate would be preferable since we

a) don't need plugin support (and even if we did, since flutter 3.7 using plugins is supported in isolates)
b) do like the sound of instant isolates and 99% reduced RAM usage

but I might be missing something here. One thing that has me worried of sorts is the quiet state of the dart_ui_isolate repository.

Let me know what you think! I will get to work on rebasing the isolates branch probably next week to see how performance changes in my little testbench and would very likely incorporate all other suggested required changes over the next week as well if possible.

[greensopinion]

Awesome, great to have someone looking into this. the comment that you linked has 4 points that summarize the work that I thought was needed.

My experiment isn't great code - just a hack to see what's feasible - so feel free to do something different. There is another implementation described and linked in this thread that might be even better.

I haven't looked into dart_ui_isolate vs flutter_isolate. I agree that we don't need plugin support, so potentially dart_ui_isolate is preferable. Something else to consider is long-term support and maintainability of the library.

Have fun!

[mvarendorff2]

Thanks a bunch!

A quick update from my end - I did rebase and play around with both vector_tile_it and the isolate-branch of this repository and hit much bigger surprises than anticipated in both implementations (and both dart_ui_isolate and flutter_isolate) including quite heavy visual artifacts:

and some tiles being rendered very evidently in the wrong place:

Streets were also seemingly not rendered within cities although I cannot tell with certainty whether that is related to styles or the rendering.

In case anyone is curious or wants to try it out themselves, the styleUri is https://sgx.geodatenzentrum.de/gdz_basemapde_vektor/styles/bm_web_col.json.

Since I was spearheading this advancement in the context of a client project with limited budget I unfortunately (and with a bad conscience) have decided with my PM to not pursue this route further for the time being 😔

I really hope that I can get some spare time together to take another look at this maybe in december. This package has been invaluable for me in the past and I would love to squeeze a bit more performance out of it.