Textures are never deallocated

[keenanwoodall]

Describe the bug
When panning and zooming tiles are created, but old tiles are never deallocated. I've had multiple crashes due to this as my system has completely run out of memory 😬 This happens at runtime, but is even worse in the editor as it seems impossible to destroy the all the textures without restarting.

Destroying the map renderer or switching scenes doesn't even seem to free all the tile memory. It frees a bit, but in my simple project where texture memory is (practically) only allocated by the map renderer currently about 7GB of memory is allocated by textures. Creating a new scene drops the amount by a bit, but the only way to free the memory is to restart Unity. If I restart Unity only a few hundred MB in texture memory is allocated.

To Reproduce

1. Open the Unity profiler and add open the memory profiler.
2. Create a new scene with a map renderer.
3. Press play.
4. Pan and zoom.
5. Watch as the texture memory goes up over time and never plateaus or decreases.

Expected behavior
The map renderer should have a maximum amount of memory allowed to be used and tiles should be deallocated when the limit is surpassed.

Environment (please complete the following information):

• Unity version: 2019.2.14f1
• Maps SDK NuGet Package version: 0.4.1

Additional context

• Scripting Backend - Mono
  Api Compatibility Level - .NET 4.x

[kircher1]

Thanks for the clear repro steps. We'll take a look at this!

[keenanwoodall]

@kircher1 Just got this error

NullReferenceException: Object reference not set to an instance of an object
#=zYVCayxNbIUpAkgO0LdMgcQU=.#=zhTsCWJW6Pr5xO9XQ3VUmEwU= (Microsoft.Geospatial.TileId #=zk86ETyE=, UnityEngine.Texture2D #=ze5Zkpszghgx_, System.Collections.Generic.List1[System.Tuple2[Microsoft.Geospatial.TileId,UnityEngine.Texture2D]]& #=zG24mnepgHzBkmNL6JA==) (at <4c68cbcda0404cb7de7d9d1b8786b061>:0)
Microsoft.Maps.Unity.MapRenderer.#=zieQLcolh8hvT (System.Collections.Generic.List`1[T] #=z2WcZVBWm55TdK$Fa0uQRZKE=, Microsoft.Geospatial.MercatorBoundingBox #=zSOCewKAI2Ya1WV4JXOUOZgvOmQn$zM$2YQ==, System.Double #=zlhtaEXylx1Rd) (at <4c68cbcda0404cb7de7d9d1b8786b061>:0)
Microsoft.Maps.Unity.MapRenderer.LateUpdate () (at <4c68cbcda0404cb7de7d9d1b8786b061>:0)

I'm not sure if it's related since it's the first I've seen of it and the member names seem to be obfuscated, but I figured I'd let you know. It was thrown during some routine panning and zooming. I wasn't doing anything crazy. Maybe I just zoomed in too far, but I imagine the map renderer's min/max zoom level should prevent loading tiles at invalid zoom levels.

[keenanwoodall]

Sorry to bug you about this; I'm sure it's being looked into, but I just wanted to check in to see if you've got any new info on the issue, even if it's tentative. The project I'm working on has a pretty tight deadline and we won't be able to use this sdk right now if tiles aren't being deallocated.

[kircher1]

No worries. Unfortunately I've been unable to reproduce the memory leak... Would you be able to zip up a simple repro project? That would be really helpful. If so, please leave the Bing Maps key field empty on the MapRenderer! Thanks.

[kircher1]

Just to add, specifically I see the memory getting freed up when the MapRenderer is destroyed. Old tiles should also be deallocated over the lifetime of the map but that is based on a fixed memory threshold. When that threshold is surpassed, older tiles are destroyed. That threshold is computed by taking a percentage of system memory. 7GB seems high but maybe not if your machine has many GBs of memory.

[kircher1]

Thanks for sharing the repro project! I went ahead and removed the links in case there were some private keys in there.

A couple of observations:

• The quality offset on the MapRenderer is max'd and the texture resolution for the HTTP end point is 1024x1024. The aerial imagery the MapRenderer loads by default is 512x512, so this is 4x more texture data than usual plus the increased quality. This is causing memory to inflate more rapidly. If you can get a way with a lower quality setting and/or lower resolution textures, that may be one avenue to reduce the memory overhead.
• Something else worth noting is that the textures are being translated to RGBA which is wasting a byte for alpha. This is out of your control but a good feature request here would be to indicate on the TextureTileLayer whether or not alpha is needed. Then the most optimal format can be chosen.
• The map's caching memory limit is calculated as 2/3 of total system memory. This is very high. This logic needs to be improved with some sort of maximum amount, or smaller portion used by default. Making this configurable via an API would also be a good feature request.
• I am seeing the map data get de-allocated (the textures) when the map is destroyed.

If your machine isn't coping with the memory usage, and lowering the resolution or quality doesn't help enough, a hacky workaround would be to periodically re-create the MapRenderer to clear the cache. Maybe keying that off of how many textures have been created so far. I know far from ideal, but maybe enough to unblock you in the short term.

Hope that helps!

[kircher1]

On the point of RGB vs RGBA, the TextureTileLayer ultimately uses Unity's LoadImage extension which converts JPG or PNGs to Unity textures. According to their docs, JPGs are converted to RGB texture format and PNGs are converted to RGBA. Since this is automatically determined by Unity, there isn't a way to change it, other than changing the underlying image data to JPGs.

[keenanwoodall]

Gotcha, thanks for this info. I did another test with lower texture resolution and the texture memory didn't balloon nearly up as fast which is an improvement.

On my machine with 16 GB it still ended up filling almost all the way since more than a third of memory was already being used by other programs. However, I saw that the used memory plateaued right below 16 GB in the task manager, but weirdly kept increasing in the Unity profiler beyond 16 GB. I'm not sure why Unity would report more used memory than my PC actually has, but I think (in addition to the large memory limit) that was what mislead me into thinking there was a leak.

That being said, I've continued testing by panning until my system memory is full and then restarting. Sometimes I'd exit play mode the moment it plateaued and sometimes I'd keep panning for a minute or so to see how it handled not having any more available memory. I tried this 6 times and it crashed once when trying to exit play mode; presumably from struggling to free the 10 or so GB of memory it was using. The application definitely starts struggling to work when the memory is maxed out and it has some pretty severe hitching. Exiting play mode would also take upwards of 45 seconds.

Like you said, being able to set the max memory for tiles is definitely needed. Unless there's some reason it shouldn't be changed on the fly, some inspector property on the map renderer would suffice I think.

[kircher1]

v0.4.2 allows for customizing the max cache size and significantly reduces the default amount of memory used by the cache.