Possible leak of framebufferMemory in XVideoInit

[abaire]

XVideoInit does a check to see if the previously saved framebuffer is already the size of the requested framebuffer at

nxdk/lib/hal/video.c

Line 373 in f2c634f

if (previousFBSize == screenSize) {

. If it is the same size, it reuses the previously saved framebuffer, but does not deallocate framebufferMemory or pass the newly reassigned _fb pointer to AvSetDisplayMode.

I'm not sure if it's a leak or if it's important that there's a mismatch between the framebuffer used in AvSetDisplayMode and the data persisted via AvSetSavedDataAddress and XVideoGetFB.

[kosmas12]

Good day!

I'm the person that authored the reusage code, so I believe I'm the most qualified person for this.

There indeed seems to be a leak there, nothing weird going on with framebuffers.

As for passing _fb, I don't get what you mean. No matter what happens with the framebuffer, _fb is always passed to AvSetSavedDataAddress. Maybe I understood your question wrong. If I did, then please elaborate.

Thanks!

[abaire]

Hi!

The issue I meant wrt _fb is that it is not passed to AvSetDisplayMode if it's reused, so the display framebuffer and the saved framebuffer differ, which seems odd.

I suspect the right approach might be to check the saved framebuffer size before allocating any new memory and before calling AvSetDisplayMode so if it's possible to reuse it the method can bomb out early.

Also ran into #507, though I think reusing the framebuffer would actually fix that.