FFMpeg Video -> GIF conversion memory error

[nipunbatra]

Hi, I am converting videos to GIF using FFMPEG. This works fine for small videos (around 4 MB), but throws Out of memory error on larger videos (around 40 MB). I’m sure this is mainly a limitation of the iPadOS memory restrictions? I guess we do not really have something like top/ps to figure the memory consumed in real time. So, I can not report how much the conversion took.

Testing setup

The files I use for testing

$ ls -lah|grep Video.MP4 
-rw-r--r--@  1 mobile  mobile    35M Jun 16 14:04 HEIC-Video.MP4
-rw-r--r--@  1 mobile  mobile   3.7M Jun 14 12:58 Video.MP4

The command that I use for converting

Works (smaller file)

ffmpeg -i Video.MP4  -filter_complex "[0:v]  fps=12,scale=720:-1,split [a][b];[a] palettegen [p];[b][p] 
paletteuse" CombinePDF.gif

Does not work (larger file)

ffmpeg -i Video.MP4  -filter_complex "[0:v]  fps=12,scale=720:-1,split [a][b];[a] palettegen [p];[b][p] 
paletteuse" CombinePDF.gif

Error

Error while filtering: Out of memory890kB time=00:00:00.00 bitrate=N/A speed=   0x    
Failed to inject frame into filter network: Out of memory
Error while processing the decoded data for stream #0:1

It should be notes that the 890kB you see in the above error is technically not a part of the error and is printed on the console before the error occurs. The 890kB is from

frame=    0 fps=0.0 q=0.0 size=     890kB time=00:00:00.00 bitrate=N/A speed=   0x 

While I think that ffmpeg will run faster when multi-threading support comes to WASM, I don’t think the memory bottleneck will be solved. Is that correct?

I saw that the latest dev notes allow for asking for more memory: https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_kernel_increased-memory-limit

[holzschu]

It's possible to know the memory used by a process... if you're running from Xcode.

You are right about the multi-threading and the memory entitlement. The WebAssembly execution is running in a separate process, which may or may not be included inside the application maximum. It's possible that WebAssembly has a different memory maximum, independent from the application. 40 MB seems a bit small compared to what's available on a normal iPad, even after decompression of all frames. I'm thinking of including a different WebAssembly interpreter, to work around the existing limitations.