Disable or decrease stream read buffer size

[zbyte]

Overview

I`m read RTSP stream with PyAV frame by frame. And for each frame make some processing. All good, but sometimes processing takes longer than the time between frames and my processed video starts lagging behind real time. This gap is constantly growing.
Obviously it's because of the input buffer.

Expected behavior

I want to decrease buffer size to one frame.

Actual behavior

To synchronize with real time i need read all frames from buffer.

Investigation

try set flag:
container.flags |= av.container.Flags.NOBUFFER

try set low delay:
container.streams.video[0].codec_context.flags |= av.codec.context.Flags.LOW_DELAY

Research

I have done the following:

• Checked the PyAV documentation
• Searched on Google
• Searched on Stack Overflow
• Looked through old GitHub issues
• Asked on PyAV Gitter
• ... and waited 72 hours for a response.

[jlaine]

I don't think there is anything PyAV can do for you here: if you are unable to process all the frames in time, then.. don't?

You are going to have to decimate the number of frames you process if you want to keep up with realtime. Something like:

• a main loop which pulls out frames as fast as possible and puts them in a queue.Queue
• the Queue should have a maximum size, and before putting a new item, you check whether the queue is full. If so, drop the oldest item using get()
• you move your processing to a thread which consumes items from your queue

[zbyte]

under normal conditions, processing in real time. But when lags happens, i should skip all frames and take the most recent frame.
If i made one more buffer via Queue, i not solve problem.

[jlaine]

Re-read my message, I'm precisely saying that the Queue you add will allow you to drop the oldest frame.

FFmpeg's internal buffer is not the issue, the issue is you are not reading frames fast enough, causing them to accumulate. If data was dropped internally by FFmpeg I'm not even sure you would recover, as you'd end up with corrupt frames.

[Dian-Yi]

you can set ffmpeg analyzeduration to reduce latency when you using av.open open RTSP stream