You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the VideoStreamDecoder's decode loop is single-threaded. That is, it runs on a single thread, and invoked events also run on that same thread.
This is an issue as depending on the use-case, processing a frame can take a long amount of time.
However, simply spawning a new thread inside the event handler is not safe due to how the low-level classes such as VideoStreamDecoder and FrameConverter are designed.
VideoStreamDecoder has a single destination frame, allocated once. It is guaranteed to be mutated at the next frame, which does not give enough time for a long computation in a thread to do its task without copying the frame data.
FrameConverter has a single conversion context and a single destination frame, both allocated once. As per above, it makes it impossible to use FrameConverter safely in a thread as the destination frames and the converting context will conflict.
The trivial solution to this would be to copy the frame data and hand it to a new thread at each successful iteration of the decode loop (when a new frame is available), and for the frame converter, simply create a new one each thread. However, That results in a lot of allocations, which really slow down the whole process and makes it almost not viable.
An alternative, as proposed by @lubagov in #23, would be to have a pool of buffers to reuse. That means pool of different Frames and pool of different conversion contexts to roll in for threads to use to their liking.
This solution is way less trivial to implement and does require some more thoughts, but would definitely benefit SeeShark in the long run as that will make it more than viable for use in production.
The text was updated successfully, but these errors were encountered:
Currently, the
VideoStreamDecoder
's decode loop is single-threaded. That is, it runs on a single thread, and invoked events also run on that same thread.This is an issue as depending on the use-case, processing a frame can take a long amount of time.
However, simply spawning a new thread inside the event handler is not safe due to how the low-level classes such as
VideoStreamDecoder
andFrameConverter
are designed.VideoStreamDecoder
has a single destination frame, allocated once. It is guaranteed to be mutated at the next frame, which does not give enough time for a long computation in a thread to do its task without copying the frame data.FrameConverter
has a single conversion context and a single destination frame, both allocated once. As per above, it makes it impossible to useFrameConverter
safely in a thread as the destination frames and the converting context will conflict.The trivial solution to this would be to copy the frame data and hand it to a new thread at each successful iteration of the decode loop (when a new frame is available), and for the frame converter, simply create a new one each thread. However, That results in a lot of allocations, which really slow down the whole process and makes it almost not viable.
An alternative, as proposed by @lubagov in #23, would be to have a pool of buffers to reuse. That means pool of different
Frame
s and pool of different conversion contexts to roll in for threads to use to their liking.This solution is way less trivial to implement and does require some more thoughts, but would definitely benefit SeeShark in the long run as that will make it more than viable for use in production.
The text was updated successfully, but these errors were encountered: