Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make decode loop multithreaded #32

Open
Speykious opened this issue Jul 26, 2022 · 0 comments
Open

Make decode loop multithreaded #32

Speykious opened this issue Jul 26, 2022 · 0 comments
Labels
enhancement New feature or request priority:medium

Comments

@Speykious
Copy link
Owner

Currently, the VideoStreamDecoder's decode loop is single-threaded. That is, it runs on a single thread, and invoked events also run on that same thread.
This is an issue as depending on the use-case, processing a frame can take a long amount of time.

However, simply spawning a new thread inside the event handler is not safe due to how the low-level classes such as VideoStreamDecoder and FrameConverter are designed.

  • VideoStreamDecoder has a single destination frame, allocated once. It is guaranteed to be mutated at the next frame, which does not give enough time for a long computation in a thread to do its task without copying the frame data.
  • FrameConverter has a single conversion context and a single destination frame, both allocated once. As per above, it makes it impossible to use FrameConverter safely in a thread as the destination frames and the converting context will conflict.

The trivial solution to this would be to copy the frame data and hand it to a new thread at each successful iteration of the decode loop (when a new frame is available), and for the frame converter, simply create a new one each thread. However, That results in a lot of allocations, which really slow down the whole process and makes it almost not viable.

An alternative, as proposed by @lubagov in #23, would be to have a pool of buffers to reuse. That means pool of different Frames and pool of different conversion contexts to roll in for threads to use to their liking.
This solution is way less trivial to implement and does require some more thoughts, but would definitely benefit SeeShark in the long run as that will make it more than viable for use in production.

@Speykious Speykious added enhancement New feature or request priority:medium labels Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request priority:medium
Projects
None yet
Development

No branches or pull requests

1 participant