Repository navigation
Conversation
…threads The PyAV video backend set thread_type = "SLICE" without a thread_count, so FFmpeg auto-sized min(cpu_count + 1, 16) slice workers per open codec. Under serving concurrency this multiplies across media-pool slots and processes into far more decode threads than cores, and decode threads hang permanently in pthread_cond_wait inside avpriv_slicethread_execute (PyAV-Org/PyAV#615 has the identical stack). Expose num_ffmpeg_threads for the pyav backend through media_io_kwargs, mirroring the torchcodec option (0 = FFmpeg auto sizing), and default it to a bounded 4 so the stock serving configuration cannot oversubscribe. FIX vllm-project#53973 Signed-off-by: Lanze Liu <lanzetech@gmail.com>
liulanze
requested review from
DarkLight1337,
NickLucche,
shen-shanshan,
tjtanaa and
ywang96
as code owners
August 27, 2026 23:08
Contributor
|
Documentation preview: https://vllm--54118.org.readthedocs.build/en/54118/ |
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Member
|
Closing as we have removed pyav backend (#54231) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FIX #53973
The PyAV video backend sets
stream.thread_type = "SLICE"but never setsthread_count, so FFmpeg auto-sizes the slice-thread pool tomin(cpu_count + 1, 16)workers per open codec. Under serving concurrency this multiplies into far more decode threads than cores, and decode threads hang permanently inpthread_cond_waitinsideavpriv_slicethread_execute. Each hung decode pins a media-pool slot forever and throughput collapses until restart.This PR adds
num_ffmpeg_threadsfor thepyavbackend via--media-io-kwargs, mirroring the TorchCodec option, and defaults it to a bounded4so the stock serving configuration cannot oversubscribe.Measured single-stream cost (1080p H.264, 60 frames, 12-CPU host):
0(auto=13)421Test Plan
python -m pytest tests/multimodal/test_video.py -k "pyav or backend_kwargs_are_separated or rejects_options or lazy_imported or imports_only_selected or edit_list or validates_frame_recovery" -vtest_pyav_backend_bounds_decode_threads(new, parametrized2/0): asserts the codec context'sthread_count— exact bound forN>0, resolved auto (>=1) for0. Fails without the fix on any host (auto resolves tocpu_count + 1, never2).test_video_backend_kwargs_are_separated_from_sampling_kwargs: two new pyav rows — explicit override routing plus a pin on the bounded default.test_video_backend_rejects_options_for_another_decoder: retargeted toseek_modeon pyav; also pins thatopencvstill rejectsnum_ffmpeg_threads.test_pyav_backend_loads_frames: passesnum_ffmpeg_threads=2end-to-end throughload_bytes.Root-cause repro (before/after): 8-thread pool decoding 64 videos concurrently through the real
loader.load_bytes(..., backend="pyav")path, sampling/proc/<pid>/task(the issue's accounting), on av 14.2.0 = libavcodec 61.19.100 (the exact version the issue reports).Test Result
Unit tests all pass on Linux (python:3.12 container,
VLLM_USE_PRECOMPILED=1editable install at the base commit):25 passed, 59 deselected in 98.02s.ruff checkandruff format --checkclean.Root-cause repro (12 CPUs, av 14.2.0, 8 concurrent decodes, 1280x720x60 H.264):
thread_countunset)4)This PR was developed in an AI-assisted pair-programming workflow (Claude Code); the submitter reviewed every changed line and ran all tests.