Skip to content

[Security] Bound Dots3 NOTE video audio decode duration and bytes - #56304

Open
jperezdealgaba wants to merge 1 commit into
vllm-project:mainfrom
jperezdealgaba:fix/dots3-note-video-audio-decode-bounds
Open

jperezdealgaba wants to merge 1 commit into
vllm-project:mainfrom
jperezdealgaba:fix/dots3-note-video-audio-decode-bounds

Conversation

@jperezdealgaba

Copy link
Copy Markdown
Collaborator

Summary

The Dots3 NOTE video preprocessor decoded a request-supplied video's audio track with an unbounded get_all_samples() call, so a small container with a long audio stream could expand into gigabytes of PCM during multimodal preprocessing. This routes that path through the shared torchcodec audio loader, which already enforces the server duration and decoded-byte limits.

Changes

  • vllm/models/dots3_note/common/video.py: decode video audio via load_audio_torchcodec with VLLM_MAX_AUDIO_DECODE_DURATION_S and VLLM_MAX_AUDIO_DECODE_BYTES. Duration and byte-limit errors fail the request; missing or corrupt audio tracks still skip audio.
  • tests/models/dots3_note/test_video_audio_decode.py: unit coverage for forwarded limits, over-duration and over-bytes rejection, missing-audio skip, preprocess propagation, and audio_cap=0 not decoding.

Codepath coverage

  • All HTTP and offline entry points that feed Dots3 NOTE video bytes (/v1/chat/completions, batch, /invocations, gRPC generate, LLM.generate) go through _decode_audio. That sink is now bounded.
  • Direct input_audio / audio_url still use AudioMediaIO, which already passes the same limits.
  • NanoNemotronVL video-audio extraction already passes the duration limit and is unchanged.
  • use_audio_in_video in chat parsing fetches audio through MediaConnector / AudioMediaIO and is unchanged.

Duplicate-work check

Searched open, draft, and recently merged PRs for dots3_note, _decode_audio, get_all_samples, and load_audio_torchcodec. Hits such as #53610 (processor refactor), #51894 (channel-count guard on the shared loader), and #56056 (PyAV packed-frame conversion) do not bound this sink.

Tests

  • test_decode_audio_forwards_duration_and_byte_limits
  • test_decode_audio_rejects_over_duration
  • test_decode_audio_rejects_over_decode_bytes
  • test_decode_audio_skips_missing_audio_track
  • test_preprocess_rejects_over_duration_audio
  • test_preprocess_skips_audio_decode_when_audio_cap_is_zero
.venv/bin/python -m pytest tests/models/dots3_note/test_video_audio_decode.py -v

6 passed.

.venv/bin/pre-commit run --files vllm/models/dots3_note/common/video.py tests/models/dots3_note/test_video_audio_decode.py passed.

AI assistance

This PR was developed with AI assistance.

Made with Cursor

Co-authored-by: Cursor Agent
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant