Repository navigation
Conversation
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: Guan Luo <gluo@nvidia.com>
| ) | ||
| is_cfhw = video.shape[0] in (1, 3, 4) | ||
| is_fchw = video.shape[1] in (1, 3, 4) | ||
| if is_cfhw and is_fchw: |
There was a problem hiding this comment.
vLLM-Omni's existing channel-first convention is CFHW, so valid CFHW outputs with 3 or 4 frames now satisfy both dimension checks and are rejected. This turns normal short generated videos into failed responses; the modified test explicitly drops those frame counts.
🤖 AI Fix
Preserve the established CFHW interpretation for channel-first tensors, or obtain an explicit layout indicator before rejecting a tensor as ambiguous.
| return [None] * expected_count | ||
| if isinstance(audio, (np.ndarray, torch.Tensor)): | ||
| if audio.ndim > 1 and audio.shape[0] == expected_count: | ||
| if audio.ndim >= 3 and audio.shape[0] == expected_count: |
There was a problem hiding this comment.
A rank-2 (batch, samples) tensor is the standard representation for a batch of mono waveforms. For two generated videos with audio shaped (2, samples), this condition no longer splits the batch and the formatter returns a failed video response despite the audio batch matching the video batch.
🤖 AI Fix
Accept rank-2 tensors whose leading dimension equals expected_count as batched mono audio outputs and split them along axis zero.
| return sample_rate | ||
|
|
||
| return None | ||
| return DEFAULT_AUDIO_SAMPLE_RATE |
There was a problem hiding this comment.
The change removes the model-config sample-rate fallback and always labels/encodes metadata-less audio as 24 kHz. Models whose vocoder configuration specifies another rate, such as the previously covered 48-kHz configuration, will have their raw waveform muxed at the wrong rate, causing incorrect duration, pitch, and video/audio synchronization.
🤖 AI Fix
Retain the model-config-derived sample-rate fallback after checking emitted metadata, using 24 kHz only when neither source provides a valid rate.
| assert frames.min() == 0 | ||
| assert frames.max() == 255 | ||
|
|
||
| @pytest.mark.parametrize("shape", [(4, 3, 8, 10), (3, 4, 8, 10)]) |
There was a problem hiding this comment.
The (3, 4, 8, 10) parameter is equivalent to the retained (4, 3, 8, 10) case: both only prove that a 4-D tensor whose first two dimensions are both channel-sized hits the same is_cfhw and is_fchw rejection path. Either case would catch the previous silent CFHW preference, so the second parameter adds no distinct contract coverage.
🤖 AI Fix
Collapse the shape parametrization to a single representative ambiguous shape, for example [(4, 3, 8, 10)].
033769b to
8e27b85
Compare
|
Superseded: this universal hardening commit has been folded into #13707 so the base video/audio feature is internally complete. |
Summary
Validation
pre-commit run --filespassed for all files in this PR.sha256:5b820b31f5a4b9b14eed29952623787756a38b283e2e73e35b82f96c6af86f5a.Stack
Review from bottom to top: