Add LLaVA audio (sound) model support - #4402

Merged

ericharper merged 24 commits into

NVIDIA:mainfrom

cuichenx:llava-model-audio

May 12, 2026

Commits on Apr 21, 2026

Add LLaVA audio (sound) model support, FastConformer encoder, and CP-aware vision plumbing for RADIO

committed
57e6c9f
Address review comments: parameterize align_corners and FP8 padding recipe
cuichenx
committed
8ce38ba

Commits on Apr 22, 2026

remove HF model
cuichenx
committed
933b330
keep non-fastconformer HF model stubs and vision hf:// support
cuichenx
committed
eb8b092
revert unintended
cuichenx
authored
4f4ec4c
revert
cuichenx
authored
33a7e20
revert extra state mute
cuichenx
authored
86858c3
revert fp8_pad_hook to use fp8_recipe-aware padding
cuichenx
committed
e8bdacb
revert unrelated TE import refactor in llava_model.py
cuichenx
committed
a22f4ba
add unit tests for multimodal CP helpers and RADIO state-dict hooks
cuichenx
committed
201eb75

Commits on May 1, 2026

Commits on May 2, 2026

fix(llava): restore packed dynamic-res image path and audio-only guard
cuichenx
committed
6091e22

Commits on May 4, 2026

fix(cp): split _split_num_frames in frame space; clean media-boundary splits
cuichenx
committed
03d8a57

Commits on May 5, 2026

test: cover mixed-length video last-frame duplication in temporal grouping

cuichenx
and
claude
committed
f75d38c