Repository navigation
Migrate audiox examples - #4607
Conversation
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
hsliuustc0106
left a comment
There was a problem hiding this comment.
AudioX migration to standard examples looks correct.
|
@princepride Hi ZP, can you take a look at this PR? I made the migration based on your code. |
70d088b to
9e1e6da
Compare
|
@zzehli Hi, DCO failed |
9d4518e to
6a1db2e
Compare
|
@princepride One thing about this PR, there is currently no online_serving generic client for text_to_audio. Creating and moving existing model online examples to a new client (audiox and stable audio) would touch more files than the recurrent PR scope. So I only updated the existing audiox online_serving example folder to accomodate extra_body. I can create a subsequent PR to migrate text_to_audio online_serving examples to a single client. Let me know what you think. |
Yeah, you can create a subsequent PR |
Signed-off-by: zzehli <jaeli_ottawa@outlook.com>
Signed-off-by: zzehli <jaeli_ottawa@outlook.com>
Signed-off-by: zzehli <jaeli_ottawa@outlook.com>
6a1db2e to
ce4b4a9
Compare
Signed-off-by: zzehli <jaeli_ottawa@outlook.com> Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
Signed-off-by: zzehli <jaeli_ottawa@outlook.com>
[Feature] Migrate AudioX offline to standard task example + declare
model_extrasPurpose
Closes #4540
Migrates AudioX off its bespoke examples onto the standard task example + a
declarative
model_extrasregistration, following the reference migration in#4225 (BagelPipeline / SenseNovaU1Pipeline).
What changed:
vllm_omni/model_extras/audiox.pydeclaring the request/response paramcontract (
AUDIOX_EXTRA_BODY_PARAMS,AUDIOX_EXTRA_OUTPUT_PARAMS)."AudioXPipeline"invllm_omni/model_extras/registry.py, soAudioX knobs sent via
extra_body(online) /--extra-body(offline) now flowthrough the standard
apply_declared_extra_argspath intosampling_params.extra_argsinstead of being dropped.t2a/t2m/v2a/v2m/tv2a/tv2m) into the standardexamples/offline_inference/text_to_audio/text_to_audio.pyvia--task+--video+--extra-body. Stable Audio Open behavior is preserved (fallback pathwhen the model is not registered).
vocabularies onto a single
--audio-length(→ audio length for Stable Audio,seconds_totalfor AudioX) and a single--audio-start(→audio_start_in_s/seconds_start). Dropped the 6 AudioX-only pass-through flags(
--seconds-start,--seconds-total,--sigma-min,--sigma-max,--cfg-rescale,--reference-audio); sampler/reference knobs now arrive via--extra-bodyonly, matching the helios reference migration ([Feature]Migrate helios example #4569). AudioX runsdefault their output to
audiox_output.wav.examples/offline_inference/audiox/dir (folded intothe standard offline example). Kept
examples/online_serving/audiox/as adedicated online example, but switched its request contract from the old
top-level
extra_argsescape hatch to flattenedextra_bodykeys routedthrough
apply_declared_extra_args.tests/model_extras/test_model_extras.py; updated the online e2e test to sendsigma_*knobs through the now-declaredextra_bodypath.recipes/audiox/AudioX.mdto document theextra_bodyknobsand the standard example invocations.
The AudioX pipeline itself needs no behavioral change — it already reads every
knob from
extra_args(pipeline_audiox.py:824-918).Test Plan
Offline
Text-to-audio (
t2a):python examples/offline_inference/text_to_audio/text_to_audio.py \ --model zhangj1an/AudioX --task t2a \ --prompt "Fireworks burst twice, then a clock ticking." \ --output /tmp/audiox_t2a.wavText-to-music (
t2m):python examples/offline_inference/text_to_audio/text_to_audio.py \ --model zhangj1an/AudioX --task t2m \ --prompt "Uplifting ukulele tune for a travel vlog" \ --output audiox_t2m.wavText+video-to-audio (
tv2a, requires--video):python examples/offline_inference/text_to_audio/text_to_audio.py \ --model zhangj1an/AudioX --task tv2a \ --prompt "drum beating sound and human talking" \ --video https://zeyuet.github.io/AudioX/static/samples/V2M/1XeBotOFqHA.mp4 \ --output audiox_tv2a.wavOnline
Start the server in one session:
Calls the server to generate the audio file us the sme task:
Test Result
Wasn't able to upload the wav files directly here. Format not supported by gh. Here is the zip file for all 3 results from above:
audio-combined.zip