Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 0 additions & 40 deletions examples/offline_inference/audiox/README.md

This file was deleted.

110 changes: 0 additions & 110 deletions examples/offline_inference/audiox/end2end.py

This file was deleted.

41 changes: 40 additions & 1 deletion examples/offline_inference/text_to_audio/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Text-To-Audio

A unified script for text/video-to-audio generation. Supported models:

| Model | Tasks | Notes |
|-------|-------|-------|
| `stabilityai/stable-audio-open-1.0` | text-to-audio | gated; uses `--audio-length` |
| `zhangj1an/AudioX` | `t2a` / `t2m` / `v2a` / `v2m` / `tv2a` / `tv2m` | pass `--task`; video tasks need `--video` |

The `stabilityai/stable-audio-open-1.0` pipeline generates audio from text prompts.

## Prerequisites
Expand Down Expand Up @@ -43,13 +50,45 @@ python text_to_audio.py \
--output stable_audio_output.wav
```

### AudioX

AudioX supports six tasks. Sampler and reference knobs (declared in
`vllm_omni/model_extras/audiox.py`) are passed via the generic `--extra-body`
JSON flag, routed into sampling `extra_args`.

Text tasks (`t2a` / `t2m`):

```bash
python text_to_audio.py \
--model zhangj1an/AudioX --task t2a \
--prompt "Fireworks burst twice, followed by a clock ticking." \
--num-inference-steps 250 --guidance-scale 6.0 --audio-length 10.0 --seed 42 \
--extra-body '{"sigma_min": 0.03, "sigma_max": 1000.0}' \
--output t2a.wav
```

Video-conditioned tasks (`v2a` / `v2m` / `tv2a` / `tv2m`) require `--video`:

```bash
python text_to_audio.py \
--model zhangj1an/AudioX --task tv2a \
--prompt "drum beating sound and human talking" \
--video https://zeyuet.github.io/AudioX/static/samples/V2M/1XeBotOFqHA.mp4 \
--num-inference-steps 250 --guidance-scale 6.0 --audio-length 10.0 \
--output tv2a.wav
```

Key arguments:

- `--prompt`: text description (string).
- `--task`: [AudioX] one of `t2a`/`t2m`/`v2a`/`v2m`/`tv2a`/`tv2m`.
- `--video`: [AudioX `v2*`/`tv2*`] video file/URL for conditioning (→ `video_path`).
- `--audio-start`: audio start offset in seconds (→ `audio_start_in_s` for Stable Audio, `seconds_start` for AudioX).
- `--audio-length`: audio duration in seconds (audio length for Stable Audio, `seconds_total` for AudioX).
- `--extra-body`: JSON dict of model-specific knobs (declared in `vllm_omni/model_extras/`), merged into sampling `extra_args`. For AudioX, sampler/reference knobs go here, e.g. `'{"sigma_min": 0.03, "sigma_max": 1000.0, "cfg_rescale": 0.0, "audio_path": "ref.wav"}'`.
- `--negative-prompt`: negative prompt for classifier-free guidance.
- `--seed`: integer seed for deterministic generation.
- `--guidance-scale`: classifier-free guidance scale.
- `--audio-length`: audio duration in seconds.
- `--num-inference-steps`: diffusion sampling steps.(more steps = higher quality, slower).
- `--use-hsdp`: enable HSDP weight sharding for the Stable Audio DiT.
- `--hsdp-shard-size`: number of GPUs used for HSDP sharding.
Expand Down
Loading
Loading