Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/.nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ nav:
- design/feature/hsdp.md
- design/feature/cache_dit.md
- design/feature/teacache.md
- design/feature/async_chunk_design.md
- design/feature/async_chunk.md
- design/feature/vae_parallel.md
- design/feature/diffusion_step_execution.md
- Module Design:
Expand Down
17 changes: 8 additions & 9 deletions docs/contributing/model/adding_omni_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,18 +408,17 @@ Understanding the data structures is crucial for implementing stage transitions:

**Input to your function:**
- `stage_list[source_stage_id].engine_outputs`: List of `EngineCoreOutput` objects
- Each contains `outputs`: List of `RequestOutput` objects
- Each `RequestOutput` has:
- `token_ids`: Generated token IDs
- `multimodal_output`: Dict with keys like `"code_predictor_codes"`, etc.
- These are the hidden states or intermediate outputs from the model's forward pass
- `prompt_token_ids`: Original prompt token IDs
- - Each contains `outputs`: List of `RequestOutput` objects
- Each `RequestOutput` has:
- - - `token_ids`: Generated token IDs
- `multimodal_output`: Dict with keys like `"code_predictor_codes"`, etc.These are the hidden states or intermediate outputs from the model's forward pass
- `prompt_token_ids`: Original prompt token IDs

**Output from your function:**
- Must return `list[OmniTokensPrompt]` where each `OmniTokensPrompt` contains:
- `prompt_token_ids`: List[int] - Token IDs for the next stage
- `additional_information`: Dict[str, Any] - Optional metadata (e.g., embeddings, hidden states)
- `multi_modal_data`: Optional multimodal data if needed
- - `prompt_token_ids`: List[int] - Token IDs for the next stage
- `additional_information`: Dict[str, Any] - Optional metadata (e.g., embeddings, hidden states)
- `multi_modal_data`: Optional multimodal data if needed

### How Model Outputs Are Stored

Expand Down
4 changes: 2 additions & 2 deletions docs/contributing/model/adding_tts_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ and can be placed on different devices. Qwen3-TTS has two stages:

Each stage is a separate model class configured independently via YAML. The two stages
are connected by the `async_chunk` framework, which enables inter-stage streaming for
low first-packet latency (see [Async Chunk Design](../../design/feature/async_chunk_design.md)).
low first-packet latency (see [Async Chunk Design](../../design/feature/async_chunk.md)).

### Without async_chunk (batch mode)

Expand Down Expand Up @@ -591,5 +591,5 @@ Adding a TTS model to vLLM-Omni involves:
For more information, see:

- [Architecture Overview](../../design/architecture_overview.md)
- [Async Chunk Design](../../design/feature/async_chunk_design.md)
- [Async Chunk Design](../../design/feature/async_chunk.md)
- [Stage Configuration Guide](../../configuration/stage_configs.md)
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Async Chunk Design
# Async Chunk

## Table of Contents

Expand Down Expand Up @@ -88,8 +88,9 @@ The following diagram illustrates the **Async Chunk Architecture** for multi-sta
</p>

**Diagram Legend:**

| Step | Stage Type | Description |
|:------:|:-----------:|:------------|
|------|-----------|------------|
| `prefill` | Initialization | Context processing, KV cache initialization |
| `decode` | Autoregressive | Token-by-token generation in AR stages |
| `codes` | Audio Encoding | RVQ codec codes from Talker stage |
Expand Down
Binary file modified docs/source/architecture/async-chunk-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading