[Model] Add Audio8 TTS Preview 0.1B (FalconH1 hybrid DualAR) by NancyFyong · Pull Request #6593 · vllm-project/vllm-omni

NancyFyong · 2026-08-25T02:19:14Z

Adds Audio8/Audio8-TTS-Preview-0.1b, the second checkpoint of the arktts DualAR TTS family. Addresses #6495.

It shares model_type: "arktts", the 44.1 kHz codec (byte-identical codec.pth), the Fast AR, RAS sampling, the prompt format and the streaming pipeline with the 0.6b — but the Slow AR backbone is different:

0.6b Slow AR = Qwen2 (pure attention)
0.1b Slow AR = FalconH1 (Mamba2 + attention parallel hybrid, with muP multipliers)

Ownership and license — please read

[New Model]: Audio8/Audio8-TTS-Preview-0.1b (FalconH1 hybrid slow AR + DualAR fast branch) #6495 was filed by @linyueqian, and @BruceLoveDecimal later said I'm welcome to take it and continue. I built this from the 0.6b work; happy to hand the branch over, co-author, or close in favour of someone else's version.
The 0.1b ships under the Audio8 Community License v1.0, not Apache-2.0 like the 0.6b. Whether vllm-omni wants to carry a model under that license is a maintainer decision; this PR does not presume the answer.

Rebased onto the merged 0.6b

#6157 (the 0.6b) is now merged, so this is a single 0.1b commit rebased onto current main (no longer stacked). The diff below is the net 0.1b delta.

What's in it

Backbone-aware Audio8TTSConfig: slow_backbone="falcon_h1" builds a FalconH1Config (annotated with the DualAR fields the backbone itself lacks); the 0.6b Qwen2 path is untouched.
Audio8TTS01BSlowARForConditionalGeneration: a FalconH1 Slow AR with its own 4097-wide semantic head, loading the backbone through vLLM's own FalconH1 weight loader (so qkv/gate_up/mamba fusion is correct by construction).
The arktts pipeline is now a resolver keyed on slow_backbone; new deploy/audio8_tts_01b.yaml.
Shared codec, Fast AR, RAS sampling and prompt_utils reused unchanged — the prompt ids are byte-identical to the 0.1b reference processor.
Registry entry, shared TTS adapter stage key, supported-models row, benchmark entry.

Two integration points are specific to wrapping vLLM's FalconH1 as a sub-backbone; both showed up on the first real GPU run and are now fixed:

vLLM's FalconH1Model reads vllm_config.model_config.hf_config directly (unlike Qwen2Model, which calls get_text_config()), so the Slow AR hands it a config view whose hf_config is the FalconH1 text_config.
The Slow AR declares IsHybrid/HasInnerState and forwards the mamba state shape/dtype/copy classmethods to FalconH1ForCausalLM (through the same view), so vLLM sizes the Mamba2 state cache. This follows the nemotron_voicechat precedent cited in [New Model]: Audio8/Audio8-TTS-Preview-0.1b (FalconH1 hybrid slow AR + DualAR fast branch) #6495.

Testing

6 CPU unit tests (config for both variants, slow_backbone resolver routing, registry mapping, adapter detection, Fast AR weight remap) plus offline and online e2e (advanced_model, real weights).
Validated end to end on 1x H20 against current main: the offline e2e passes at advanced_model, and manual runs produce clean 44.1 kHz speech for English, Chinese, Japanese, a long English passage, and zero-shot voice cloning.

Measured on 1x H20 (warm, `enforce_eager` per the shipped deploy)

Single-request, text-only:

language	audio	RTF
English	5.4 s	0.83
Chinese	5.0 s	0.83
Japanese	5.0 s	0.82
English (long)	13.5 s	0.82
voice clone	4.4 s	1.53

Offline batch throughput (same prompt, default deploy max_num_seqs: 4):

concurrency	req/s	audio s / wall s	aggregate RTF
1	0.22	1.2	0.86
4	0.77	3.4	0.30
8	0.94	4.0	0.25

The 0.1b is slower per request than the 0.6b (which reaches ~0.19 RTF): it runs enforce_eager (full-model CUDA-graph capture over the per-step Mamba2 state advance isn't validated yet) and the Mamba2 selective-scan/causal-conv kernels carry launch overhead. Batching recovers most of it (RTF 0.86 → 0.25). Voice cloning is higher because it encodes the reference audio each request.

Audio samples (vLLM-Omni, 1x H20)

Offline output for five prompts (mono PCM_16, 44.1 kHz). WAVs and a drag-and-drop guide are on the fork release audio8-tts-01b-samples; inline players below are added by dragging each file onto its placeholder in the browser editor.

sample	text	audio
English	"Welcome to Audio8 TTS, a compact multilingual text to speech model."	5.4 s
Chinese	"欢迎使用 Audio8 语音合成，这是一个轻量的多语种模型。"	5.0 s
Japanese	"Audio8 音声合成へようこそ。これは軽量な多言語モデルです。"	5.0 s
English (long)	long passage	13.5 s
voice clone	reference = the English sample above (no third-party voice)	4.4 s

The 0.1b's official HuggingFace transformers reference code imports FalconHybridMambaAttentionDynamicCache, which current transformers has removed, so an A/B against the HF reference (as done for the 0.6b) could not be produced on this stack — the vllm-omni path runs because it builds on vLLM's own maintained FalconH1 rather than the checkpoint's remote code.

Known gaps

enforce_eager is on by default for Stage 0; a piecewise-graph Mamba run could close the per-request gap but hasn't been measured.
Voice-clone streaming and multi-language output were exercised manually, not in CI. CI marks are core_model (CPU) with e2e at advanced_model.

BruceLoveDecimal · 2026-08-25T08:30:25Z

@NancyFyong No worries—please feel free to claim the issue and continue with the 0.1B integration. Since you’ve already done substantial work on the 0.6B PR, it makes perfect sense for you to carry this through to 0.1B as well

hsliuustc0106 · 2026-08-26T01:17:47Z

This PR touches vllm_omni/model_executor/ tests/model_executor/ examples/online_serving/ vllm_omni/entrypoints/ benchmarks/tts/ (42 files). Based on CODEOWNERS coverage of the changed files, the most-related reviewers appear to be:

@linyueqian @gcanlin @tzhouam

Could one of you take a look when you get a chance? Thanks!

linyueqian

Ran GPU e2e on this at c9f03e5e4 (1x L20X 141GB, vLLM 0.27.0 / torch 2.13.0+cu129 / transformers 5.14.1, real Audio8/Audio8-TTS-Preview-0.1b weights). Thanks for putting it up as a draft with the e2e gap called out; that gap turns out to be the whole story, so here is what it looks like from a live run.

Since the PR ships no e2e test for the 0.1b, I wrote one mirroring tests/e2e/offline_inference/test_audio8_tts.py against audio8_tts_01b.yaml.

Blocking

1. The 0.1b Slow AR never constructs. `vllm_omni/model_executor/models/audio8_tts/audio8_tts_falcon_slow_ar.py:83`

Every boot dies at model init:

File "vllm_omni/model_executor/models/audio8_tts/audio8_tts_falcon_slow_ar.py", line 83, in __init__
  self.model = FalconH1Model(vllm_config=vllm_config, prefix=maybe_prefix(prefix, "model"))
File "vllm/model_executor/models/falcon_h1.py", line 425, in __init__
  self.vocab_size = config.vocab_size
AttributeError: 'Audio8TTSConfig' object has no attribute 'vocab_size'

FalconH1Model.__init__ reads vllm_config.model_config.hf_config directly (falcon_h1.py:418). Qwen2Model reads hf_config.get_text_config() (qwen2.py:343), which is why the 0.6b path works. So the backbone receives the outer Audio8TTSConfig, which takes vocab_size as a ctor argument and never assigns it.

This is the module's stated design assumption rather than a slip. The comment at audio8_tts_falcon_slow_ar.py:57 reads "For 0.1b, get_text_config() returns a FalconH1Config annotated with the DualAR fields the parent reads", and FalconH1Model is the one consumer that does not call it.

Smallest fix, matching what is already in tree: shallow-copy vllm_config and model_config, set hf_config / hf_text_config to self.text_config, and pass that to the backbone. See higgs_audio_v3/higgs_audio_v3_talker.py:152-158 and moss_tts/modeling_moss_tts_talker.py:820-826.

2. Hybrid-Mamba interfaces are not declared. `audio8_tts_falcon_slow_ar.py:44`

With finding 1 patched, startup dies one step later, in KV cache spec construction:

File "vllm/v1/worker/gpu_model_runner.py", line 7827, in get_kv_cache_spec
  if spec := attn_module.get_kv_cache_spec(self.vllm_config):
File "vllm/model_executor/layers/mamba/abstract.py", line 65, in get_kv_cache_spec
  assert mamba_block_size is not None
AssertionError

Audio8TTS01BSlowARForConditionalGeneration inherits only from the Qwen2 Slow AR, so it declares neither IsHybrid / HasInnerState nor get_mamba_state_shape_from_config / get_mamba_state_dtype_from_config. Measured on this head:

class	`is_hybrid`
`Audio8TTS01BSlowARForConditionalGeneration`	False
`FalconH1ForCausalLM`	True
`NemotronVoiceChatThinkerForConditionalGeneration`	True

ModelRegistry derives is_hybrid from getattr(model, "is_hybrid", False), and platforms/interface.py:644 gates _align_hybrid_block_size on it. That call is the only thing that sets cache_config.mamba_block_size, hence the assertion.

Both in-tree hybrid siblings declare the interfaces and delegate the state classmethods: nemotron_voicechat/nemotron_voicechat_thinker.py:135-172 and audex/audex_omni.py:287-310. One caveat when you add them: FalconH1ForCausalLM.get_mamba_state_shape_from_config also reads .hf_config directly (falcon_h1.py:555), so it needs the same sub-config shim as finding 1.

3. Stage-0 EngineCore dies during generation

With findings 1 and 2 patched locally using the in-tree pattern above, the model does load (343 weights, mamba page sizing succeeds, attention block size aligned to 208 tokens), and then:

Path	Result
Offline, single request, `advanced_model`	passed, 4.10 s, 44.1 kHz, codec decoded
Offline, 4 concurrent (`max_num_seqs: 4`, the shipped default)	stage-0 core dies, `OmniEngineDeadError`; reproduced twice, including under `CUDA_LAUNCH_BLOCKING=1`
Online `vllm serve --omni`, single `/v1/audio/speech`	HTTP 500 `EngineDeadError` after 4.9 s

The core process exits with no Python traceback and no kernel signal in dmesg. These two patches are mine, so the final shape of this one may differ once you fix 1 and 2 properly. Flagging it because it means the path needs more than config plumbing.

Test coverage

The 6 new CPU tests pass (43 across the whole Audio8 suite, matching the PR body), but they all assert metadata: config shapes, resolver routing, registry entry, adapter stage key, weight remap. None constructs Audio8TTS01BSlowARForConditionalGeneration, which is why both blockers survived a green suite.

tests/model_executor/models/audio8_tts/test_audio8_tts_01b.py:68 asserts get_text_config() returns a FalconH1Config. That pins the producer side of exactly the contract finding 1 shows the consumer does not honor. A single test that instantiates the model class under a dummy VllmConfig would have caught both.

There is also no e2e test for the 0.1b: tests/e2e/{offline_inference,online_serving}/test_audio8_tts.py stay hardcoded to audio8_tts.yaml and the 0.6b, so nothing in CI exercises this path even at advanced_model.

Suggestion

compute_logits (audio8_tts_falcon_slow_ar.py:135-145) scatters the compact [B, 4097] semantic head into a full [B, 69633] -inf tensor so the inherited _compact_semantic_logits can slice 4097 columns back out. The parent's own docstring at audio8_tts_slow_ar.py:283 calls avoiding that full-vocabulary tensor "the difference between a cheap and an expensive AR step". Hoisting a _semantic_logits() hook onto the parent would let the 0.1b return compact directly and skip the round trip.

Checked and fine

The released config.json does carry slow_backbone: "falcon_h1", so the resolver routes correctly. Engine log confirms Resolved architecture: Audio8TTS01BSlowARForConditionalGeneration. It also carries num_key_value_heads: 2, so the #6424 NaN-logits trap does not apply here.
muP multipliers reach the config correctly through kwargs.get() (embedding_multiplier 0.10888671875, lm_head_multiplier 0.078125).
enforce_eager: true on stage 0 is well justified for the Mamba2 state path.
The 0.6b findings I raised on #6157 are fixed in this stack: the tokenized helper moved to tests/helpers/client.py with no .request_output hop, and the concurrency test now uses 4 distinct prompts with assert_distinct_outputs.

Not verified

Audio correctness is still open. The single-request offline test passes duration and sample-rate assertions, but I never got a waveform out to transcribe because both the online path and a standalone generator hit finding 3. Voice cloning is untested. Whether semantic_output should be scaled by lm_head_multiplier (the code says the reference does not) is unchecked against modeling_arktts.py.

On ownership and the Audio8 Community License: both are maintainer calls, not review findings. No opinion from me on either beyond noting they are still open.

NancyFyong · 2026-09-10T10:14:18Z

Thanks for the live run — that was exactly the gap, and all three findings were on the money. They're fixed now and I re-ran the whole thing on current main.

1 & 2 (config view + hybrid interfaces). Fixed the way you pointed at. The Slow AR now inherits HasInnerState/IsHybrid, and a small module-level _falcon_config_view() shallow-copies vllm_config/model_config and swaps hf_config/hf_text_config to the FalconH1 text config. So FalconH1Model.__init__ finds vocab_size and the mamba-state classmethods get the mamba dims — that's your caveat about get_mamba_state_shape_from_config reading .hf_config directly, so those classmethods go through the same view rather than the raw config.

3 (core death at concurrency / online 500). Doesn't reproduce once 1 & 2 are done properly. Offline single + 4-concurrent (the shipped max_num_seqs: 4) and online single /v1/audio/speech + streaming all pass at advanced_model with real weights on 1x H20. The online path runs the ASR/HNR asserts too, so there's intelligible audio coming out now, including voice cloning — that answers the correctness bit you couldn't get to.

Tests. Added the two e2e files (offline + online, mirroring the one you wrote) so the path actually runs in CI at advanced_model, plus two CPU regressions aimed squarely at your "nothing constructs the class" point: one pins is_hybrid and the mamba classmethods, the other pins that the config view hands FalconH1 a config carrying vocab_size. Both are mutation-checked — drop the interface or revert the shim and they go red.

On the compute_logits suggestion: agreed, the full-vocab scatter does an extra [B, 69633] alloc per step that the parent's own docstring flags as the expensive path. I left it as-is to keep this diff additive, but I'm happy to hoist a _semantic_logits() hook onto the parent so 0.1b can return the compact tensor directly — say the word and I'll fold it in, otherwise I'll file it as a follow-up.

Rebased down to a single commit on current main now that the 0.6b has merged. Thanks again for digging in.

vllm-omni-review-bot · 2026-09-10T10:18:56Z

This PR appears to belong to: docs/design/module/model_integration.md, docs/design/module/ar_runtime.md.

Module owners: @gcanlin @tzhouam @fake0fan

Routing: @gcanlin via module of the changed files, semantic router; @tzhouam via module of the changed files, semantic router; @fake0fan via module of the changed files

@NancyFyong, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

lishunyang12 · 2026-09-15T17:52:45Z

+        )
+
+        # Fast AR: separate compilation context, same as the 0.6b Slow AR.
+        fast_ar_compilation = copy.copy(vllm_config.compilation_config)


Extract the shared Fast AR initialization into a helper used by both backbones.

Done — moved it to _init_fast_ar on the parent, both backbones call that now. Re-ran the 0.6b and 0.1b offline e2e to be safe, both still pass. Thanks!

Adds Audio8/Audio8-TTS-Preview-0.1b, a second checkpoint of the arktts DualAR TTS family. It shares model_type "arktts", the 44.1 kHz codec (byte-identical codec.pth), the Fast AR, RAS sampling, prompt format and streaming pipeline with the 0.6b, but replaces the Slow AR backbone: 0.6b Slow AR = Qwen2 (pure attention) 0.1b Slow AR = Falcon-H1 (Mamba2 + attention parallel hybrid, muP) Because both report model_type "arktts", the arktts pipeline is now a resolver that selects the Slow AR by the config's slow_backbone field, and the config builds a Falcon-H1 text_config (annotated with the DualAR fields) for the 0.1b while leaving the 0.6b Qwen2 path unchanged. The Slow AR loads its Falcon-H1 backbone through vLLM's own FalconH1 weight loader and reuses the shared codec / Fast AR / heads remap. Two integration points are specific to wrapping vLLM's Falcon-H1 as a sub-backbone (both surfaced by the first GPU run): - FalconH1Model reads vllm_config.model_config.hf_config directly (unlike Qwen2Model, which calls get_text_config()), so the Slow AR hands it a config view whose hf_config is the Falcon-H1 text_config. - The Slow AR declares IsHybrid/HasInnerState and forwards the mamba state-shape/dtype/copy classmethods to FalconH1ForCausalLM (through the same config view) so vLLM sizes the Mamba2 state cache. - config: backbone-aware Audio8TTSConfig (slow_backbone -> FalconH1Config) - model: Audio8TTS01BSlowARForConditionalGeneration (Falcon-H1 Slow AR) - pipeline: resolve_arktts_pipeline + AUDIO8_TTS_01B_PIPELINE - registry + shared tts adapter stage key + deploy/audio8_tts_01b.yaml - prompt_utils reused unchanged (byte-identical to the 0.1b reference) - 8 CPU tests (incl. hybrid-interface + config-view regressions) plus offline/online e2e (advanced_model, real weights) Validated end to end on 1x H20: offline e2e passes at advanced_model, and text-only synthesis runs at ~0.82 warm single-request RTF (44.1 kHz) with aggregate throughput up to ~0.94 req/s at concurrency 8. Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>

NancyFyong · 2026-09-23T05:29:32Z

@lishunyang12 @linyueqian could you PTAL when you have time?

vllm-omni-review-bot · 2026-09-23T08:34:37Z

Omni ReviewBot: superseded

The CI failure noted on 0f0fa0d4f4f8 refers to an earlier head; the pull request now points at 6b28d56c9d2e.

vllm-omni-review-bot · 2026-09-30T12:14:53Z

Omni ReviewBot: no human activity for 7 days

@NancyFyong this pull request has had no human commit, comment or review since 2026-09-23. Please confirm the current plan and next step. The author or a maintainer decides whether to change the PR state.

To keep it moving, any one of these is enough: push an update, reply to the open blocker, or post the current plan and timeline.

Reconcile main 27a6321 with Audio8 TTS 0.1B: retain the backbone resolver in the sorted registry and keep upstream model/platform documentation updates. Reject unrelated HF configs while preserving the existing no-config 0.6B default. Add production-registry routing and rejection regressions. Validation: core_model and cpu suite: 1031 passed, 1 skipped. Five advanced_model/tts/cuda e2e cases collected only; real-weight GPU inference and performance were not rerun. Registry mutation check fails as expected. All applicable pre-commit checks including mypy passed; actionlint skipped after proxy 403 on installation (no workflow changes). AI assistance: OpenAI coding assistant via Pi assisted conflict resolution and tests. Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>

vllm-omni-review-bot · 2026-10-04T20:02:26Z

Omni ReviewBot routing record

Assigned Strict on zcode (GLM-5.3-Flash) under experiment fleet-strict-cursor-grok46-zcode-glm53flash-5050-c5-z10-20261002.

vllm-omni-review-bot · 2026-10-07T05:41:54Z

Omni ReviewBot: superseded

The CI failure noted on a010ae32a675 refers to an earlier head; the pull request now points at 072597abf299.

vllm-omni-review-bot · 2026-10-10T05:04:02Z

Omni ReviewBot attempt record

Review attempt ended as failed (step 'review' (agent.review_diff): unhandled error: RuntimeError: zcode exited 1 without a result event: statusCode: undefined } Error: Turn execution failed (traceId: 8520feef-67ab-42d3-a11c-99733cc32f2d) — check zcode login and the CLI version; retrying strict/zcode/GLM-5.3-Flash in 120s (try ).

vllm-omni-review-bot · 2026-10-10T06:28:19Z

Omni ReviewBot attempt record

Review attempt ended as failed (step 'review' (agent.review_diff): unhandled error: RuntimeError: zcode exited 1 without a result event: statusCode: undefined } Error: Turn execution failed (traceId: 7acce813-c331-4d98-9d54-28ad42fe0661) — check zcode login and the CLI version; retrying strict/zcode/GLM-5.3-Flash in 600s (try ).

vllm-omni-review-bot · 2026-10-10T12:20:16Z

Omni ReviewBot attempt record

Review attempt ended as failed (step 'review' (agent.review_diff): unhandled error: RuntimeError: zcode exited 1 without a result event: statusCode: undefined } Error: Turn execution failed (traceId: 05f3aa1c-9d19-42d0-a0e6-2fc80e20a589) — check zcode login and the CLI version; falling back to direct/cursor/auto).

vllm-omni-review-bot

Omni ReviewBot review

Changes since the previous review

0 new inline finding(s); 0 finding(s) below.

CI at 072597abf299 (2026-10-10T12:20:20.782390+00:00): required check(s) blocking: buildkite/vllm-omni (missing).

Note: The assigned review arm strict/zcode/GLM-5.3-Flash could not complete this review, so it was produced by the fallback arm direct/cursor/auto. It is excluded from the routing experiment.

Full review analysis

PR description

This change adds the Audio8 TTS Preview 0.1B checkpoint on the existing arktts DualAR pipeline. The 0.6B Qwen2 Slow AR stays the default; slow_backbone="falcon_h1" builds a Falcon-H1 text config, selects Audio8TTS01BSlowARForConditionalGeneration, and loads audio8_tts_01b.yaml. Stage 0 wraps vLLM's Falcon-H1 model with a compact semantic head and the hybrid Mamba cache hooks, while the Fast AR, codec, prompt format, and speech adapter stay shared. Serving still exposes text-only synthesis and voice cloning at 44.1 kHz, with Stage 0 eager execution and prefix caching left off.

Change flow

flowchart LR
  cfg["[EXISTING] arktts config slow_backbone"]:::existing
  text["[CHANGED] Audio8TTSConfig Falcon text_config"]:::changed
  slow["[NEW] Audio8TTS01BSlowAR FalconH1"]:::new
  pipe["[CHANGED] resolve_arktts_pipeline"]:::changed
  yaml["[NEW] audio8_tts_01b.yaml"]:::new
  adapter["[CHANGED] Audio8TTSAdapter stage key"]:::changed
  codec["[EXISTING] shared Fast AR and codec"]:::existing
  cfg --> text --> slow
  cfg --> pipe
  pipe --> yaml
  pipe --> adapter
  slow --> codec
  classDef existing fill:#e5e7eb,stroke:#6b7280,color:#111827
  classDef changed fill:#fef3c7,stroke:#d97706,color:#451a03,stroke-width:2px
  classDef new fill:#dcfce7,stroke:#16a34a,color:#052e16,stroke-width:2px
  classDef removed fill:#fee2e2,stroke:#dc2626,color:#450a0a,stroke-width:2px

_{🤖 This review was generated by InferMatrix Copilot, an open-source repo-maintenance agent for PR review, CI debugging and issue triage. Try it on your own repo, and ⭐ star it if it helped!}

amy-why-3459

Reviewed commit 072597abf29990a4cb527c84798546f3eda2ea3d.

No actionable bugs were identified in this static review. This is not proof that the code is correct.

No actionable issues found in this static review.

Validation: Verified the exact head and inspected the complete diff against d5a3380, PR metadata, changed code and tests, pipeline resolution callers, inherited sampling and preprocessing, Fast AR, serving adapter, and codec transfer paths. Runtime/GPU tests were not run; no repository code was executed.

vllm-omni-review-bot · 2026-10-18T23:37:03Z

Omni ReviewBot: no human activity for 7 days

@NancyFyong this pull request has had no human commit, comment or review since 2026-10-11. Please confirm the current plan and next step. The author or a maintainer decides whether to change the PR state.

To keep it moving, any one of these is enough: push an update, reply to the open blocker, or post the current plan and timeline.

NancyFyong force-pushed the feat/audio8-tts-01b branch 2 times, most recently from 4e98734 to c9f03e5 Compare August 25, 2026 04:26

hsliuustc0106 added benchmark/profiler/metrics/logger Codes related to benchmarks, profiler, metrics and logger system new model add new model tts code related to tts models labels Aug 25, 2026

linyueqian reviewed Aug 27, 2026

View reviewed changes

NancyFyong force-pushed the feat/audio8-tts-01b branch 5 times, most recently from 2c44e21 to bd99fd6 Compare August 30, 2026 06:50

NancyFyong force-pushed the feat/audio8-tts-01b branch 3 times, most recently from f7c4efd to f090455 Compare September 10, 2026 07:02

NancyFyong marked this pull request as ready for review September 10, 2026 10:14

NancyFyong requested review from alex-jw-brooks and lishunyang12 as code owners September 10, 2026 10:14

NancyFyong requested review from Gaohan123, NickCao, Sy0307, ZeldaHuang, amy-why-3459, gcanlin, hsliuustc0106, princepride, tzhouam and yenuo26 as code owners September 10, 2026 10:14

NancyFyong requested a review from david6666666 as a code owner September 10, 2026 10:14

NancyFyong force-pushed the feat/audio8-tts-01b branch from 3df41d4 to d4e8c62 Compare September 13, 2026 09:21

hsliuustc0106 added the high priority high priority issue, needs to be done asap label Sep 13, 2026

lishunyang12 reviewed Sep 15, 2026

View reviewed changes

NancyFyong force-pushed the feat/audio8-tts-01b branch from d4e8c62 to 4691ed7 Compare September 16, 2026 02:03

NancyFyong added 2 commits September 18, 2026 18:48

Merge branch 'main' into feat/audio8-tts-01b

73a53b0

Merge branch 'main' into feat/audio8-tts-01b

0f0fa0d

NancyFyong added 2 commits September 23, 2026 16:46

Merge branch 'main' into feat/audio8-tts-01b

6b28d56

Merge branch 'main' into feat/audio8-tts-01b

9a581fc

NancyFyong added 2 commits October 5, 2026 10:17

Merge branch 'main' into feat/audio8-tts-01b

d9e37f8

Merge branch 'main' into feat/audio8-tts-01b

a010ae3

NancyFyong requested review from linyueqian and lishunyang12 October 7, 2026 02:01

Merge branch 'main' into feat/audio8-tts-01b

072597a

vllm-omni-review-bot reviewed Oct 10, 2026

View reviewed changes

amy-why-3459 reviewed Oct 11, 2026

View reviewed changes

Conversation

NancyFyong commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ownership and license — please read

Rebased onto the merged 0.6b

What's in it

Testing

Measured on 1x H20 (warm, enforce_eager per the shipped deploy)

Audio samples (vLLM-Omni, 1x H20)

Known gaps

Uh oh!

BruceLoveDecimal commented Aug 25, 2026

Uh oh!

hsliuustc0106 commented Aug 26, 2026

Uh oh!

linyueqian left a comment

Choose a reason for hiding this comment

Blocking

1. The 0.1b Slow AR never constructs. vllm_omni/model_executor/models/audio8_tts/audio8_tts_falcon_slow_ar.py:83

2. Hybrid-Mamba interfaces are not declared. audio8_tts_falcon_slow_ar.py:44

3. Stage-0 EngineCore dies during generation

Test coverage

Suggestion

Checked and fine

Not verified

Uh oh!

NancyFyong commented Sep 10, 2026

Uh oh!

vllm-omni-review-bot commented Sep 10, 2026

Uh oh!

lishunyang12 Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

NancyFyong Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

NancyFyong commented Sep 23, 2026

Uh oh!

vllm-omni-review-bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Omni ReviewBot: superseded

Uh oh!

vllm-omni-review-bot commented Sep 30, 2026

Omni ReviewBot: no human activity for 7 days

Uh oh!

vllm-omni-review-bot commented Oct 4, 2026

Uh oh!

vllm-omni-review-bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Omni ReviewBot: superseded

Uh oh!

vllm-omni-review-bot commented Oct 10, 2026

Uh oh!

vllm-omni-review-bot commented Oct 10, 2026

Uh oh!

vllm-omni-review-bot commented Oct 10, 2026

Uh oh!

vllm-omni-review-bot left a comment

Choose a reason for hiding this comment

Omni ReviewBot review

Changes since the previous review

PR description

Change flow

Uh oh!

amy-why-3459 left a comment

Choose a reason for hiding this comment

Uh oh!

vllm-omni-review-bot commented Oct 18, 2026

Omni ReviewBot: no human activity for 7 days

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

NancyFyong commented Aug 25, 2026 •

edited

Loading

Measured on 1x H20 (warm, `enforce_eager` per the shipped deploy)

1. The 0.1b Slow AR never constructs. `vllm_omni/model_executor/models/audio8_tts/audio8_tts_falcon_slow_ar.py:83`

2. Hybrid-Mamba interfaces are not declared. `audio8_tts_falcon_slow_ar.py:44`

vllm-omni-review-bot commented Sep 23, 2026 •

edited

Loading

vllm-omni-review-bot commented Oct 7, 2026 •

edited

Loading