[tzhouam]

Purpose

Reduce repeated final-frame processing in LingBot World's eight-rank VAE and allow post-attention graph consumers to borrow outputs safely.

• Add three opt-in pixel paths: root_float, root_uint8_allgather, and root_uint8_p2p. Only group rank zero assembles and returns CPU RGB; every rank still performs decoder, halo, and temporal-cache work. Pixel conversion preserves the existing precision and rounding order.
• Add two-slot post-attention graphs with fixed projection inputs and consumer completion events. Parameter-owner, callable-code, and LayerNorm-backend changes invalidate graph reuse. Execution failures retain active owners and require worker restart.
• Add CPU contract tests and document independent switch validation.

LingBot-specific output helpers live in vllm_omni/diffusion/models/lingbot_world/, alongside their model integration. Package exports load lazily so shared VAE/communication hooks can import the helpers without initializing the full pipeline. Failure handling retains in-flight owners and callers propagate the original exception; this PR adds no environment variables or file-reporting protocol. Worker teardown/relaunch remains the deployment's responsibility.

Review status (2026-10-14): ready for code review. Real SP8 switch-by-switch parity, leased-output consumption/slot reuse, and session close/reopen validation remain outstanding, as suggested in the component-validation comment.

Both features are disabled by default: lingbot_npu_pixel_output_mode: native and lingbot_npu_leased_post_attention: false. NPU execution requires the attention enablement in #8112.

Current performance (2026-10-13): the reconstructed eight-die stack measures 924.6–928.3 ms mean / 939.5–944.0 ms P99 per chunk across three independent screening runs. The mean≤900 ms target and final warm40+600 acceptance are still outstanding. These are full-stack measurements, not an isolated speedup for this PR; see the results below.

Test Plan

Run the CPU contract tests without model weights or NPU initialization:

TORCH_DEVICE_BACKEND_AUTOLOAD=0 OMP_NUM_THREADS=1 python -m pytest \
  --noconftest -c pyproject.toml -o addopts='' \
  tests/diffusion/models/lingbot_world/test_lingbot_pixel_output.py \
  tests/diffusion/models/lingbot_world/test_lingbot_leased_post_attention.py \
  tests/diffusion/models/lingbot_world/test_lingbot_world_imports.py -q

With the repository's normal test dependencies, run the same files without the isolated-config options and with -m "core_model and diffusion and cpu" --run-level=core_model.

CPU validation environment: Python 3.12, torch 2.10.0+cpu, pytest 8.3.2, and vLLM 0.31.0. No checkpoint or NPU execution was used for these checks. Current validation commit: 5ad6f64ce9b20bca966a54b63e88cc033fdb14a5; main baseline: ebe2c18271db4fdd12221fbb4def6061e14a9379.

Before enabling either feature, validate each switch independently on eight ranks, then test combinations: consecutive latent/RGB parity, actual projection and borrowed-output consumption, frame counts/PTS, window sliding, cancellation, disconnect, close/reopen, and prompt updates.

Test Result

• 46 isolated CPU output/import contract tests passed, using the command above. Coverage includes exact pixel rounding, P2P ordering, temporal counters, graph-slot ownership, consumer events, quarantine lifetime after a failed collective, and lazy package imports/exports.
• 414 related CPU tests passed, 8 deselected across LingBot model tests, environment inventory, spatially sharded Wan decoding, and streaming decode. The selection excludes compile/accelerator cases and the known CPU-only UnspecifiedOmniPlatform.synchronize profiler test; the latter also fails on the unchanged baseline.
• All applicable changed-file pre-commit hooks and git diff --check passed.
• Eight-rank model, lifecycle, and performance acceptance remains pending before production enablement. These CPU checks do not establish an NPU speedup or the mean≤900 ms / P99≤950 ms target.

Current full-stack performance — 2026-10-13

Four physical 910C NPUs / eight dies; SP8/TP1/ring1, 832×480, four DMD steps, full history, BF16 attention, FFN W8A8 NZ, and 12 FPS encoding. Each fresh service completed 40 warmup chunks followed by 161 measured chunks. Results below use client media inter-arrival over chunks 11–159 (149 samples/run), with linearly interpolated P99.

Run	Mean (ms/chunk)	P99 (ms/chunk)	Maximum (ms/chunk)
A1	928.3	942.1	946.3
A2	927.1	944.0	946.8
A3	924.6	939.5	944.5

Current reproducible performance is 924.6–928.3 ms mean and 939.5–944.0 ms P99 per chunk. With 12 frames per steady-state chunk, the P99 intervals correspond to 12.71–12.77 FPS of generation supply; this is not measured browser presentation FPS.

All three screening runs satisfy P99≤950 ms, with zero intervals over 1000 ms in the measured window; mean remains 24.6–28.3 ms above the 900 ms target. Each measured request produced 1,929 frames with complete, ordered chunks.

These measurements belong to the reconstructed full stack at source f48c20b31a25ac3d2e42a471c0cc2248c9456c3c, including the recv/VAE/pixel paths and Stage A cache. QK-combined, cross combined guard, and async metadata were disabled. They do not isolate this PR's contribution or validate its leased graph path. Three independent warm40+600 runs and final playback/lifecycle acceptance remain pending. Full paired-screen report and configuration details.

W8A8 benchmark dependency: neither this PR nor #8112 includes the experimental LingBot W8A8 adapter. The measurements above additionally use the backed-up quantization implementation and its post-load pipeline integration. The adapter performs per-output-channel weight quantization and NZ preparation, then calls native torch_npu.npu_dynamic_quant and torch_npu.npu_quant_matmul. Its benchmark settings are lingbot_npu_quant: w8a8, lingbot_npu_quant_scope: ffn, and lingbot_npu_quant_weight_format: nz (80 FFN linears; attention stays BF16). Stacking #8112 and #8610 alone does not enable this W8A8 path or reproduce the complete benchmark stack.

Separate optimization experiments (outside this PR)

The wider optimization experiments are backed up separately: recovery performance report, QK/CANN correctness and speed screening, and warm-handle source and device screening (192 real inputs passed correctness; three-round speed gate failed, candidate remains disabled). The follow-up private-retention native experiment also passed fresh 192-input correctness and bounded allocation-pressure checks, but failed the three-round worst-case speed gate; it remains disabled. Those results apply to separate experimental stacks, not this PR head.

Additional separate experiment: the Stage B original-modulation cache passed all40-layer logical-output checks on eight ranks but its first complete-call screen regressed 4.46×; it stays disabled and is excluded from this PR. The same backup includes the current eight-rank baseline trace analysis, with explicit reconstructed-boundary and profiling limitations.

The independent per-layer C++ Stage B guard preserves the original check points and passed logical correctness in two eight-rank rounds. Its complete-region improvements were 10.25% and 5.28%; round two missed the required 10%, so round three and model testing were stopped. It remains disabled, and current full-stack performance above is unchanged.

The separate eight-rank host diagnosis records actual 40-layer call counts and retains the Python 3.12 compatibility fix and independent evidence. Its instrumented host intervals are diagnostic only; a different typed host binding is proposed for fresh screening, with all per-layer checks preserved. No new model speedup is claimed.

AI assistance: Codex implemented the invalidation fixes, standardized tests and documentation, ran CPU/static checks, and updated this description.

[Dong1017]

Additional component validation on 050f2de with the #8112 prerequisite at 7d7d994, using an Ascend 910B1:

• Nine synthetic pixel-conversion cases covering BF16/FP16/FP32 and different frame counts matched the native RGB bytes exactly.
• The production _NPURuntime.capture wrapper captured and replayed real BF16 Linear + LayerNorm operations. After changing the inputs, replay outputs matched eager execution exactly.

Static review found no confirmed blocking defect. These component checks do not yet cover the actual LingBot projection provider, leased-output consumption and slot reuse, or eight-rank pixel communication.

Before marking this ready, I suggest validating the real SP8 path with #8112: compare native mode against each switch independently, check consecutive chunks and RGB parity, and verify session close/reopen. The target 910C end-to-end performance measurements remain a separate acceptance step.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/diffusion/index.md.

Module owners: @david6666666 @Isotr0py @princepride

Routing: @david6666666 via module of the changed files, module named in the PR description, CODEOWNERS; @Isotr0py via module of the changed files, module named in the PR description; @princepride via module of the changed files, module named in the PR description

@tzhouam, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Resolved as of 5ad6f64ce9b2: the high-priority or low-quality signal noted on an earlier commit no longer applies.

[vllm-omni-review-bot]

Omni ReviewBot: superseded

The CI failure noted on 852e00cc87b2 refers to an earlier head; the pull request now points at 2c93c3ee8c2d.

[FayeSpica]

Just curious, how is this failure-reporting mechanism used, and where is the supervisor implemented?

[tzhouam]

You're right—the wording implied that a supervisor was included. This PR provides only the reporting side of an optional external-launcher protocol: it retains active tensor owners, optionally writes an atomic failure receipt, and callers propagate the original exception. With neither channel variable set, owners are still retained but no receipt is written. restart_scope: all8 is advisory metadata; the module does not stop or restart workers.

Clarified in 194ee20: the module comments, recipe, and PR description now explicitly identify receipt consumption, process-identity checks, and cleanup/relaunch of the eight-worker launch as deployment responsibilities. The legacy ROOT_ prefix does not require OS root privileges. End-to-end supervision is not supplied by this PR.

Validation: 43 pixel/leased-graph CPU contract tests and changed-file pre-commit checks passed. Execution behavior is unchanged apart from the clarified nonce-validation error wording.

[FayeSpica]

Adds opt-in LingBot Ascend SP8 output optimizations, disabled by default with limited impact.

LGTM.

[gcanlin]

Left some code style comments.

[gcanlin]

Could we move this model-specific layer to the modeling module?

[gcanlin]

Ditto, I think it's better not to add env.

[vllm-omni-review-bot]

Omni ReviewBot: superseded

The CI failure noted on 5ad6f64ce9b2 refers to an earlier head; the pull request now points at fa6f9d64f774.