[lsyyysky]

Purpose

Refs #7075.

Add opt-in automatic prefix caching for MammothModa2:

• Add mammoth_moda2_prefix_cache.yaml for AR → DiT generation.
• Add a CPU unit test for cached hidden-state reconstruction and AR-to-DiT token alignment.
• Add a live-engine E2E test covering actual cache accounting, varied prompt lengths, mixed requests, and DiT conditioning inputs.

Prefix caching remains opt-in because BF16 attention may produce numerical differences between full prefill and cache-hit tail execution.

Test Plan

CPU unit test

pytest -q -o addopts='' \
  tests/model_executor/stage_input_processors/test_mammoth_moda2.py \
  -m "core_model and cpu" \
  --run-level=core_model

Live-engine E2E test

MAMMOTH_MODA2_TEST_MODEL=/root/models/MammothModa2-Preview \
pytest -s -v -o addopts='' \
  tests/e2e/offline_inference/test_mammoth_moda2_prefix_cache.py \
  --run-level=full_model

The E2E test:

• Starts a real Omni engine with mammoth_moda2_prefix_cache.yaml.
• Repeats requests in the same engine to exercise actual miss/hit paths.
• Covers prompt lengths of 28, 32, and 37 tokens with block size 16.
• Runs a mixed batch containing one warmed request and one synthetic cold
  token request whose first cache block differs.
• Strictly verifies warmed.cached_tokens > 0 and
  cold.cached_tokens == 0.
• Observes the real ar2diffusion result without replacing it.
• Checks per-request cache accounting and AR-to-DiT conditioning.

Performance protocol

All performance measurements use:

• Model: MammothModa2-Preview
• Hardware: NVIDIA A800-SXM4-80GB
• Precision: BF16
• Execution mode: eager
• Image grid: 256 × 256
• Generated AR tokens: 273
• Seed: 42
• Warmup: 1 request
• Measured requests: 10 per configuration

Configurations:

• A: Prefix caching disabled.
• B1: Prefix caching enabled, followed by a cache reset before every measured request. Every sample must report cached_tokens=0.
• B2: Prefix caching enabled with the target prefix warmed before measurement. Every sample must report a genuine cache hit.

vLLM Version: 0.28.0
vLLM-Omni Commit: 86546ef

Test Results

Unit and live-engine tests

• CPU unit test: passed.
• Live-engine E2E test with strict mixed hit/miss accounting: 1 passed in 154.16s.
• All pre-commit hooks passed.

For the 28-, 32-, and 37-token E2E prompts, cache-hit requests reuse 16, 16, and 32 tokens respectively.

The test verifies:

• full_hidden_states.shape[0] == len(full_token_ids).
• The cached prefix hidden states are exactly equal between miss and hit.
• Recomputed prompt-tail cosine similarity is at least 0.999.
• Text-conditioning cosine similarity is at least 0.999.
• Image-conditioning cosine similarity is at least 0.8.
• Mixed requests retain independent cache accounting and reconstruction.

A/B1/B2 Performance

Long-prompt workload

The long workload uses a synthetic 5,627-token prompt to measure prefix-cache scaling. B1 creates 5,616 cache tokens on every request, while B2 reuses 5,616 tokens and computes only the remaining 11 prompt tokens.

A — cache disabled:

• Mean TTFT: 383.02 ms
• TTFT p50: 382.04 ms
• TTFT p95: 388.32 ms
• Mean E2E latency: 5986.45 ms
• E2E p50: 5991.42 ms
• E2E p95: 6037.59 ms
• Mean throughput: 45.60 tokens/s
• Peak GPU memory: 40408.8 MiB

B1 — cache enabled, verified miss:

• Mean TTFT: 392.92 ms
• TTFT p50: 392.64 ms
• TTFT p95: 398.10 ms
• Mean E2E latency: 5997.09 ms
• E2E p50: 5989.83 ms
• E2E p95: 6082.15 ms
• Mean throughput: 45.53 tokens/s
• Peak GPU memory: 40410.8 MiB
• All samples: cached_tokens=0, cache_creation_tokens=5616

B2 — cache enabled, verified hit:

• Mean TTFT: 60.72 ms
• TTFT p50: 59.88 ms
• TTFT p95: 64.58 ms
• Mean E2E latency: 5634.94 ms
• E2E p50: 5631.33 ms
• E2E p95: 5715.54 ms
• Mean throughput: 48.45 tokens/s
• Peak GPU memory: 40410.8 MiB
• All samples: cached_tokens=5616, cache_creation_tokens=0

B2 compared with B1:

• Mean TTFT improves by 84.5%.
• Mean E2E latency improves by 6.0%.
• Throughput improves by approximately 6.4%.
• Computed prompt tokens decrease from 5,627 to 11.
• Prefix-cache memory overhead is approximately 2 MiB.

Shorter-prompt behavior

With a 587-token prompt and the same 273-token output:

• B1 mean TTFT: 75.94 ms.
• B2 mean TTFT: 56.86 ms, a 25.1% improvement.
• B1 mean E2E latency: 5728.24 ms.
• B2 mean E2E latency: 5878.77 ms.

For this shorter prompt, the approximately 19 ms prefill saving is too small relative to the 273-token AR decode and normal run-to-run variation, so it does not produce a measurable E2E improvement.

The long-prompt result demonstrates that prefix caching improves E2E latency once prefill becomes a sufficiently large portion of the request.

Profiler Analysis

A diagnostic PyTorch Profiler comparison was collected separately using the 587-token AR-only workload. Profiler timings are not used as formal latency measurements.

Observed behavior:

• Scheduled prefill decreases from 587 tokens to 11 tokens.
• Profiled prefill CUDA time decreases from 42.11 ms to 31.09 ms.
• Total GEMM time remains nearly unchanged: 2.393 s versus 2.378 s.
• GPU busy time remains nearly unchanged: 3.064 s versus 3.049 s.
• D2H traffic increases from approximately 0.002 MiB to 1.938 MiB, adding approximately 1.4 ms for hidden-state cache handling.
• Decode kernel counts remain unchanged.

The trace confirms that prefix caching skips the expected prefill work. It does not optimize autoregressive decode, explaining why E2E benefit depends strongly on the ratio of prompt length to generated length.

The profiler workload is AR-only and therefore does not quantify DiT execution or AR-to-DiT H2D transfer.

Numerical Behavior

The cached prefix is reconstructed exactly. The newly computed BF16 tail is not guaranteed to be bitwise equivalent to full prefill because attention executes with a different query shape.

The following image is retained as an illustrative manual comparison and is not treated as multi-prompt or cross-seed quantitative evidence:

Reproducibility artifacts

• Benchmark harness
• Commands and methodology
• Raw per-request results

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/vllm_omni_config.md.

Module owners: @lishunyang12 @alex-jw-brooks

@lsyyysky, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[hsliuustc0106]

Nit: the miss/hit path here is exercised entirely through mocks (SimpleNamespace input batch / completion) with the cache and ar2dit called directly, so the engine-level contract on a real cache hit — num_computed_tokens accounting feeding a truncated prefill into ar2dit — isn't covered. Worth a follow-up that drives the hit path through the actual stage-input-processor wiring.

[hsliuustc0106]

The numerics evidence is currently n=1: one 34-token prompt, seed 42, one image, and "highly similar" as the qualitative bar. Since the whole reason this stays opt-in is that BF16 attention on a cache-hit tail isn't bitwise equivalent to full prefill, could we get validation over a real workload dataset instead of a single prompt? Specifically:

• A set of real prompts with varied lengths, including lengths that cross block boundaries (partial-block tails) and short prompts that don't fill one block.
• A batch with mixed hit/miss requests so the per-request cache merge is exercised concurrently, not just a single request.
• A quantitative similarity metric (e.g. LPIPS/CLIP score or pixel-diff stats between miss and hit generations) reported across the set, rather than eyeballing one pair.

That would give the opt-in default a real evidence base. The CPU unit test + config layering look correct otherwise.

[hsliuustc0106]

Reviewed head 13bd191268caf63ab95d7d0823a004cb74b74e56 for #7075 / v0.30.0. The new test plus shared prefix-cache and async-write suites passed 33 tests; the actual deploy-config loader preserved the base settings and enabled caching on AR.

[P2, existing coverage finding] Exercise a real cache hit through the engine and configured stage path. This follows up on the existing unresolved thread. At test lines 71–89, cached-token accounting and the hit marker are supplied manually, the cache helper is called directly, and fabricated completion objects are passed to ar2dit. This validates helper arithmetic but would not catch broken runner hit detection, reconstruction of skipped prefill states, or stage dispatch. Please add repeated requests in one live engine, assert a genuine cache hit, and compare token-aligned conditioning through the configured AR→DiT path.

Requested A/B evaluation and detailed profiling

Please provide a reproducible harness covering A: caching disabled, B1: caching enabled with a verified miss, and B2: caching enabled with a verified hit. Keep code/checkpoint revisions, hardware, runtime versions, placement, prompts, seeds, resolution, steps, CFG, and other features identical. Warm model/kernel execution separately from prefix-cache state so startup or compilation does not explain the result. Record cache state and cached/created token counts for every request.

Please include:

• A real prompt set spanning sub-block lengths, exact block boundaries, partial-block tails, longer prefixes, and mixed hit/miss batches. Cover AR→DiT and representative Preview/Dev AR-only text/image-understanding workloads claimed by the profiles.
• Equality checks for the reused prefix and explicit numerical tolerances for the recomputed BF16 tail, full hidden-state length/token alignment, and quantitative miss/hit image comparisons across prompts and seeds. Report the metric distribution and worst cases, not only one visually similar pair.
• Repeated unprofiled measurements with sample counts, warmup policy, variability, p50/p95 latency, throughput, and memory usage. Separate prefill/TTFT, decode, hidden-state reconstruction and transfer, DiT, and E2E timing.
• Representative baseline/miss/hit Nsight Systems or PyTorch Profiler traces with a written explanation of skipped prefill work, actual scheduled-token counts, attention-kernel changes, hidden-state cache writes and reads, D2H/H2D traffic, reconstruction/concatenation, host synchronization, and queueing. Quantify how much saved AR computation survives as E2E benefit and where overhead or numerical differences arise.
• Exact commands/configs and raw results or accessible trace artifacts so the comparison can be reproduced.

The current example invokes generation once and does not itself reproduce a warm cache-hit comparison. Please include the live repeated-request harness with the reported real-model evidence.

Please coordinate with #7102: both PRs add the same test file, and prefix caching makes _should_defer_full_payload_d2h() false. Preserve both test suites and separately validate/profile the combined configuration before claiming that caching and the transfer optimization compose.

[vllm-omni-review-bot]

Omni ReviewBot: three questions on the performance claim

@lsyyysky this PR reads as a performance or value claim:

• claim: - Mean TTFT improves by 84.5%.
• claim: - Mean E2E latency improves by 6.0%.
• claim: - Throughput improves by approximately 6.4%.

Before the full evidence checklist, three short questions:

1. Bottleneck — what is the current bottleneck, and which profile, trace or per-stage measurement shows it?
2. Value — what does the change buy the user or the system (latency, throughput, memory, cost), and at which workload?
3. A/B or ablation — is there a same-workload, same-head/config comparison that isolates each main claim on its own? For stacked optimizations, one number per item rather than a blended delta.

When you answer, the evidence that settles it is: base and head SHA, hardware, model, workload, warm-up and repeat count, mean or percentiles with their spread, and a correctness/quality-equivalence signal; an end-to-end claim also needs stage attribution.

[lsyyysky]

hi @hsliuustc0106 , Thanks for the review. The PR now includes:

• A live-engine E2E test covering genuine miss/hit paths, varied prompt lengths, strict mixed hit/miss batching, cache accounting, and AR-to-DiT conditioning.
• Reproducible A/B1/B2 benchmarks with raw results and profiler analysis.
• Verified long-prompt improvements: 84.5% lower TTFT and 6.0% lower AR latency for B2 versus B1.

I also tested the merged branch with #7102. The prefix-cache path passes, but #7102’s request-end payload path blocks before sending AR output to DiT, including with caching disabled. #7102’s flush lifecycle needs to be resolved first.

[hsliuustc0106]

Re-reviewed head b93836cae9db592cc12d5dc4a85e70e8b1e50baa for #7075 / v0.30.0.

The earlier mock-only integration finding is addressed. I independently ran the live-engine test: 1 passed in 274.29 s, plus 33 passing prefix-cache unit tests. The test executes real requests through the configured AR→DiT path, observes the actual ar2dit output, and verifies cache accounting, exact reused-prefix states, BF16-tail/conditioning tolerances, and mixed hit/miss accounting. The assertions cover the 28/32/37-token prompts without replacement of the cache implementation.

Two benchmark issues remain:

[P2] Accept genuine block-boundary hits. The B2 validator requires zero cache-creation tokens. A warmed 32-token prompt legitimately reports 16 cached and 16 created tokens, as the passing live test explicitly expects. An isolated call to the current validator rejects that valid sample with scenario b2 cache-state verification failed: [(16, 16)], and main raises before writing results. Derive expected hit/creation counts from the prompt and block size rather than equating every hit with zero creation.

[P2] Sample the physical GPU that actually runs the engine. MemoryMonitor takes a raw NVML index, defaulting to physical GPU 0, while the deploy config uses logical CUDA device 0. With a GPU allocation exposing physical GPU 2, the engine runs on 2 but the default monitor samples 0 and can report the wrong peak memory/hardware. Resolve the engine's device to its physical NVML handle/UUID and validate any explicit override. This does not invalidate the supplied results if the author's engine actually ran on physical GPU 0.

Validation environment: NVIDIA L20X, Python 3.12, torch 2.13.0, vLLM 0.28.0; MammothModa2-Preview checkpoint ef5a5e41dbf0de1ef6275586b7580f0d4248b4c6, offline, with the original test/config unchanged. The live run was correctness validation, not a new timing benchmark.

PYTHONDONTWRITEBYTECODE=1 PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest \
  -p pytest_mock -p no:cacheprovider -o addopts='' --run-level=full_model -s -v \
  tests/e2e/offline_inference/test_mammoth_moda2_prefix_cache.py

A/B and profiling assessment. The checked-in ten-request results support the reported 84.5% TTFT and 6.0% AR-only latency reductions for the synthetic 5,627-token workload. Recomputed AR latency mean ± sample SD: A 5986.45 ± 41.84 ms, B1 5997.09 ± 64.08 ms, B2 5634.94 ± 55.55 ms. The separate 587-token profile explains reduced prefill, nearly unchanged decode/GEMM, and increased D2H; it does not profile the long workload or full image-generation path.

Please retain the A/B1/B2 protocol and add quantitative image-quality comparisons across prompts/seeds, full transfer/DiT/E2E stage attribution, CPU hidden-state-cache memory, and accessible representative raw traces with commands and analysis of copies, synchronization and reconstruction. The mixed-batch test also currently checks accounting without comparing both members' conditioning to separate references; that is a useful next coverage extension.

Benchmark review from @alex-jw-brooks or @Bounty-hunter would help. Preserve both stage-input test suites when integrating #7102.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Resolved as of 47c992351179: the high-priority or low-quality signal noted on an earlier commit no longer applies.

[lsyyysky]

@hsliuustc0106 Thanks for the detailed re-review. The requested follow-up validation is now available.

Reviewer follow-up validation

The two benchmark P2 findings are addressed:

• B2 validation now accepts valid exact-block-boundary hits that recompute the
  final block needed to produce logits.
• The benchmark resolves its logical CUDA ordinal through
  CUDA_VISIBLE_DEVICES to the physical NVML handle/UUID and rejects invalid
  explicit device indices.

The mixed-batch E2E test now compares both warmed and cold members against
independent standalone AR-to-DiT conditioning references.

CPU prefix-cache memory

The B1/B2 CPU hidden-state cache allocates 1,062,813,696 bytes
(1,013.58 MiB), all pinned. Pending-write and multimodal-cache bytes were zero
after warmup. The previously reported approximately 2 MiB difference is peak
GPU memory, not CPU cache capacity.

Full AR → DiT attribution

For the same 5,627-token, 50-step workload, using 10 unprofiled requests:

• A: E2E 20,948.66 ± 91.07 ms; AR 5,965.55 ± 81.73 ms; handoff
  108.69 ± 4.15 ms; DiT 14,859.15 ± 15.21 ms.
• B2: E2E 20,804.66 ± 40.45 ms; AR 5,754.00 ± 37.61 ms; handoff
  99.29 ± 17.71 ms; DiT 14,934.20 ± 19.76 ms.
• The float32 AR-to-DiT hidden-state payload is 84,568,064 bytes.

The earlier 5,986.45 / 5,997.09 / 5,634.94 ms A/B1/B2 values are AR-only
latencies, not full image-generation E2E latencies.

Image quality

Across three prompts × three seeds, cache-disabled A versus warm-hit B2:

• Image-image CLIP cosine: 0.9800 ± 0.0152, minimum 0.9583.
• SSIM: 0.9121 ± 0.0960, minimum 0.7219.
• PSNR: 28.28 ± 7.08 dB, minimum 18.99 dB.
• Mean text-image CLIP cosine: A 0.2214, B2 0.2242
  (B2 minus A: +0.0028).

The quality run used an independent request-local DiT seed fix, intentionally
kept outside this prefix-cache change and to be submitted separately.

Full-pipeline profiler traces

• Stage 0: 3.137 s GPU busy in a 9.066 s span; pinned D2H copies total
  3.202 ms.
• Stage 1: 13.894 s GPU busy in a 16.389 s span; pageable H2D copies total
  8.606 ms. Attention and GEMM dominate DiT; gaps ≥ 1 ms total 0.126 s.

Profiler runs are diagnostic only; the latency values above come from
unprofiled measurements.

Follow-up reproduction artifacts

• Current commands and methodology
• Full-pipeline benchmark
• Image-quality benchmark
• Raw full-pipeline results, CPU-memory result, quality metrics, and stage traces

[lsyyysky]

Thanks for the contribution! I tested this on the current head and prefix caching works as expected. A few things to address before merging:

1. Fix the E2E test. [Model] Migrate MammothModa2 DiT to the shared diffusion runtime #7134 renamed ar2dit to ar2diffusion, so the test now fails during setup with an AttributeError. Stage 1 also needs OmniDiffusionSamplingParams now. In addition, the test isn't part of any CI pipeline; if we want to keep it, please add it to test-weekly.yml.
2. Narrow the scope. As mentioned in the inline comments, the metrics changes (memory stats, RPC, and orchestrator timings) aren't directly related to enabling prefix caching. Could you move them into a separate PR? That would also give us room to revisit their design.
3. Reconsider the AR-only YAML. Nothing in this config consumes the hidden states, but they're still cached, which adds a D2H copy on every step. Please either opt out of hidden-state caching for this config or drop the YAML.

Thanks for the review. All three items have been addressed:

Updated the E2E test to use ar2diffusion and OmniDiffusionSamplingParams, and added it to test-weekly.yml. It passes locally on A100
Removed the prefix-cache memory stats, worker RPC, orchestrator timings, related tests, and dependent benchmark attribution from this PR. They will be handled separately.
Removed mammoth_moda2_ar_prefix_cache.yaml. The AR-only benchmark now uses the base AR config with an explicit benchmark-only stage override; the supported serving preset remains the full AR→DiT prefix-cache config.
PTAL, thanks! @ZhengWG @hsliuustc0106

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Strict on zcode (GLM-5.3-Flash) under experiment fleet-strict-cursor-grok46-zcode-glm53flash-5050-c5-z10-20261002.

[vllm-omni-review-bot]

Omni ReviewBot attempt record

Review attempt ended as failed (step 'review' (agent.review_diff): unhandled error: RuntimeError: zcode exited 1 without a result event: statusCode: undefined } Error: Turn execution failed (traceId: e8737c29-fb6a-48e9-9356-5b4dd28f44b3) — check zcode login and the CLI version; retrying strict/zcode/GLM-5.3-Flash in 120s (try ).

[ZhengWG]

@hsliuustc0106 The prefix-cache side looks right to me now:
MammothModa2 uses the default policy (full prompt hidden states merged on a hit), which is exactly what the DiT conditioning split needs, and the cache is enabled on the AR stage only.
The E2E checks (bit-exact cached prefix rows, ≥0.999 on the recomputed tail and text conditioning, exact cached-token accounting, mixed hit/miss batch) are the right ones. The new Numerical Behavior section is a good correction — the old expected to be identical wording overclaimed.

[hsliuustc0106]

@hsliuustc0106 The prefix-cache side looks right to me now: MammothModa2 uses the default policy (full prompt hidden states merged on a hit), which is exactly what the DiT conditioning split needs, and the cache is enabled on the AR stage only. The E2E checks (bit-exact cached prefix rows, ≥0.999 on the recomputed tail and text conditioning, exact cached-token accounting, mixed hit/miss batch) are the right ones. The new Numerical Behavior section is a good correction — the old expected to be identical wording overclaimed.

ty

[hsliuustc0106]

Re-reviewed head a4c162fbb60dda6ccc8e582f8f66b60340531917. Two P2 issues remain in the image-quality benchmark:

1. Use diffusion sampling parameters for stage 1. Line 66 still constructs AR SamplingParams. The shared diffusion client cannot accept that object, so both documented generation scenarios fail. Use OmniDiffusionSamplingParams with the intended seed, steps, and guidance, as the revised live-engine test already does.

2. Read the current PIL image output. Line 81 accesses the former tensor payload. After fixing the parameters, this still fails: generated PIL images now live in output.images, and the tensor-only check below is incompatible with them. Convert the PIL image into the expected array layout and range, updating _normalized_image consistently so the comparison does not apply the old [-1, 1] rescaling to already normalized pixels.

The earlier block-boundary validator, GPU mapping, numerical documentation, E2E migration, weekly CI wiring, and scope concerns are addressed in the current source. I found no new defect in the prefix-cache preset itself.

Validation for this re-review: static syntax and diff checks passed; native/GPU tests were not rerun. Wheel builds, pre-commit, DCO, and CodeQL passed; docs remain pending. No matching-head Buildkite test result is reported.

[lsyyysky]

Re-reviewed head a4c162fbb60dda6ccc8e582f8f66b60340531917. Two P2 issues remain in the image-quality benchmark:

1. Use diffusion sampling parameters for stage 1. Line 66 still constructs AR SamplingParams. The shared diffusion client cannot accept that object, so both documented generation scenarios fail. Use OmniDiffusionSamplingParams with the intended seed, steps, and guidance, as the revised live-engine test already does.
2. Read the current PIL image output. Line 81 accesses the former tensor payload. After fixing the parameters, this still fails: generated PIL images now live in output.images, and the tensor-only check below is incompatible with them. Convert the PIL image into the expected array layout and range, updating _normalized_image consistently so the comparison does not apply the old [-1, 1] rescaling to already normalized pixels.

The earlier block-boundary validator, GPU mapping, numerical documentation, E2E migration, weekly CI wiring, and scope concerns are addressed in the current source. I found no new defect in the prefix-cache preset itself.

Validation for this re-review: static syntax and diff checks passed; native/GPU tests were not rerun. Wheel builds, pre-commit, DCO, and CodeQL passed; docs remain pending. No matching-head Buildkite test result is reported.

done

[hsliuustc0106]

Re-reviewed tip e22ee7a.

Production delta is the opt-in AR→DiT prefix-cache deploy preset plus docs/tests/benchmarks. ZhengWG's earlier scope/E2E/AR-yaml asks look addressed in source; Numerical Behavior doc fix is correct. I found no new defect in the preset or live-engine test shape.

Please (1) refresh the PR Purpose bullets — ar_prefix_cache.yaml is gone; (2) @ZhengWG re-ACK/dismiss the lingering CHANGES_REQUESTED on this tip so merge isn't blocked on a stale review. Optional: mention the preset in the MammothModa2 recipe.

[lsyyysky]

@ZhengWG The earlier E2E, scope, and AR-only preset concerns have been addressed. The PR description is now updated to match the current source. Could you please re-ACK the latest head and dismiss the stale CHANGES_REQUESTED review? Thanks!

[hsliuustc0106]

Re-reviewed tip 7e111ca (main merge only since e22ee7a).

Still no production defect in the opt-in AR→DiT prefix-cache preset, docs, or live-engine E2E shape. Code remains APPROVE-ready from my side.

Process blockers for merge:

1. @ZhengWG please re-ACK/dismiss the lingering CHANGES_REQUESTED on 43590f2 — your Oct-13 functional ACK already covers the tip intent.
2. Please drop the stale Purpose bullet about a CPU stage-input unit test (not in the tip file list). Optional: mention mammoth_moda2_prefix_cache.yaml in the MammothModa2 recipe.

[ZhengWG]

@hsliuustc0106 Sorry for the delay. Approved last head now. cc @lsyyysky

[vllm-omni-review-bot]

Omni ReviewBot review status

Review for commit 7e111cab13d9fbeb13e24e7fdde7933bf840213a ended as blocked (step 'review' (agent.review_diff): [ensemble x3] API Error: Request rejected (429) · Your token-plan 1-month quota has been exhausted. The quota will reset at 11-20 16:00:00 UTC.).

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Strict on claude-code (qwen3.8-max) under experiment fleet-strict-glm-qwen38max-5050-c4-20261105.

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Strict on claude-code (qwen3.8-flash) under experiment fleet-strict-glm-qwen38flash-5050-c4-20261105.