Skip to content

[Bugfix][Model][SM70] Repair Qwen3.8 Flash Next correctness - #408

Merged
yangzhuxinyzx merged 2 commits into
mainfrom
codex/v100-qwen38-correctness-upstream-20260828-152418
Aug 29, 2026
Merged

yangzhuxinyzx merged 2 commits into
mainfrom
codex/v100-qwen38-correctness-upstream-20260828-152418

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

Repair the Qwen3.8-Flash-Next correctness gaps found while auditing current
1Cat public main against the live vLLM support PRs, without weakening the
existing V100/SM70 routes.

Base SHA: 62ad1e02693f4c857f3b7547cef1860ee54e8053.

This PR:

  • adapts vLLM #53896's request-layout-safe PLE n-gram ID custom op while
    preserving 1Cat's pinned-host CPU PLE offload;
  • adapts vLLM #53802/#54076 so hybrid Mamba/GDN prefill uses the real Mamba
    state grid and stops at every state boundary, including the (n - 1) tail;
  • adapts vLLM #48375 so EAGLE/MTP drops the final Mamba cache-hit state;
  • rejects the unvalidated legacy Qwen4Exp runner and keeps QSA from claiming
    batch-invariant split-K reductions;
  • pins the packed GDN FP32-beta behavior with an exact V100 regression; and
  • changes online QPN8 from default-on to explicit experimental opt-in. QPN8
    requantizes dense checkpoint weights at load time; the exact checkpoint
    NVFP4 QPN-M1 expert route remains default-on and unchanged.

Output-quality decision

The historical online-QPN8 A/B kept 96/96 GSM8K answers correct but only 8/96
token sequences matched. One answer expanded from 1,653 to 3,626 tokens and
its repeated-4-gram ratio rose from 0.199 to 0.507. This does not justify a
precision-preserving default, so VLLM_SM70_QWEN4_EXP_ONLINE_QPN8 now defaults
to 0 and requires explicit =1 opt-in with a quality warning.

The previous ~82.3 tokens/s result used online QPN8 and is not an accepted
no-loss baseline. The precision-preserving no-MTP control is about 67.6
tokens/s; reaching 80 tokens/s without QPN8 remains a separate optimization
target.

Test Plan

  • Run focused Qwen4Exp configuration, PLE, QSA, hybrid-cache, prefix-cache,
    and QPN policy tests.
  • Run the native QPN operator test against the existing source-built sidecar.
  • Run packed GDN reference and exact FP32-beta tests on V100.
  • Run all applicable pre-commit hooks on every changed file.
  • When a clean TP4 window is available, run one combined V2/no-MTP/CPU-PLE
    model gate with QPN8 unset, rather than restarting once per scenario.

Test Result

  • 124 passed, 1 deselected, 18 warnings in 52.17 seconds for the focused
    Qwen4Exp/hybrid/prefix/QPN suite.
  • Native QPN sidecar operator: 1 passed with explicit online-QPN8 opt-in.
  • Packed GDN: 7 passed, including the exact FP16-input/FP32-beta state test.
  • Changed-file pre-commit: all applicable Ruff, format, typos, markdownlint,
    mypy, SPDX, forbidden-import, configuration, and API checks passed.
  • git diff --check passed.
  • The downloaded RadixArk index was checked against all 296,475 tensors in 206
    safetensors files: zero extra and zero missing keys, so vLLM #54230 does not
    affect this checkpoint.

No branch E2E result is claimed yet. GPUs 0-3 host the existing public MTP4
service, and GPUs 4-7 were held by a separate Nsight task at validation time;
this PR deliberately did not preempt either workload or repeatedly restart the
model.

Remaining gates and dependencies

  • 1Cat [Bugfix] Avoid KVBlockZeroer H2D staging races #406 owns the independent KVBlockZeroer asynchronous H2D staging
    lifetime fix and should land before high-concurrency prefix-cache acceptance.
  • vLLM issue #54199 remains open for overlapping donor/request lifetimes in
    Mamba prefix-cache precopy. A bounds-only workaround is not included because
    it can replace a crash with silent recurrent-state corruption.
  • A final combined TP4/V2/no-MTP/CPU-PLE gate is required before merge. It must
    cover deterministic generation, tool calls, PLE request-layout changes, and
    prefix lengths around the real state boundaries.

Duplicate-work audit

  • vLLM #51599's async accepted-count race is already covered by the local MRV2
    runner-owned snapshots and request-ID remapping.
  • vLLM #53520 fixes legacy-runner prompt-logprob ordering; MRV2 computes prompt
    logprobs before its drafter, and this PR rejects Qwen4Exp legacy-runner use.
  • vLLM #53122 targets standalone DFlash draft quantization; native Qwen4Exp MTP
    already calls configure_quant_config(Qwen4ExpMTP).
  • vLLM issue #53982's circular block-table OOB path is already avoided in both
    local runners by disabling generic slot mapping for QSA circular groups.
  • Disk/mmap PLE PRs are intentionally not imported because the target contract
    keeps PLE in host RAM.

AI assistance was used for upstream comparison, implementation, tests, and the
audit write-up. Every changed line and reported result was reviewed locally.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant