Skip to content

[Bugfix][SM70] Gate DFlash2 verifier by native capability - #422

Merged
yangzhuxinyzx merged 2 commits into
mainfrom
codex/v100-dflash2-grouped-q16-guard-20260829-080317
Aug 29, 2026
Merged

yangzhuxinyzx merged 2 commits into
mainfrom
codex/v100-dflash2-grouped-q16-guard-20260829-080317

Conversation

@yangzhuxinyzx

@yangzhuxinyzx yangzhuxinyzx commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Purpose

Prevent a source-overlay deployment from routing q16 DFlash2 verification into an older Flash-V100 native extension that only supports q8. The production incident occurred on a 51,088-token cached request with a 16-token continuation: the native TORCH_CHECK killed EngineCore and left the systemd service falsely active with orphaned TP workers.

Base SHA: e315dd47096524ed8a9fd43770f925a188a59d23.
Current head SHA: f1f5741cda.

The native extension now exposes its grouped-verifier max-query capability. Python defaults legacy binaries without that entry point to q8, while newly rebuilt q16 binaries report 16. The normal DFlash2 q8 verifier remains on the grouped fast path; unsupported q16 work falls back to the exact paged-decode path.

Test Plan

  • exercise q8 and q16 grouped routing with a simulated q16-capable extension;
  • exercise q16 fallback with a simulated legacy q8 extension and assert the grouped operator is never called;
  • load an existing legacy Flash-V100 binary and verify the compatibility fallback reports max-q 8;
  • run changed-file pre-commit and Python byte compilation;
  • deploy to the four-V100 service and replay health, normal q8 decode, a 16-token final prefill chunk, prefix cache, and tool calling.

Test Result

  • policy tests: 5 passed, 112 deselected;
  • legacy binary capability probe: native entry absent, reported max-q 8;
  • changed-file pre-commit, Markdown lint, Python compileall, and git diff --check: passed;
  • exact incident shape: 49,084 prompt tokens = 12 x 4,089 + q16; completed in 13.288 s without an engine failure;
  • repeated 49,084-token prefix: 1.715 s, confirming prefix-cache health;
  • tool smoke: valid get_weather({"city":"北京"}) call;
  • historical web prompt: 512 output tokens, 206.06 token/s streaming decode, 17.463 ms/engine round, 3.599 emitted tokens/round;
  • service health: active/running, public and loopback health checks pass, no post-restart traceback or grouped-verifier shape error.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 29, 2026 08:27
@yangzhuxinyzx
yangzhuxinyzx merged commit f94bd0e into main Aug 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant