[Frontend][Model] Per-model serving defaults, DeepSeek-OCR resolution modes, and SWA async-eviction fix (SM70)

[kkobold]

What

Serving-quality improvements for long-generation OCR workloads, plus a
correctness fix for the prefix-anchored SWA capability merged in #281:

1. Bugfix (SWA/[Attention][Core] Add generic prefix-anchored SWA on SM70 #281): out-of-window gap blocks were freed on the
   scheduled-token basis; under async scheduling this can free blocks the
   worker has not yet computed past. Free on the processed-token basis
   instead. Regression tests simulate the async gap.
2. Per-model serving defaults with request-override bounds: an
   architecture-keyed registry applied at the chat endpoint. A model can
   declare its recommended sampling/extra-args recipe; requests may
   override within declared bounds. DeepSeek-OCR registers its checkpoint
   README recipe (n-gram blocker 30/90 with whitelist tokens, temperature
   0, max_tokens 8192) so an unconfigured client gets the model's intended
   behavior via plain chat completions.
3. Architecture-keyed chat template fallbacks: DeepSeek-OCR's
   model_type previously fell through to the DeepSeek-VL2 template, which
   injects role markers the OCR checkpoint was not trained with; the
   fallback now pins the raw-concatenation prompt (golden test included).
4. Repetition-detection default for OCR serving: a bounded
   exact-pattern detector as a safety net for degenerate table pages,
   request-overridable.
5. Per-request DeepSeek-OCR resolution modes: the checkpoint's five
   official modes (tiny/small/base/large/gundam) selectable via
   mm_processor_kwargs: {"image_mode": ...}; multi-image serving
   profile; max_crops plumbing. The serving default remains the
   checkpoint's gundam configuration.

Why

On document OCR, the model's own recipe is load-bearing: without the
n-gram blocker and the correct raw prompt, long pages degenerate into
loops. These changes make the endpoint serve the checkpoint's intended
recipe by default while keeping every knob per-request-overridable within
bounds. The SWA fix closes an async-scheduling correctness gap in the
capability added by #281.

Validation

All device work on Tesla V100-SXM2 32 GB (sm_70), fp16, on current main.

• 195 CPU tests green, including [Attention][Core] Add generic prefix-anchored SWA on SM70 #281's admission/mask/manager suites
  (proves coexistence with the merged SWA capability), the new
  eviction-basis regression tests, resolution-mode suite, template golden,
  defaults/bounds and repetition-detection suites.
• Endpoint (serve) on V100: deterministic 3× byte-identical generations;
  resolution-mode discrimination (default ≡ gundam as identity control;
  base/large produce distinct image-token counts); document-page scoring
  against known ground truth (per-mode table in tests receipt).

Serving DeepSeek-OCR (validated configuration)

Launch line used for every endpoint validation in this PR (Tesla
V100-SXM2 32 GB, fp16):

vllm serve /path/to/DeepSeek-OCR \
  --dtype float16 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.85 \
  --enforce-eager \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0 \
  --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor

• --logits-processors loads the n-gram blocker class; the serving-defaults
  registry supplies its per-request parameters and fails startup with the
  exact flag text if the class is missing (never silent looping).
• --no-enable-prefix-caching / --mm-processor-cache-gb 0 are the
  checkpoint recipe's engine flags.
• For multi-image requests add --limit-mm-per-prompt '{"image":8}'
  (the SM70 baseline defaults to 1 image per prompt).
• --enforce-eager is the validated configuration; CUDA graphs measured
  ~4x faster decode at ~1% window cost and can be enabled after a
  graph-mode validation pass.
• Tensor parallelism: the checkpoint has 10 attention heads, so
  --tensor-parallel-size must divide 10 (1 or 2 on a 4-GPU host); 4
  fails at startup with the head-divisibility error.

Merge base

Branch applies and merges cleanly onto current main at 34403018d
([Kernel][SM70] Optimize E4M3 page800 long decode, #290); all validation in
this PR ran on that base.

[yangzhuxinyzx]

Thanks for the careful kernel and manager work. I found two correctness blockers in the integration path:

1. The anchored metadata is only attached in FlashAttnV100MetadataBuilder.build_for_cudagraph_capture(). The normal build() path calls _attach_common_flash_metadata(), but that helper does not copy common_attn_metadata.prefix_anchor_lens or set decode_sliding_window; build_for_drafting() has the same omission. TritonAttentionMetadataBuilder.build() does not propagate arbitrary CommonAttentionMetadata fields. A direct source-level probe with CommonAttentionMetadata(prefix_anchor_lens=[16]) produced regular Flash metadata with neither prefix_anchor_lens nor decode_sliding_window. Therefore _anchored_swa_params() returns (None, 0) on ordinary eager/PIECEWISE runtime metadata. This PR itself downgrades FULL graphs to PIECEWISE, so the KV manager can evict the gap while the attention path runs unmasked. Please attach the persistent anchor buffer/window in every runtime metadata path (preferably through one shared helper), including drafting if supported, and add an integration test that exercises builder.build(), not only direct kernel calls.

2. decode_sliding_window is activated on non-None but never validated as positive. PrefixAnchoredSWAManager uses 0/negative values in its eviction arithmetic, while _anchored_swa_params() explicitly disables the mask for values <= 0. Repro with block_size=4, prompt=16, computed=28: window=0 evicts logical blocks [4,5,6] and window=-8 evicts [4,5,6,7,8], yet the backend reports mask inactive for both. Please reject non-integer/non-positive values before constraints/spec construction and add zero/negative validation tests.

Local audit results otherwise: merge against current main was clean; git diff --check passed; Ruff 0.14.0 check/format passed on all 26 changed Python files; the new mask and single-type KV-manager suites passed 14/14. I did not repeat the expensive model E2E evidence.

[yangzhuxinyzx]

Current-main revalidation on exact head e0c9095afdfc3dc81aa2f934740877dc192cb83d: keep CHANGES_REQUESTED.

The focused anchor/mask manager tests pass (8 passed), but the integration defect remains: anchor/window fields are attached only in build_for_cudagraph_capture(); the ordinary build() path still never assigns prefix_anchor_lens or decode_sliding_window, so _anchored_swa_params() disables the mask while the KV manager may evict gap blocks. Zero/negative windows also remain unvalidated.

There are now two additional integration blockers against main@66becb6423b6: the merge-tree conflicts in flash-attention-v100/include/fused_mha.h, and both PR commits lack Signed-off-by trailers. Resolve all three areas before another review.

[yangzhuxinyzx]

Superseded by #281, now merged as f6a5b57b645867d87f83ada43231f2dd25b40a4a. The replacement preserves the prefix-anchored SWA idea while rebasing it onto current main and removing model/checkpoint/architecture-specific activation. Admission is now an explicit, default-off inference-engine contract over SM70, backend, dtype, KV/cache, graph, and concurrency properties. It also fixes non-positive-window validation, eager/piecewise metadata propagation, extension capability checks, fail-closed runtime handling, and current Flash-V100 kernel integration. Focused evidence: 32 CPU tests, 19 V100 operator tests, SM70 build, changed-file static gates, and byte-identical default-off SASS for all 119 compared kernel instances. The original branch is thousands of commits stale, conflicts with current main, and contains unsigned/model-specific wiring, so it should not itself be merged.

[yangzhuxinyzx]

公开开发已恢复：本 PR 已取消归档、解锁并重新打开。为避免把旧基线上的问题直接带入最新 main，现统一作为 Draft 审计项；恢复可见性不代表已经批准合并。 该组属于旧版功能/重放实现，需先与当前主线及同类 PR 去重，提取仍有价值的通用推理引擎改动，再做聚焦源码与质量验证。

[yangzhuxinyzx]

Latest-main deduplication audit completed at main@18b0b44e06fae1b0e18ee48ef1ccdb2573835d13.

The actual PR head e0c9095afdfc3dc81aa2f934740877dc192cb83d contains only the old prefix-anchored SWA implementation; despite the current title, it does not contain the later serving-default/OCR/async-eviction stack. Every useful capability is already present in current main through cleaner replacements:

• [Attention][Core] Add generic prefix-anchored SWA on SM70 #281: generic, explicit prefix-anchored SWA with non-positive-window checks, eager/piecewise metadata propagation, extension capability guards, and no model/checkpoint identity activation;
• [Bugfix] Track in-flight KV eviction boundaries #296: processed-token in-flight KV eviction boundaries plus regression coverage;
• [Model] Add explicit DeepSeek-OCR resolution modes #293: explicit per-request DeepSeek-OCR resolution modes, rebased without model identity defaults.

The architecture-keyed serving recipe/default files from the rejected identity-default stack are absent from the current tree; skip_special_tokens remains a normal boolean request default, and chat-template fallback remains model-type based. Therefore #239 has no unique source left to merge and would only reintroduce unsigned, conflicting, model-specific wiring. Per project policy it remains Open/Draft rather than being closed.