Skip to content

[Attention][Core] Add generic prefix-anchored SWA on SM70 - #281

Merged
yangzhuxinyzx merged 2 commits into
mainfrom
audit/v100-pr239-20260824-160842
Aug 24, 2026
Merged

yangzhuxinyzx merged 2 commits into
mainfrom
audit/v100-pr239-20260824-160842

Conversation

@yangzhuxinyzx

@yangzhuxinyzx yangzhuxinyzx commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Purpose

Replacement for #239, rebased onto current main. Adds prefix-anchored sliding-window attention as an explicit, default-off inference-engine capability. Admission is based only on hardware, operator, tensor/cache, graph, and concurrency contracts; it does not inspect model/checkpoint/architecture identity.

The engine option is AttentionConfig.prefix_anchored_decode_window. The prefix remains globally visible while generated-token KV outside the live window is masked and reclaimed. Unsupported combinations fail up front, and missing runtime metadata fails closed.

Key audit fixes over #239

  • removed DeepSeek/OCR-specific classes and checkpoint config wiring
  • moved activation to model-agnostic AttentionConfig and base Attention
  • rebased the CUDA changes over the current Flash-V100 kernel stack
  • rejected non-positive windows, non-fp16 KV, non-SM70, MLA/local/sink attention, CP, speculation, KV connectors/offload, and unsupported backends
  • fixed the missing eager/piecewise metadata path
  • made masked extension availability and metadata consistency hard startup/runtime gates
  • preserved original-author attribution without carrying unsigned commits

Test Plan / Result

  • pytest -q tests/config/test_prefix_anchored_swa.py tests/v1/attention/test_prefix_anchored_swa_mask.py tests/v1/core/test_single_type_kv_cache_manager.py: 32 passed
  • SM70 source build: CUDA_HOME=/usr MAX_JOBS=4 TORCH_CUDA_ARCH_LIST=7.0 python setup.py build_ext --inplace: passed
  • pytest -q tests/kernels/attention/test_sm70_flash_v100_anchored_swa.py on V100: 19 passed
  • changed-file pre-commit suite, including Ruff, clang-format, mypy, SPDX and config/docs checks: passed
  • same-toolchain SASS comparison to current origin/main: 55/55 decode and 64/64 paged-prefill default-off instances byte-identical

No model end-to-end or throughput claim is made. Refs #239.

yangzhuxinyzx and others added 2 commits August 25, 2026 00:16
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Port the prefix-anchored sliding-window cache and SM70 kernels to current main as an explicit model-agnostic AttentionConfig capability. Validate operator/runtime contracts up front, fail closed on missing metadata, preserve default-off SASS, and cover eager, piecewise, decode, prefill, gap eviction, and multi-request anchors.

Co-authored-by: kkobold <sabbaghdanilo@gmai.com>

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 24, 2026 16:57
@yangzhuxinyzx yangzhuxinyzx changed the title [Attention][Core] Port generic prefix-anchored SWA to current main [Attention][Core] Add generic prefix-anchored SWA on SM70 Aug 24, 2026
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Audit note: the required pre-run-check passed. The repository-wide manual pre-commit job failed because the all-files formatter rewrote 228 pre-existing files; its rewritten-file set has zero overlap with this PR's 26 changed files. The complete changed-file pre-commit suite passes locally on the PR head. This known main-branch formatting drift is therefore not attributed to #281.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant