[Perf] Ship the qualified SM70 long-context layout and drop its context fences

[yangzhuxinyzx]

What

#602 moved the grouped E4M3 FP32 long-context operator into the tree under the source digest of the qualified candidate but shipped the body of an earlier revision of that candidate.

	digest recorded by the manifest	body actually shipped in #602
source	eb7a85511f58…f6b820e1 (4,985 lines)	8459d57c6b72…f070f738 (4,316 lines)
module	sm70_grouped_swizzle_eb7a85511f58	sm70_grouped_attention_8459d57c6b72
DSO	9db33737adb8…62fee21	dac8262f3d02…76a97339

All five header files are byte-identical between the two revisions, so neither the build, nor the manifest check, nor any test could tell them apart. Only the kernel body differs, by 669 lines: the shipped revision lacks the probability swizzle, the early probability store, PV-value reuse, the full-q8 specialization, the all-visible tile walk and the shared E4M3 lookup table that the recorded digest names.

This PR ships the layout the manifest already claimed, and removes the literal context fences that also kept the q1 verifier tail off its graph.

Changes

• grouped-attention.cu replaced with the qualified source. Its only difference from the candidate is registration (PYBIND11_MODULE → TORCH_LIBRARY_FRAGMENT + a double-signature adapter; torch/extension.h → torch/library.h + explicit c10/cuda/CUDAException.h). BUILTIN_MANIFEST now records that source digest so a rebuilt kernel cannot inherit the workspace identity of a different layout.
• MAX_CONTEXT returns to 262144. #602 raised it to 262152 for one page of headroom, but no descriptor bucket can equal 262152, and it is larger than the capacity the route exists to cover.
• The four comparisons against a literal 262144 now compare against the live long-attention contract. Under 262152 they silently dropped the q1 tail out of the long-context graph variant — that is why #602 measured 38.24 ms at 261888 with the whole tail eager.
• The compact scalar tail operator ships inside _vllm_fa2_C and needs no environment variable. VLLM_SM70_DFLASH2_SCALAR_ATTENTION_MANIFEST remains an override for an external candidate.
• VLLM_SM70_DFLASH2_TAIL_CUDAGRAPHS defaults on; =0 restores the eager tail.
• The 256K contract admits the whole B1 verifier tail range again (query_rows [2..8]). An earlier revision of this work narrowed it to [8] after measuring a regression, but that regression belonged to the predecessor kernel.
• Both ported kernels join the existing verbatim-port entries in the clang-format exclude, so the shipped body stays byte-comparable with the digest its manifest records.

Measurement

Four V100-SXM2-32GB, TP4/B1, E4M3 target KV, the PR596 serving environment, the official request path, and the PR596 same-startup protocol (alternating tail_switch arms inside one serving session, three steady pairs per cell, median of request averages):

context/seed	control	candidate	rounds	accepted/round
1024/0	17.439	17.460	54	3.778
131072/0	23.802	23.780	59	3.339
261888/0	36.654	32.834	50	4.020
261888/2	38.451	32.669	44	4.727

Both arms produced byte-identical token ids, finish reasons, round counts, accepted-draft counts and emitted-tokens-per-round.

The same cells measured with the qualified external DSOs were 15.81–15.90, 22.56–22.89, 35.24–36.20 → 31.39–31.47 and 37.32–37.88 → 31.25–31.35. Every cell of this run sits 1.5–10% above those — including the 1024 cell, which touches no long-context route at all — so the residual is host contention (a co-tenant was running on the other four cards), not the operator. 1024 and 131072 differ between arms by less than 0.03 ms in both runs.

Net effect at the capacity: #602's 38.24 ms → 32.8 ms, no regression at 1K or 128K.

Caveats

• One startup, not a median of three independent startups.
• Only the 261888/seed0 and 261888/seed2 cells show any arm difference; 1024 and 131072 are controls.
• Not a natural-EOS quality run. The quality evidence is token-exactness between the two arms inside the session, not a dataset score.

Assisted-by: DeepSeek Harness

[yangzhuxinyzx]

Follow-up commit: the context bound is no longer a literal

94610608d9 removes the remaining hard-coded context. The route used to capture its graph and admit its operators at a fixed 262144; that value is the capacity the shipped kernel was qualified for, but it is not the context a deployment necessarily serves, and every consumer compared against the literal (the captured bound, the scalar tail's descriptor bucket, its max_seq_len_hint, and the q1 threshold at 131072). That is also how #602 silently lost the q1 tail: one constant moved and four comparisons against it stopped matching.

The contract now separates capability from the served window:

• long_attention_capability(manifest) — the context the operator was qualified for; stays a manifest field and is the ceiling.
• long_attention_contract(manifest, capacity) — the bound the graph is captured at, min(capability, model_config.max_model_len).
• The graph builder stamps that bound onto the descriptor; the wrappers admit any bucket within their capability instead of re-deriving the window. One place reads the service capacity, so the two cannot drift apart again.
• The q1 threshold becomes "half the captured bound" instead of 131072.
• The compact scalar tail reports itself inactive when the long-context route is off.

Verified across three served windows

--max-model-len	captured graph bound (from the worker log)	operator capability	1K steady (ms)
262144	262144	262144	16.003 / 15.961 / 15.922
131072	131072	262144	16.178 / 15.963 / 15.901
65536	65536	262144	15.876 / 16.122 / 15.989

The bound follows the flag; the operator capability does not move. 1K is unaffected by the bound.

On the 1K number

The 1K round is 15.88–16.12 ms here, which matches the PR596 reference (15.81–15.90). The 16.8–19.2 ms figures seen earlier in this work came from host contention — a co-tenant was running on the other four cards — not from the route. As a control, the same request with VLLM_SM70_E4M3_LONG_ATTENTION=0 costs 16.370/16.378/16.324 ms in an adjacent window, so the long route is not what the 1K round pays for.

Tests

tests/v1/worker/test_sm70_long_attention_graphs.py, tests/v1/attention/test_sm70_e4m3_scalar_tail.py and tests/v1/attention/test_sm70_e4m3_grouped.py are updated for the new contract. Three expectations they still carried from the 132096 era are corrected: a default of 132096, a rejected max_context above 262144, and the manifest variable treated as the on/off switch. The three files go from 5 failures / 44 passes to 51 passes.