perf(deepseek-v41): preserve decode capacity and optimize NVMe Engram

[MadeBy561]

Forward parallel DSpark’s decode reservation to B12X so maxseq 32 retains the decode split policy: 352 rows at K5, 480 at K7. Also add opt-in original Engram scale residency, bounded NVMe prefetch, WKV output-column TP and batch-local metadata reuse. Engram options require B12X #360. Rebased onto JJ 9342b1a; the resolved files exactly match the tested R37 candidate.

Matched R37 comparison; + PRs means #360 and #736 together. Four RTX PRO 6000 Max-Q GPUs, 300 W, memory +6000; TP4/DCP1, native precision, NVMe Engram, K7/adaptive, batch 4096, maxseq 32, 1M context, utilization 0.97, full target/draft graphs through 256. All four arms include R37’s existing dispatch/prefill improvements.

Metric	R37 greedy	R37 probabilistic	+ PRs greedy	+ PRs probabilistic
32K uncached prefill, client	12,754	12,882	13,720	13,729
C1 decode	182.5	163.5	208.7	229.6
C1 verifier steps/s	69.1	69.8	82.9	82.4
Pagoda thinking median	178.1	205.2	218.8	261.3
Sieve median	274.8	297.1	318.5	337.3
C32 aggregate decode	1,106.5	1,147.6	1,226.2	1,220.1

Tok/s unless marked otherwise. llm_decode_bench 0.4.34: one 30s decode cell per concurrency after 15s warmup, context 0, T1/top-p.95/high75; 32K prefill has 2–3 uncached samples. Thinking: three 35s runs, first 3s excluded, max100/top-p1. Sieve: five runs after one warmup, 2000-token cap, normal EOS. Output includes reasoning tokens.

All C1–C32 cells passed isolation checks with zero errors/queues. Each arm passed 3 seeded answer checks and 32/32 concurrent badge checks. No reduced model precision; full model equivalence and individual PR ablations remain unmeasured.

23 vLLM + 16 B12X GPU checks passed. Ruff/diff checks passed; full pre-commit retains inherited mypy errors (13 candidate/15 untouched base), plus existing pickle-import and CUDA-fixture failures.

GPU test commands

PY=/opt/venv/bin/python
$PY -m pytest -p no:cacheprovider -x -q tests/models/test_deepseek_v4_1_engram.py -k 'disk or budget or storage_selection'
$PY -m pytest -p no:cacheprovider -x -q tests/v1/worker/test_attn_utils.py -k reuses_equivalent
$PY -m pytest -p no:cacheprovider -x -q tests/v1/attention/test_b12x_v41_workspace.py -k 'parallel_draft_reservation or output_projection or post_load_packs or block_linear_capture_retains'

AI-assisted implementation/testing; owner requested ready-for-review status. The R37 source audit found these changes absent from the release; its dispatch/attention optimizations address different paths.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 1efc8c97-3d57-4d81-a519-36d6aff9546c

📥 Commits

Reviewing files that changed from the base of the PR and between 311f105 and 464e439.

📒 Files selected for processing (2)

• tests/v1/attention/test_b12x_v41_workspace.py
• vllm/models/deepseek_v4_1/attention.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds disk Engram residency and prefetch controls, tensor-parallel projection support, revised attention planning, and shared attention metadata buffers. Tests cover configuration validation, disk preparation, decode splitting, cache-group reuse, and token-mapping updates.

Changes

Disk Engram controls

Layer / File(s)	Summary
Engram configuration and validation vllm/config/engram.py, tests/models/test_deepseek_v4_1_engram.py	EngramConfig validates disk controls and checks resident scale RAM using scale bytes only.
Disk Engram prefetch and projection vllm/models/deepseek_v4_1/common/engram.py	Engram propagates disk options, supports prefetch completion and cancellation, and selects ColumnParallelLinear when projection sharding is enabled.
Disk Engram model integration and tests vllm/models/deepseek_v4_1/nvidia/model.py, tests/models/test_deepseek_v4_1_engram.py	Disk preparation validates hash state, completes configured prefetches, and isolates cleanup failures. Tests cover option combinations, graph hashes, preparation, and validation.

Decode reservation and attention execution

Layer / File(s)	Summary
Attention execution and index planning vllm/models/deepseek_v4_1/attention.py	Attention returns custom-operation tensors directly, uses eager boundaries for selected preparation steps, and selects short index resources for eligible scans.
Decode capacity planning and validation vllm/models/deepseek_v4_1/attention.py, tests/v1/attention/test_b12x_v41_workspace.py	The decode MLA plan receives reserved capacity. Tests verify decode split capacity and separate extend planning for parallel drafting.

Attention metadata ownership

Layer / File(s)	Summary
Shared token-mapping ownership vllm/v1/worker/gpu/attn_utils.py, tests/v1/worker/test_attn_utils.py	Metadata construction shares the first builder’s persistent token-mapping buffer across cache groups.
Cache-group reuse validation tests/v1/worker/test_attn_utils.py	Tests distinguish matching and different cache specifications and verify build, update, reuse, and token-mapping behavior.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~30 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant Model as NVIDIAModel
  participant Engram
  participant Table as DiskTable
  Model->>Engram: prepare_disk(hash_ids)
  Engram->>Table: prefetch(indices)
  Model->>Engram: finish_disk()
  Engram->>Table: complete pending lookup
  Table-->>Engram: disk output
  Engram-->>Model: prepared Engram output

Loading

Suggested reviewers: lukealonso

Merge Risk: ⚪ Minimal · up to 464e4

The reviewed configuration forwarding and attention-planning changes have no remaining concrete merge-blocking risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 11.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 34 functions across 8 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the two primary changes: preserving DeepSeek-V4.1 decode capacity and optimizing NVMe Engram behavior.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Independent DS4.1 R38 component validation at public JJ 66c293578412 and
B12X ce419b52681b, RTX PRO 6000 Workstation: 13 vLLM Engram tests and 16
B12X disk tests passed. The test container uses unlocked memory and io_uring
permissions matching the supplied Compose profile.

Coverage includes original E8M0 scale residency/accounting, exact disk versus
resident row results, duplicate/edge rows, graph consumers, bounded prefetch,
submission/read failure cleanup and stale-row rejection. Test-only tblib==3.1.0
loads the repository's distributed fixture; serving sources were not patched.

Full-image TP4 RAM-Engram text/Vision and prefix checks also pass. The combined
R37/R38 C1/C8 gains in the linked receipt are not isolated SSD gains:
whole-model SSD throughput and opt-in projection partitioning are not qualified
by this RAM benchmark. Resident scales, prefetch and partitioning stay off by
default. Original contributor attribution is retained in the committed sources.

Qualification, source lock and raw measurements.