[justinchuby]

Summary

• establish an Obsidian-compatible wiki/ with stable English note titles and paths
• add a Chinese beginner-oriented explanation of allocator, virtual backing, shared mapping, governance, EP ordering, and process-level ownership
• document note conventions and distinguish proposed architecture from current implementation

The formal sources under docs/memory/ remain authoritative; this wiki is explanatory.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.55%. Comparing base (8219fa2) to head (24687fc).
⚠️ Report is 33 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1185      +/-   ##
==========================================
+ Coverage   80.49%   80.55%   +0.05%     
==========================================
  Files         359      357       -2     
  Lines      157323   154719    -2604     
  Branches   157323   154719    -2604     
==========================================
- Hits       126642   124632    -2010     
+ Misses      26120    25490     -630     
- Partials     4561     4597      +36     

Flag	Coverage Δ
mlas	?
offline	80.55% <ø> (+0.15%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	matmul/small_generic_f32_threads=8/1x256x256	37.85 µs	57.36 µs	+51.6%
🔴	matmul/medium_generic_f32_threads=8/32x512x512	1.12 ms	1.66 ms	+47.7%
🔴	add/small_f16_threads=1-internal/1024	422.6 ns	602.3 ns	+42.5%
🔴	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.23 ms	4.51 ms	+39.4%
🔴	matmul/medium_generic_f32_threads=1/32x512x512	2.46 ms	3.24 ms	+31.9%
⚠️	matmul/medium_generic_bf16_threads=8/32x512x512	380.21 µs	485.08 µs	+27.6%
⚠️	gather/large_bf16_threads=1-internal/131072	12.45 µs	15.61 µs	+25.4%
⚠️	qwen3_sampling_processors/top_p_fast_after_top_k	482.69 µs	600.15 µs	+24.3%
⚠️	add/small_f32_threads=1-internal/1024	217.6 ns	268.9 ns	+23.6%
⚠️	gather/large_f16_threads=1-internal/131072	12.85 µs	15.18 µs	+18.1%
⚠️	qwen3_sampling_processors/top_k_full_sort_baseline	2.00 ms	2.36 ms	+18.1%
⚠️	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.24 ms	6.04 ms	+15.2%
✅	sampling_latency/top_k_per_token	48.07 µs	54.72 µs	+13.8%
✅	add/large_f32_threads=1-internal/4194304	662.36 µs	751.94 µs	+13.5%
✅	qwen3_sampling_processors/top_k_top_p_fast	611.70 µs	690.25 µs	+12.8%
✅	kv_cache/alloc_dealloc_pages	37.47 µs	42.00 µs	+12.1%
✅	matmul/small_generic_f32_threads=1/1x256x256	37.65 µs	42.00 µs	+11.5%
✅	qwen3_sampling_processors/top_k_partial_selection	131.45 µs	145.61 µs	+10.8%
✅	add/medium_f16_threads=1-internal/262144	106.50 µs	113.89 µs	+6.9%
✅	sampling_latency/min_p_per_token	207.36 µs	221.04 µs	+6.6%
✅	matmul/small_generic_f16_threads=8/1x256x256	31.40 µs	32.59 µs	+3.8%
✅	add/medium_f32_threads=1-internal/262144	26.10 µs	26.60 µs	+1.9%
✅	sampling_latency/greedy_per_token	3.00 µs	3.04 µs	+1.5%
✅	tokenization/decode_tokens_per_second	5.74 ms	5.77 ms	+0.7%
✅	logit_processing/seven_processor_chain_per_step	323.59 µs	322.34 µs	-0.4%
✅	reduce_mean/medium_f32_threads=1-internal/65536	254.60 µs	252.76 µs	-0.7%
✅	add/large_bf16_threads=1-internal/4194304	1.80 ms	1.75 ms	-2.8%
✅	matmul/small_generic_f16_threads=1/1x256x256	35.25 µs	33.90 µs	-3.8%
✅	matmul/medium_generic_f16_threads=1/32x512x512	34.49 µs	33.09 µs	-4.1%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.04 ms	997.54 µs	-4.5%
✅	add/medium_bf16_threads=1-internal/262144	109.90 µs	104.98 µs	-4.5%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	555.61 µs	527.94 µs	-5.0%
✅	sampling_latency/top_p_per_token	408.02 µs	386.28 µs	-5.3%
✅	add/small_bf16_threads=1-internal/1024	487.9 ns	461.7 ns	-5.4%
✅	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	469.83 µs	440.30 µs	-6.3%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.14 ms	1.99 ms	-7.2%
✅	matmul/medium_generic_f16_threads=8/32x512x512	33.97 µs	31.41 µs	-7.5%
✅	tokenization/encode_tokens_per_second	390.75 µs	360.99 µs	-7.6%
✅	add/large_f16_threads=1-internal/4194304	1.76 ms	1.60 ms	-9.3%
✅	matmul/small_generic_bf16_threads=8/1x256x256	47.45 µs	42.82 µs	-9.8%
✅	reduce_mean/small_f32_threads=1-internal/4096	16.65 µs	14.79 µs	-11.1%
✅	gather/medium_f32_threads=1-internal/32768	4.36 µs	3.86 µs	-11.6%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	178.88 µs	156.83 µs	-12.3%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.43 ms	1.25 ms	-12.7%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	6.03 ms	5.24 ms	-13.1%
✅	gather/medium_f16_threads=1-internal/32768	2.83 µs	2.45 µs	-13.6%
✅	gather/small_f16_threads=1-internal/4096	605.0 ns	520.4 ns	-14.0%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	10.45 ms	8.99 ms	-14.0%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	105.61 µs	90.43 µs	-14.4%
✅	gather/medium_bf16_threads=1-internal/32768	2.77 µs	2.36 µs	-14.6%
🟢	grammar_masking/llguidance_compute_mask/32	85.65 µs	69.73 µs	-18.6%
🟢	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	703.91 µs	557.58 µs	-20.8%
🟢	gather/small_f32_threads=1-internal/4096	872.2 ns	670.9 ns	-23.1%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	125.40 µs	96.36 µs	-23.2%
🟢	matmul/small_generic_bf16_threads=1/1x256x256	43.65 µs	33.48 µs	-23.3%
🟢	gather/small_bf16_threads=1-internal/4096	646.0 ns	476.9 ns	-26.2%
🟢	gather/large_f32_threads=1-internal/131072	38.32 µs	25.38 µs	-33.8%
🟢	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	66.59 µs	40.97 µs	-38.5%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	66.19 µs	40.51 µs	-38.8%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 5.97 5.13 6.54 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)