docs: expand wiki with repository knowledge map (replaces #1190)

[justinchuby]

Replacement for #1190, rebased onto current main (the original could not be squash-merged because #1210 is stacked on its head branch, which triggers GitHub's stacked-PR 403). Content is identical to #1190; only the base is refreshed.

What this adds

A 20-file explanatory wiki under wiki/ (crate architecture, execution backends, EP contracts, memory model, metadata-driven runtime, request lifecycle, testing/verification, performance playbook, repository map). It repeatedly labels itself non-authoritative and points at docs/ and source for verification.

Accuracy review (the bar for docs in this repo)

I spot-checked the load-bearing claims against main as it stands today (after the memory capability-split revert #1247, session-state unification #1255, reasoning_content #1257, CUDA mask-reservation fix #1264):

• Crate architecture — the crate roster and layering match the actual workspace Cargo.toml members and crates/ tree. The map is explicitly non-exhaustive and says so.
• Memory model (the highest-risk area) — wiki/memory/Memory Management for Beginners.md is marked status: proposed and frames the capability split as a recommendation, while describing the current combined DeviceAllocator as the present reality. I verified against crates/onnx-runtime-memory-governor/src/allocator.rs: the trait indeed still carries allocate, commit_allocation_range/decommit_allocation_range, allocation_committed_bytes, mapped_bytes, create_shared_prefix, and the successful-no-op default it warns about. Because the doc calls the split a proposal and the combined trait the current state, it survives the feat(memory): minimal DeviceAllocator contract + optional VirtualBacking/SharedMapping capabilities (#1186 phases 1-2) #1192→revert(memory): withdraw unsafe memory capability split #1247 revert rather than being invalidated by it. The proposed onnx-runtime-memory-api crate correctly does not exist. with_memory(...) — cited as the mechanism that already satisfies issue [memory] Extract a standalone DeviceAllocator trait out of ExecutionProvider #513's core goal — exists (onnx-genai-ort, ep-cpu, ep-cuda).
• Execution backends — the ORT-vs-native-nxrt description and EP contract are accurate and high-level; nothing contradicts the refactor(engine): unify native + ORT session-state policy behind SessionStore #1255 session-state unification.
• Repository Map — every code path it points at exists (decode_loop.rs, sampling.rs, speculative/, all crate dirs).
• Links — all 58 repo-relative links resolve. (The one apparent miss, docs/path.md, is inside a fenced note-template code block, not a real link.)
• Benchmark-number rule (docs/benchmarks/2026-08-15-... §32.2) — no perf figure is copied without metadata; the only "speedup" mentions are the docs teaching that a bare speedup is insufficient, and the GB values are illustrative diagram quantities, not measurements.

Verification

• Nothing under a test, build script, or CI job references the new wiki/ files (checked with git grep), so this cannot break the build or suite.
• No in-tree markdown lint / mdbook config exists, so there is no docs build to run here — stated rather than invented.

No inaccuracies found inside the diff; no corrections were needed.

Closes #1190.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.95%. Comparing base (63e8035) to head (9a7ccc0).
⚠️ Report is 26 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1267      +/-   ##
==========================================
+ Coverage   79.69%   79.95%   +0.25%     
==========================================
  Files         358      361       +3     
  Lines      155271   156900    +1629     
  Branches   155271   156900    +1629     
==========================================
+ Hits       123747   125445    +1698     
+ Misses      26940    26822     -118     
- Partials     4584     4633      +49     

Flag	Coverage Δ
offline	79.95% <ø> (+0.25%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 19 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

✅ Benchmarks — No Regression

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	52.75 µs	59.75 µs	+13.3%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.27 ms	3.47 ms	+6.2%
✅	tokenization/encode_tokens_per_second	391.29 µs	412.67 µs	+5.5%
✅	gather/medium_f16_threads=1-internal/32768	2.27 µs	2.36 µs	+3.5%
✅	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	344.00 µs	348.10 µs	+1.2%
✅	matmul/small_generic_bf16_threads=8/1x256x256	30.70 µs	30.95 µs	+0.8%
✅	gather/medium_f32_threads=1-internal/32768	3.53 µs	3.52 µs	-0.2%
✅	matmul/large_generic_f16_threads=8/32x1024x1024	79.79 µs	79.57 µs	-0.3%
✅	logit_processing/seven_processor_chain_per_step	301.56 µs	299.51 µs	-0.7%
✅	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	548.37 µs	542.12 µs	-1.1%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.31 ms	1.29 ms	-1.5%
✅	reduce_mean/medium_f32_threads=1-internal/65536	236.05 µs	228.85 µs	-3.0%
✅	gather/small_bf16_threads=1-internal/4096	456.7 ns	442.8 ns	-3.1%
✅	gather/large_bf16_threads=1-internal/131072	12.16 µs	11.74 µs	-3.5%
✅	grammar_masking/llguidance_compute_mask/32	83.60 µs	80.65 µs	-3.5%
✅	sampling_latency/greedy_per_token	3.12 µs	3.00 µs	-3.8%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	503.79 µs	484.52 µs	-3.8%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.01 ms	1.93 ms	-3.9%
✅	add/medium_bf16_threads=1-internal/262144	117.76 µs	110.90 µs	-5.8%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.11 ms	1.99 ms	-5.9%
✅	kv_cache/alloc_dealloc_pages	39.14 µs	36.76 µs	-6.1%
✅	gather/small_f32_threads=1-internal/4096	723.5 ns	677.6 ns	-6.3%
✅	reduce_mean/small_f32_threads=1-internal/4096	15.70 µs	14.50 µs	-7.6%
✅	matmul/medium_generic_f16_threads=8/32x512x512	30.63 µs	28.28 µs	-7.7%
✅	matmul/small_generic_bf16_threads=1/1x256x256	31.60 µs	29.15 µs	-7.7%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.61 ms	8.86 ms	-7.8%
✅	add/large_bf16_threads=1-internal/4194304	1.68 ms	1.54 ms	-8.0%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	81.89 µs	75.36 µs	-8.0%
✅	gather/small_f16_threads=1-internal/4096	498.8 ns	456.5 ns	-8.5%
✅	sampling_latency/min_p_per_token	211.19 µs	192.93 µs	-8.6%
✅	gather/medium_bf16_threads=1-internal/32768	2.54 µs	2.32 µs	-8.8%
✅	qwen3_sampling_processors/top_k_top_p_fast	668.68 µs	609.53 µs	-8.8%
✅	sampling_latency/top_p_per_token	387.79 µs	352.50 µs	-9.1%
✅	tokenization/decode_tokens_per_second	6.29 ms	5.72 ms	-9.1%
✅	add/medium_f16_threads=1-internal/262144	113.50 µs	102.69 µs	-9.5%
✅	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	48.58 µs	43.88 µs	-9.7%
✅	matmul/medium_generic_f16_threads=1/32x512x512	31.51 µs	28.30 µs	-10.2%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.44 ms	2.16 ms	-11.1%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.03 ms	909.97 µs	-11.3%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	421.94 µs	371.31 µs	-12.0%
✅	qwen3_sampling_processors/top_k_partial_selection	147.34 µs	129.63 µs	-12.0%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	568.48 µs	495.05 µs	-12.9%
✅	add/small_f32_threads=1-internal/1024	226.1 ns	196.0 ns	-13.3%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	6.07 ms	5.25 ms	-13.5%
✅	add/small_f16_threads=1-internal/1024	552.7 ns	475.5 ns	-14.0%
✅	matmul/small_generic_f16_threads=8/1x256x256	42.38 µs	36.44 µs	-14.0%
✅	add/small_bf16_threads=1-internal/1024	532.4 ns	455.1 ns	-14.5%
✅	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	48.42 µs	41.25 µs	-14.8%
✅	sampling_latency/top_k_per_token	57.11 µs	48.62 µs	-14.9%
🟢	matmul/small_generic_f32_threads=1/1x256x256	39.86 µs	33.51 µs	-15.9%
🟢	matmul/large_generic_f32_threads=8/32x1024x1024	4.99 ms	4.13 ms	-17.2%
🟢	add/large_f16_threads=1-internal/4194304	2.00 ms	1.59 ms	-20.7%
🟢	matmul/small_generic_f16_threads=1/1x256x256	37.38 µs	29.06 µs	-22.3%
🟢	add/large_f32_threads=1-internal/4194304	880.66 µs	669.13 µs	-24.0%
🟢	gather/large_f16_threads=1-internal/131072	17.00 µs	12.76 µs	-25.0%
🟢	add/medium_f32_threads=1-internal/262144	35.00 µs	25.46 µs	-27.3%
🟢	matmul/medium_generic_f32_threads=8/32x512x512	1.26 ms	894.70 µs	-29.1%
🟢	gather/large_f32_threads=1-internal/131072	42.71 µs	24.94 µs	-41.6%
🟢	matmul/small_generic_f32_threads=8/1x256x256	54.78 µs	31.98 µs	-41.6%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.30 2.97 4.68 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)