fix(session): stop --phase-profile paying for the activation-memory planner

[justinchuby]

The defect

The executor ran its activation-memory planner whenever phase profiling was on:

let measure_activation_plan = !nested
    && mode == RunMode::Eager
    && stage2.plan.is_none()
    && activation_memory_planning_enabled();   // == phase_profile::enabled()

The planner rebuilds the view map and re-plans every activation, on every run -
work the shipped runtime never does. So --phase-profile perturbed the run it was
measuring, and then reported its own cost back as a phase of that run
(run_scoped.activation_memory_plan) that no unprofiled run ever pays.

Magnitude - corrected after review

The planner's own span reads 1.9-6.0 us per run. Removing it lowers native_min
on sm_decode_h32_kv4096 from 0.065 ms to 0.063 ms - median of 15 interleaved
repetitions, 300 runs each, --native-threads 1 --ort-intra-threads 1, both arms in
one loop. About 3%, in line with the span. 13 of 15 repetitions favour the fixed
arm.

Two claims from the first draft of this PR are withdrawn. It claimed a 35%
inflation (0.065 -> 0.088 ms, ratio 1.023 -> 1.080) and that the planner also
polluted the adjacent exec_kernel.compute figure (84.5 vs 60.3 us/call). The
reviewer could not reproduce either. A properly interleaved re-measurement showed
both were artifacts of a bimodal, contended host and of comparing arms across
separate invocation windows: the 0.065/0.088 split appears in both arms, and
exec_kernel.compute swings 75-168 us/call regardless of the planner. My original
numbers came from non-interleaved runs, which is exactly the discipline this repo's
own benchmark notes insist on and which I did not follow.

The change stands on the smaller, reproducible effect and on the profiler no longer
reporting a phantom phase.

The fix

The planner gets its own gate, separate from the profiler's:

• --phase-profile calls enable_exec_phase_profile_for_process, which now drives
  the profiler only.
• NXRT_EXEC_PHASE_PROFILE=1 in the environment still enables both, so the CLI
  memory report (interactive.rs -> activation_memory_plan_stats) keeps working
  through the path it already used.
• NXRT_ACTIVATION_MEMORY_PLAN=1, or enable_activation_memory_plan_for_process(),
  opts into planning without profiling.

Reading the environment directly, rather than calling enabled(), is what separates
the programmatic enable from the environment one.

Deliberate behaviour change: the CUDA step profiler
(native_decode/cuda.rs) called enable_exec_phase_profile_for_process and so
switched the planner on as a side effect - taxing the very decode steps it exists to
time. That coupling is not restored, and the reason is now a comment at the call
site. A CUDA-step-profiling user who was relying on the incidental activation-plan
numbers must now ask for them explicitly. Raised by review as finding 1.

The first regression test was vacuous

The gate was hardcoded true under cfg(test):

fn activation_memory_planning_enabled() -> bool {
    #[cfg(test)] { true }
    #[cfg(not(test))] { phase_profile::enabled() }
}

so the production wiring was the one line no unit test could reach. My first test
asserted on the phase_profile module functions, passed, and still passed when I
re-coupled the planner to phase profiling in run.rs. It pinned the parts, not the
wiring.

The cfg(test) override is gone. Tests and production share one gate, the two tests
that need planning ask for it through an RAII guard that clears it again on drop, and
re-coupling the wiring now fails both of them (verified).

Review findings addressed

1. CUDA step profiler behaviour change - documented at the call site; the
   "works exactly as before" wording is corrected to "the environment path works as
   before".
2. Planner tests leaked PLAN_STATE = on - replaced the bare
   force_activation_plan_enabled(true) with an RAII guard that takes the globals
   lock and clears the gate on drop, including on panic.
3. cost -> costs typo - fixed.
4. activation_memory_plan_stats doc implied native+executed was sufficient -
   now states the gate is required.

Tests

• cargo test -p onnx-runtime-session --lib - 181 passed
• cargo test -p onnx-genai-engine --lib - 405 passed
• cargo test -p onnx-runtime-ep-cpu --lib - 1340 passed
• cargo test -p onnx-runtime-ep-cpu-plugin --test plugin_ort_e2e - 54 passed
• cargo fmt --all --check, cargo clippy clean

No ORT fallback, no defer, no runtime MLAS, no scheduler changes.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	46.08 µs	71.66 µs	+55.5%
⚠️	gather/large_f16_threads=1-internal/131072	12.20 µs	15.57 µs	+27.5%
⚠️	matmul/small_generic_f16_threads=1/1x256x256	29.52 µs	34.50 µs	+16.9%
✅	qwen3_sampling_processors/top_k_partial_selection	142.13 µs	162.00 µs	+14.0%
✅	gather/small_f32_threads=1-internal/4096	784.7 ns	868.1 ns	+10.6%
✅	reduce_mean/small_f32_threads=1-internal/4096	15.77 µs	17.03 µs	+8.0%
✅	add/large_f16_threads=1-internal/4194304	1.82 ms	1.95 ms	+6.7%
✅	matmul/small_generic_f32_threads=8/1x256x256	36.72 µs	39.12 µs	+6.5%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.63 ms	3.82 ms	+5.1%
✅	matmul/small_generic_f32_threads=1/1x256x256	36.20 µs	37.92 µs	+4.8%
✅	add/large_f32_threads=1-internal/4194304	809.31 µs	842.54 µs	+4.1%
✅	gather/medium_bf16_threads=1-internal/32768	2.44 µs	2.50 µs	+2.5%
✅	add/large_bf16_threads=1-internal/4194304	1.85 ms	1.86 ms	+0.2%
✅	gather/medium_f32_threads=1-internal/32768	4.00 µs	3.99 µs	-0.3%
✅	reduce_mean/medium_f32_threads=1-internal/65536	263.26 µs	262.30 µs	-0.4%
✅	logit_processing/seven_processor_chain_per_step	335.77 µs	334.33 µs	-0.4%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	6.45 ms	6.41 ms	-0.6%
✅	grammar_masking/llguidance_compute_mask/32	84.30 µs	83.58 µs	-0.9%
✅	kv_cache/alloc_dealloc_pages	40.20 µs	39.79 µs	-1.0%
✅	gather/small_f16_threads=1-internal/4096	539.8 ns	532.6 ns	-1.3%
✅	qwen3_sampling_processors/top_k_top_p_fast	686.06 µs	673.82 µs	-1.8%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.21 ms	2.17 ms	-1.9%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.36 ms	9.12 ms	-2.6%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.21 ms	2.15 ms	-2.8%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	561.00 µs	544.75 µs	-2.9%
✅	gather/medium_f16_threads=1-internal/32768	2.48 µs	2.40 µs	-3.2%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	171.61 µs	165.96 µs	-3.3%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	4.08 ms	3.94 ms	-3.4%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	521.97 µs	502.89 µs	-3.7%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.19 ms	1.13 ms	-4.3%
✅	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	615.59 µs	588.99 µs	-4.3%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.36 ms	1.30 ms	-4.4%
✅	matmul/medium_generic_f16_threads=1/32x512x512	29.35 µs	27.87 µs	-5.0%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.04 ms	1.93 ms	-5.2%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	76.90 µs	72.92 µs	-5.2%
✅	matmul/medium_generic_f32_threads=8/32x512x512	964.43 µs	911.70 µs	-5.5%
✅	matmul/small_generic_bf16_threads=8/1x256x256	30.85 µs	29.16 µs	-5.5%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	396.37 µs	363.84 µs	-8.2%
✅	matmul/medium_generic_f16_threads=8/32x512x512	30.62 µs	27.95 µs	-8.7%
✅	sampling_latency/top_p_per_token	389.46 µs	354.01 µs	-9.1%
✅	add/medium_f16_threads=1-internal/262144	128.14 µs	115.29 µs	-10.0%
✅	matmul/small_generic_f16_threads=8/1x256x256	32.13 µs	28.73 µs	-10.6%
✅	gather/small_bf16_threads=1-internal/4096	579.2 ns	516.7 ns	-10.8%
✅	add/small_f16_threads=1-internal/1024	543.7 ns	484.9 ns	-10.8%
✅	add/small_bf16_threads=1-internal/1024	602.2 ns	535.7 ns	-11.0%
✅	add/medium_f32_threads=1-internal/262144	30.04 µs	26.53 µs	-11.7%
✅	sampling_latency/min_p_per_token	219.54 µs	193.34 µs	-11.9%
✅	matmul/small_generic_bf16_threads=1/1x256x256	35.07 µs	30.54 µs	-12.9%
✅	tokenization/encode_tokens_per_second	416.00 µs	361.55 µs	-13.1%
✅	gather/large_bf16_threads=1-internal/131072	14.32 µs	12.42 µs	-13.2%
✅	add/small_f32_threads=1-internal/1024	279.0 ns	239.9 ns	-14.0%
✅	add/medium_bf16_threads=1-internal/262144	142.39 µs	122.39 µs	-14.0%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	51.66 µs	43.45 µs	-15.9%
🟢	sampling_latency/top_k_per_token	57.84 µs	48.53 µs	-16.1%
🟢	sampling_latency/greedy_per_token	3.65 µs	3.05 µs	-16.6%
🟢	tokenization/decode_tokens_per_second	7.02 ms	5.72 ms	-18.5%
🟢	gather/large_f32_threads=1-internal/131072	30.80 µs	24.24 µs	-21.3%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	104.48 µs	79.60 µs	-23.8%
🟢	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	686.86 µs	504.48 µs	-26.6%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 4.95 3.96 6.47 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)