CPU EP: native Celu and Mish kernels, AVX2 Log (removes the last silent ORT activation deferrals)

[justinchuby]

What this changes

Adds native CPU-EP kernels for Celu (opset 12) and Mish (opset 18), which had no kernel at all and were therefore silently handed to ORT's CPU EP, and vectorises Log (f32), which was still taking a scalar libm call per element. Celu/Mish are wired through all four capability filters (registry, PHASE1_OPS, dtype table, shape-inference table, plugin descriptors) and added to the two activation-coverage tests.

Behavioural claim — verified independently (reviewer)

The headline "removes the last silent ORT activation deferrals" is a behavioural claim, so I falsified it rather than trusting it, on Intel Core i7-13800H, Windows 11:

• The deferral was real and silent. GetCapability runs three fail-closed filters; an op absent from the shape-inference table (Filter 2) or the dtype descriptor list (Filter 3) is handed to ORT with no diagnostic. Celu/Mish had no kernel, so they never reached supports_op.
• Falsification (RED/GREEN). With the PR applied, activation_and_norm_ops_clear_every_capability_filter passes. Removing just the Celu/Mish entries from the shape-inference table (compute.rs, Filter 2) turns it RED with ::Celu: no shape rule (shape filter declines it) … would be handed to ORT's CPU EP for both ops; restoring returns it to GREEN. The test genuinely guards the deferral.
• "The last" is defended structurally, not by spot-check. The coverage test's OWNED list now enumerates the full activation/normalisation family and holds every member to both the shape and dtype filters; the companion every_registered_op_has_a_shape_rule_or_is_a_known_gap enumerates all registered ops, and its remaining shape-table gap (data-dependent, internal-fusion, and inferrable-but-unwritten ops) contains no activations. Log was never a deferral — it already had a scalar kernel — so this PR is a perf change for it, not a coverage change.

Correctness

The kernels' own tests (dense sweeps vs the scalar reference, subnormals, and the full IEEE special set) pass; the scalar reference, the f64 path, and the AVX2 path are held to agree, including NaN propagation (f32::max/min drop NaN, so each kernel guards it to match ORT). Full cargo test -p onnx-runtime-ep-cpu --lib = 1362 passed, 0 failed, 17 ignored (base 1353 + 9 new). cargo clippy --all-targets -D warnings clean. map_ps's #[target_feature] (#1227) verified intact.

Performance — measured on this box, ORT-independent

The gaps doc's ratios were taken on an AMD EPYC 9V74 against ORT. I have no ORT reference on this host, so I measured the ORT-independent internal win instead: the AVX2 kernel vs the scalar-native fallback (which is what runs on non-AVX2 hosts and small tensors), on Intel Core i7-13800H (14C/20T), Windows 11, single thread, 1,048,576 elements, min-of-200 × 9 rounds.

kernel	scalar-native (ms)	AVX2 (ms)	speedup
Log (positive domain)	2.89	1.14	~2.54×
Celu (alpha=1, mixed)	2.77	0.68	~4.07×
Mish (mixed)	17.82	2.50	~7.13×

These are hardware-dependent (see docs/performance/CPU_ACTIVATION_GAPS.md §32 lesson) and are not the same quantity as the doc's ORT-relative ratios — mine compare AVX2 against the native scalar path on this machine; the AMD EPYC figures compare native against ORT. Presented as peers, each with its hardware named. Both agree on direction: the native kernels are a clear win.

[github-actions]

⚠️ Benchmark Change Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
⚠️	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	6.57 ms	7.58 ms	+15.4%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	10.14 ms	11.46 ms	+13.0%
✅	qwen3_sampling_processors/top_k_partial_selection	142.79 µs	156.24 µs	+9.4%
✅	qwen3_sampling_processors/top_k_top_p_fast	696.33 µs	761.06 µs	+9.3%
✅	gather/large_f16_threads=1-internal/131072	12.34 µs	13.34 µs	+8.1%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.65 ms	2.83 ms	+6.7%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.66 ms	3.89 ms	+6.1%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	398.11 µs	421.98 µs	+6.0%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	539.00 µs	563.41 µs	+4.5%
✅	tokenization/decode_tokens_per_second	7.02 ms	7.23 ms	+2.9%
✅	matmul/small_generic_bf16_threads=8/1x256x256	48.58 µs	49.72 µs	+2.3%
✅	grammar_masking/llguidance_compute_mask/32	72.53 µs	74.18 µs	+2.3%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	570.48 µs	580.60 µs	+1.8%
✅	tokenization/encode_tokens_per_second	390.62 µs	395.11 µs	+1.2%
✅	sampling_latency/greedy_per_token	3.32 µs	3.36 µs	+1.1%
✅	matmul/small_generic_f32_threads=8/1x256x256	49.23 µs	48.82 µs	-0.8%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.40 ms	2.38 ms	-1.0%
✅	gather/medium_bf16_threads=1-internal/32768	2.61 µs	2.58 µs	-1.0%
✅	logit_processing/seven_processor_chain_per_step	325.40 µs	316.26 µs	-2.8%
✅	gather/small_bf16_threads=1-internal/4096	506.6 ns	490.4 ns	-3.2%
✅	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	676.86 µs	648.93 µs	-4.1%
✅	add/large_f32_threads=1-internal/4194304	983.45 µs	941.61 µs	-4.3%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	180.17 µs	172.23 µs	-4.4%
✅	matmul/medium_generic_f16_threads=1/32x512x512	37.19 µs	35.19 µs	-5.4%
✅	matmul/medium_generic_f16_threads=8/32x512x512	37.71 µs	35.64 µs	-5.5%
✅	sampling_latency/top_k_per_token	57.36 µs	53.61 µs	-6.5%
✅	matmul/small_generic_f16_threads=8/1x256x256	46.29 µs	42.64 µs	-7.9%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	4.73 ms	4.29 ms	-9.2%
✅	matmul/small_generic_f32_threads=1/1x256x256	44.44 µs	39.83 µs	-10.4%
✅	gather/medium_f16_threads=1-internal/32768	2.70 µs	2.42 µs	-10.5%
✅	kv_cache/alloc_dealloc_pages	42.25 µs	37.61 µs	-11.0%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.51 ms	2.23 ms	-11.4%
✅	add/large_f16_threads=1-internal/4194304	2.85 ms	2.52 ms	-11.4%
✅	add/small_f32_threads=1-internal/1024	222.3 ns	196.9 ns	-11.4%
✅	sampling_latency/min_p_per_token	252.32 µs	221.77 µs	-12.1%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.35 ms	1.17 ms	-13.3%
🟢	add/medium_bf16_threads=1-internal/262144	156.37 µs	132.10 µs	-15.5%
🟢	gather/small_f16_threads=1-internal/4096	620.4 ns	518.1 ns	-16.5%
🟢	reduce_mean/medium_f32_threads=1-internal/65536	317.96 µs	261.43 µs	-17.8%
🟢	sampling_latency/top_p_per_token	481.08 µs	391.56 µs	-18.6%
🟢	matmul/large_generic_f16_threads=1/32x1024x1024	111.60 µs	90.16 µs	-19.2%
🟢	gather/large_f32_threads=1-internal/131072	32.88 µs	26.18 µs	-20.4%
🟢	reduce_mean/small_f32_threads=1-internal/4096	20.53 µs	16.09 µs	-21.6%
🟢	matmul/large_generic_bf16_threads=8/32x1024x1024	2.34 ms	1.82 ms	-22.1%
🟢	reduce_mean/large_f32_threads=1-internal/262144	1.29 ms	986.77 µs	-23.3%
🟢	gather/medium_f32_threads=1-internal/32768	5.48 µs	4.11 µs	-24.9%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	120.32 µs	89.59 µs	-25.5%
🟢	matmul/small_generic_bf16_threads=1/1x256x256	55.38 µs	40.39 µs	-27.1%
🟢	add/small_f16_threads=1-internal/1024	616.0 ns	440.8 ns	-28.4%
🟢	matmul/small_generic_f16_threads=1/1x256x256	43.72 µs	31.10 µs	-28.9%
🟢	add/large_bf16_threads=1-internal/4194304	2.51 ms	1.76 ms	-30.0%
🟢	gather/small_f32_threads=1-internal/4096	948.3 ns	658.5 ns	-30.6%
🟢	add/small_bf16_threads=1-internal/1024	634.5 ns	435.0 ns	-31.4%
🟢	add/medium_f16_threads=1-internal/262144	183.58 µs	117.48 µs	-36.0%
🟢	add/medium_f32_threads=1-internal/262144	38.04 µs	23.61 µs	-37.9%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	97.04 µs	58.95 µs	-39.3%
🟢	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	134.31 µs	81.50 µs	-39.3%
🟢	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	1.28 ms	749.62 µs	-41.4%
🟢	gather/large_bf16_threads=1-internal/131072	17.15 µs	10.04 µs	-41.4%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.53 3.75 6.77 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[justinchuby]

Reviewer verification — reproduced, merging

Validated on Intel Core i7-13800H (14C/20T), Windows 11, off current main (1353-test baseline confirmed first).

Behavioural claim falsified and confirmed. The "silent ORT deferral" is real: an activation absent from the shape-inference table or the dtype descriptor list is handed to ORT with no diagnostic. With the PR, activation_and_norm_ops_clear_every_capability_filter passes; deleting just the Celu/Mish shape-inference entries turns it RED (::Celu: no shape rule … would be handed to ORT's CPU EP), restoring returns GREEN. "The last" holds structurally — the coverage test enumerates the whole activation/norm family, and the registered-op shape-gap list contains no activations. Log was already kernelled, so it is a perf change, not a deferral fix.

Correctness: kernels' dense sweeps, subnormals, and IEEE specials pass; scalar/f64/AVX2 paths agree incl. NaN propagation. cargo test --lib = 1362 passed / 0 failed / 17 ignored (1353 + 9); clippy -D warnings clean; #1227's map_ps target_feature intact.

Performance (ORT-independent, this box): AVX2 vs native-scalar, 1M elems, single-thread, min-of-200 × 9 rounds — Log 2.89→1.14 ms (~2.54×), Celu 2.77→0.68 ms (~4.07×), Mish 17.82→2.50 ms (~7.13×). The doc's AMD-EPYC vs-ORT ratios are a peer, not reproduced here (no ORT reference on this host).

Merging by squash.