metal: gate the M3-class fusion whitelists on the pre-M5 predicate

[kk1987]

Fixes #769.

The exact-fusion campaign's newest decode ports gate on ds4_gpu_ported_m5_decode_feature_enabled(), whose pre-M5 predicate already admits M1–M4. The older fusion sites still used literal device_name_contains("M3") || ("M5") whitelists, so pre-M5 devices other than M3 silently skipped fifteen eligible fusions, and ds4_test failed its 29 forced-fusion exactness assertions there because the entry points refuse to run at all.

This switches those fifteen sites to the same predicates the ported gate uses: is_pre_m5 || is_m5 where M5 was admitted, plain is_pre_m5 for the M3-only sites (those screened negative on M5, so M5 stays out). Net −15 lines. Left untouched: the Metal 4 tensor API sites and the M5 private-scratch heuristic (pre-M5 hardware lacks them), and the router SIMD sites (already on the predicate). ds4.c needs nothing.

Verified on M4 Max 128 GiB, DS4F 0731 IQ2XXS resident:

• make test goes from 29 failures to fully green, including the bit-exact fused-vs-reference comparisons for both previously failing kernels.
• Per-fusion A/B with metal_decode_schedule_bench / metal_prefill_variant_bench, 3 samples per fusion, cool-down rounds, medians; every run bit-exact over the full vocabulary with identical token selection:

fusion	side	median
zero-prefix prefill mask cache	prefill	+4.5%
inplace/affine RoPE pair	decode	+3.8%
gathered KV stage	decode	+1.8%
persistent zero attention mask	decode	+0.8%
router weights batch	prefill	+0.8%
remaining nine	both	neutral (±0.15%)

• Full ds4-bench curves (ABBA, this branch vs main, 32 ctx points): median generation +1.55%, prefill +0.7%.

On M1 Ultra 128 GB, an equivalent generalization measured roughly 23.5 → 25 tok/s generation on an empty context (thanks @lmr — see comments). M2 has no tester, but every site keeps its DS4_METAL_DISABLE_* rollback and the forced-fusion test assertions now cover any device the predicate admits.

A benchmarking note: on a MacBook the numbers drift heavily once thermal throttling kicks in — enough that two prefill deltas came out with the wrong sign on back-to-back hot runs. The numbers above are cool-down multi-round medians.

🤖 Generated with Claude Code

[lmr]

Just as a comment, I made a more generalized version of your change to admit any pre-M5 hardware to these M3-class fusion whitelists and it seems to work well on my M1 Ultra 128GB. I would have to get the precise benchmark numbers to validate it, but I am getting around 25 tops of generation on an empty context, which is better than I had before (23.5 on an empty context, consistent with what another M1 Ultra user reported in #706).

[lmr]

So I can test a more generalized version of your PR in case you want to make it (or leave it like this since we don't have an M2 user to verify it on that hardware too).

[kk1987]

Nice, thanks for testing! That M1 Ultra bump matches what I'd hope for — the RoPE pair fusions were the biggest single win on M4 too.

I've switched the PR to the pre-M5 predicate: is_pre_m5 || M5-name for the sites that admitted M3/M5, plain is_pre_m5 for the M3-only ones (those are the fusions that screened negative on M5, so M5 stays out). Nobody can verify M2, but every site keeps its DS4_METAL_DISABLE_* rollback and ds4_test's forced-fusion assertions now cover any device the whitelist admits, so an M2 user would find out quickly and can opt out per fusion.

If you want precise M1 numbers, the harness I used: ./speed-bench/metal_decode_schedule_bench -m model.gguf --candidate-env DS4_METAL_DISABLE_INPLACE_ROPE_PAIR --include-selection (same for the other DISABLE names) — it aborts unless both variants are bit-identical. One caveat from my runs: on my MacBook the numbers drift heavily once thermal throttling kicks in, enough that two prefill deltas came out with the wrong sign on back-to-back hot runs. Letting the machine cool down before each run and taking the median of 3 runs was what it took to get numbers that were stable run-to-run.

[chilang]

Independent replication on a second M4 Max (128 GiB, DS4F 0731 IQ2XXS resident). I measured these before finding this PR, so the numbers are uncoordinated.

Applies clean to current main 84cc882, 169 commits after it was opened. On that tree: make clean, make cpu clean, ./ds4_test --metal-kernels OK, full ./ds4_test exit 0 (ds4 tests: ok). Reverting it reproduces the 29 assertion failures from #769.

Per-fusion A/B with metal_decode_schedule_bench --include-selection, control = fused:

candidate-env	control t/s	candidate t/s	delta	tokens
DISABLE_INPLACE_ROPE_PAIR	35.0378	33.8086	+3.64%	1024
DISABLE_AFFINE_ROPE_PAIR	28.7764	27.6956	+3.90%	1024
DISABLE_GATHERED_KV_STAGE	21.8532	21.5448	+1.43%	1024
DISABLE_GATHERED_KV_STAGE	32.7534	32.3385	+1.28%	512
DISABLE_COMPRESSOR_PAIR_PROJ	25.2436	25.2255	+0.07%	1024
DISABLE_COMPRESSOR_PAIR_PROJ	33.8912	33.9329	-0.12%	512
DISABLE_PERSISTENT_ZERO_ATTN_MASK	19.9442	19.8918	+0.26%	1024

Every run bit-exact: exact_rows=1041 exact_floats=134580480 exact_selected_ids=1040 vocab=129280.

Two notes on reading those rows:

• INPLACE and AFFINE both gate decode_attn_rope_fuse_available and kv_rope_fp8_fuse_available, so those two rows measure the same group twice rather than summing. The agreement between them is the useful part.
• My +0.26% on the persistent zero mask is heat-soak, not a disagreement with your +0.8%. It was the third of three back-to-back runs, with absolute decode falling 35.0 -> 28.8 -> 19.9 t/s across them. Same drift you documented.

I had first patched this by adding device_name_contains("M4") alongside the M3 sites. The predicate version here is better: it covers M1/M2 too and removes lines instead of adding them.