tools : split Metal FA-vec tuning into a standalone tuner

[forforever73]

Overview

Stacked on top of #26570

Layer	Branch	Content
L1	stack/metal-split	Per-op source split (#24021), plus all kernels merged into master since then
L2	stack/metal-fa-vec	Per-device FlashAttention vector tuning (Q, NE) (#25750)
L3 (this PR)	stack/metal-tuner	Standalone FA-vector tuning tool (#26498)

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes,claue opus5

[ggerganov]

Nice, the tuning reports a few optimizations on M2 Ultra. For example, for head size 64:

# noise dk=64 dv=64 ne11=8192 ne01=1 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=1:  Q1NE2=40.2  Q2NE2=86.7  Q4NE2=296.4  Q1NE4=33.4*  Q2NE4=72.0  Q4NE4=153.9  => Q1,NE4  1.21x
# noise dk=64 dv=64 ne11=8192 ne01=2 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=2:  Q1NE2=66.8  Q2NE2=88.1  Q4NE2=299.4  Q1NE4=53.8*  Q2NE4=73.9  Q4NE4=157.2  => Q1,NE4  1.24x
# noise dk=64 dv=64 ne11=8192 ne01=3 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=3:  Q1NE2=94.1  Q2NE2=136.2  Q4NE2=301.5  Q1NE4=75.0*  Q2NE4=124.6  Q4NE4=160.1  => Q1,NE4  1.25x
# noise dk=64 dv=64 ne11=8192 ne01=4 spread=0.2%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=4:  Q1NE2=121.3  Q2NE2=138.2  Q4NE2=306.4  Q1NE4=96.2*  Q2NE4=126.3  Q4NE4=163.7  => Q1,NE4  1.26x
# noise dk=64 dv=64 ne11=8192 ne01=5 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=5:  Q1NE2=149.1  Q2NE2=195.7  Q4NE2=498.5  Q1NE4=118.6*  Q2NE4=184.8  Q4NE4=262.0  => Q1,NE4  1.26x
# noise dk=64 dv=64 ne11=8192 ne01=6 spread=0.4%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=6:  Q1NE2=176.7  Q2NE2=197.4  Q4NE2=501.3  Q1NE4=139.7*  Q2NE4=185.3  Q4NE4=264.8  => Q1,NE4  1.27x
# noise dk=64 dv=64 ne11=8192 ne01=7 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=7:  Q1NE2=204.1  Q2NE2=255.1  Q4NE2=505.2  Q1NE4=161.6*  Q2NE4=235.2  Q4NE4=268.1  => Q1,NE4  1.26x
# noise dk=64 dv=64 ne11=8192 ne01=8 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=8:  Q1NE2=231.5  Q2NE2=256.9  Q4NE2=506.7  Q1NE4=182.6*  Q2NE4=235.7  Q4NE4=269.9  => Q1,NE4  1.27x
# noise dk=64 dv=64 ne11=8192 ne01=16 spread=0.2%
# dtype=f16 dk=64 dv=64 ne11=8192 ne01=16:  Q1NE2=449.0  Q2NE2=490.6  Q4NE2=943.6  Q1NE4=353.8*  Q2NE4=454.5  Q4NE4=503.9  => Q1,NE4  1.27x
# noise dk=64 dv=64 ne11=32768 ne01=1 spread=0.1%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=1:  Q1NE2=116.2  Q2NE2=230.4  Q4NE2=1015.0  Q1NE4=92.4*  Q2NE4=184.3  Q4NE4=402.9  => Q1,NE4  1.26x
# noise dk=64 dv=64 ne11=32768 ne01=2 spread=0.2%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=2:  Q1NE2=207.2  Q2NE2=235.3  Q4NE2=1014.8  Q1NE4=162.3*  Q2NE4=189.3  Q4NE4=409.8  => Q1,NE4  1.28x
# noise dk=64 dv=64 ne11=32768 ne01=3 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=3:  Q1NE2=295.9  Q2NE2=410.2  Q4NE2=1024.7  Q1NE4=231.6*  Q2NE4=342.9  Q4NE4=416.5  => Q1,NE4  1.28x
# noise dk=64 dv=64 ne11=32768 ne01=4 spread=0.1%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=4:  Q1NE2=390.0  Q2NE2=413.7  Q4NE2=1027.8  Q1NE4=302.4*  Q2NE4=346.9  Q4NE4=421.8  => Q1,NE4  1.29x
# noise dk=64 dv=64 ne11=32768 ne01=5 spread=0.2%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=5:  Q1NE2=480.8  Q2NE2=594.8  Q4NE2=1758.6  Q1NE4=376.6*  Q2NE4=496.9  Q4NE4=757.3  => Q1,NE4  1.28x
# noise dk=64 dv=64 ne11=32768 ne01=6 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=6:  Q1NE2=572.4  Q2NE2=597.6  Q4NE2=1775.7  Q1NE4=446.8*  Q2NE4=498.9  Q4NE4=765.7  => Q1,NE4  1.28x
# noise dk=64 dv=64 ne11=32768 ne01=7 spread=0.0%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=7:  Q1NE2=664.4  Q2NE2=778.5  Q4NE2=1782.7  Q1NE4=516.4*  Q2NE4=649.5  Q4NE4=773.7  => Q1,NE4  1.29x
# noise dk=64 dv=64 ne11=32768 ne01=8 spread=0.2%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=8:  Q1NE2=755.3  Q2NE2=783.8  Q4NE2=1795.6  Q1NE4=585.0*  Q2NE4=655.0  Q4NE4=780.7  => Q1,NE4  1.29x
# noise dk=64 dv=64 ne11=32768 ne01=16 spread=0.1%
# dtype=f16 dk=64 dv=64 ne11=32768 ne01=16:  Q1NE2=1473.3  Q2NE2=1526.8  Q4NE2=3415.4  Q1NE4=1137.3*  Q2NE4=1276.6  Q4NE4=1481.3  => Q1,NE4  1.30x

I think we can squash this PR into the previous one #25750 to reduce the churn on test-backend-ops.cpp. Do you agree?

[forforever73]

@ggerganov Agreed. This avoids leaving master with an unnecessarily complex intermediate state in test-backend-ops.cpp.

[forforever73]

Just to clarify, this PR was auto-closed by GitHub as merged when its base branch (stack/metal-fa-vec) advanced to include its head commit — nothing landed on master.