ggml-cpu: optimize avx2 q6_k

[netrunnereve]

Basically I took the optimizations I did for AVX a while back and brought them over to AVX2.

PR:

model	size	params	backend	threads	test	t/s
llama 1B Q6_K	860.86 MiB	1.10 B	CPU	4	pp512	63.15 ± 0.34
llama 1B Q6_K	860.86 MiB	1.10 B	CPU	4	tg128	15.63 ± 0.08

  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  345 runs -  3408.30 us/run - 117.44 MFLOP/run -  34.46 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  315 runs -  3435.15 us/run - 234.88 MFLOP/run -  68.38 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  207 runs -  4928.92 us/run - 352.32 MFLOP/run -  71.48 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  252 runs -  4204.82 us/run - 469.76 MFLOP/run - 111.72 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  210 runs -  5041.41 us/run - 587.20 MFLOP/run - 116.48 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  144 runs -  7309.56 us/run - 939.52 MFLOP/run - 128.53 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  3 runs - 417778.33 us/run -  60.13 GFLOP/run - 143.93 GFLOPS

Master:

model	size	params	backend	threads	test	t/s
llama 1B Q6_K	860.86 MiB	1.10 B	CPU	4	pp512	49.75 ± 0.40
llama 1B Q6_K	860.86 MiB	1.10 B	CPU	4	tg128	15.53 ± 0.19

  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  276 runs -  4364.08 us/run - 117.44 MFLOP/run -  26.91 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  280 runs -  3598.24 us/run - 234.88 MFLOP/run -  65.28 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  207 runs -  5086.99 us/run - 352.32 MFLOP/run -  69.26 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  180 runs -  5805.74 us/run - 469.76 MFLOP/run -  80.91 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  168 runs -  6138.51 us/run - 587.20 MFLOP/run -  95.66 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  108 runs -  9657.79 us/run - 939.52 MFLOP/run -  97.28 GFLOPS
  MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):                  2 runs - 594931.50 us/run -  60.13 GFLOP/run - 101.07 GFLOPS

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO