flash attention addition

[Tushar-ml]

Quality Benchmark

With Triton

 +--------------------+-----------+----------+-------------------------------------+-------+---------+----------------+
| gemma-4-26B-A4B-it | mmlu      | mean_acc | OVERALL                             |  5700 |  0.8963 | -              |
+--------------------+-----------+----------+-------------------------------------+-------+---------+----------------+

With FlashAttention

+--------------------+-----------+----------+-------------------------------------+-------+---------+----------------+
| gemma-4-26B-A4B-it | mmlu      | mean_acc | OVERALL                             |  5700 |  0.8976 | -              |
+--------------------+-----------+----------+-------------------------------------+-------+---------+----------------+

Performance benchmark (64% Improvement)

With Triton

Total End-to-End Latency (ms)
    median  : 1391.00
    p90     : 1653.00
    p95     : 1858.00
    p99     : 2700.80
    max     : 3020.00

With Flashattention

Total End-to-End Latency (ms)
    median  : 670.00
    p90     : 686.00
    p95     : 777.00
    p99     : 935.80
    max     : 1063.00

Usage

vllm serve google/gemma-4-26B-A4B-it --language-model-only --max-model-len 32768 --attention-backend GEMMA4_FLASH_ATTN