[Hardware][Power]Add Power VSX Attention Backend and fix l2 Cache Crash

[Akashcodes732]

Purpose

This PR adds native PowerPC (ppc64le) VSX support for the vLLM CPU backend and resolves a initialization crash caused by IndexError: unordered_map::at.

Test Plan

vllm serve ibm-granite/granite-3.3-8b-instruct --port 8000 --max-model-len 512 --max-num-batched-tokens 8192 --dtype bfloat16 

Test Result

============ Serving Benchmark Result ============
Successful requests:                     100       
Failed requests:                         0         
Benchmark duration (s):                  87.22     
Total input tokens:                      12800     
Total generated tokens:                  3200      
Request throughput (req/s):              1.15      
Output token throughput (tok/s):         36.69     
Peak output token throughput (tok/s):    200.00    
Peak concurrent requests:                100.00    
Total token throughput (tok/s):          183.44    
---------------Time to First Token----------------
Mean TTFT (ms):                          46663.89  
Median TTFT (ms):                        39152.59  
P99 TTFT (ms):                           61091.73  
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          1297.55   
Median TPOT (ms):                        1534.57   
P99 TPOT (ms):                           1546.81   
---------------Inter-token Latency----------------
Mean ITL (ms):                           1297.55   
Median ITL (ms):                         854.39    
P99 ITL (ms):                            21945.52  

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[gemini-code-assist]

Code Review

This pull request introduces VSX (PowerPC) support for the CPU attention backend, including a dedicated VSX implementation for attention kernels and updates to the dispatch and architecture detection logic. Critical issues were identified in the new VSX implementation, including a vector merging bug on little-endian systems and incorrect pointer arithmetic when handling 16-bit types. Furthermore, there are opportunities to optimize performance by vectorizing the half-precision loading logic and to improve the robustness of the L2 cache size detection globally to prevent potential crashes.

[gemini-code-assist]

The else block in copy_q_heads_tile contains a critical bug when scalar_t is c10::Half. The pointer arithmetic (float*)curr_src + d incorrectly increments the address by d * sizeof(float) instead of d * sizeof(scalar_t). Additionally, it loads data using vec_xl and treats it as float without proper conversion, which is invalid for 16-bit types. Since load_row8_B_as_f32 is already specialized to handle float, BFloat16, and Half correctly, it should be used here to fix the bug and simplify the implementation.

          __vector float v0, v1;
          load_row8_B_as_f32<scalar_t>(curr_src + d, v0, v1);

          v0 = vec_mul(v0, scale_vec);
          v1 = vec_mul(v1, scale_vec);

          vec_xst(v0, 0, curr_dst + d);
          vec_xst(v1, 0, curr_dst + d + 4);

[gemini-code-assist]

The load_row8_B_as_f32 specialization for c10::Half uses a scalar loop for half-to-float conversion. This function is called within the inner loop of the attention micro-kernel (gemm_micro_ppc64le_Mx8_Ku4), making this a significant performance bottleneck for FP16 models. For Power9 and newer architectures, you should use the xvcvhpsp intrinsic for vectorized conversion. Even for older architectures, bitwise vector operations can be used to perform this conversion more efficiently than the current scalar fallback.

[Akashcodes732]

Hi @bigPYJ1151 , can you please take a look at the changes ?

[Akashcodes732]

Hi @bigPYJ1151 ,

Can you please take a look at the changes made in this PR ?

[Akashcodes732]

Hi @bigPYJ1151 ,

Any update on this PR on when this can be reviewed ?

[bigPYJ1151]

Hi @Akashcodes732 Please reslove the conflicts.

[mergify]

Hi @Akashcodes732, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[mergify]

Hi @Akashcodes732, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[Akashcodes732]

Hi @bigPYJ1151 , the failures look unrelated

[Akashcodes732]

Hi @bigPYJ1151 , can we merge this ?

[Akashcodes732]

Hi @DarkLight1337 @mgoin ,

The changes are approved and the failures look unrelated. Can we merge this PR ?

[DarkLight1337]

Done