[ROCm] Fix build problem resulted from previous commit related to FP8 kv-cache support

[hongxiayang]

Fixes: #2725
Current head failed to build on ROCm, and I got errors like:

g++ -pthread -B /opt/conda/envs/py_3.8/compiler_compat -Wl,--sysroot=/ -pthread -shared -B /opt/conda/envs/py_3.8/compiler_compat -L/opt/conda/envs/py_3.8/lib -Wl,-rpath=/opt/conda/envs/py_3.8/lib -WlEN,--no-as-needed -Wl,--sysroot=/ /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/activation_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/hip_utils_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/layernorm_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/moe_align_block_size_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/pos_encoding_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/pybind.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/quantization/gptq/q_gemm.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/quantization/squeezellm/quant_hip_kernel.o -L/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib -L/opt/rocm/lib -L/opt/rocm/hip/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lamdhip64 -lc10_hip -ltorch_hip -o build/lib.linux-x86_64-cpython-38/vllm/_C.cpython-38-x86_64-linux-gnu.so
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__float2bfloat16(float)':
cache_kernels.hip:(.text+0x0): multiple definition of `__float2bfloat16(float)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x0): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__bfloat1622float2(__hip_bfloat162)':
cache_kernels.hip:(.text+0x40): multiple definition of `__bfloat1622float2(__hip_bfloat162)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x40): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__double2bfloat16(double)':
cache_kernels.hip:(.text+0x60): multiple definition of `__double2bfloat16(double)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x60): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__float22bfloat162_rn(HIP_vector_type<float, 2u>)':
cache_kernels.hip:(.text+0xa0): multiple definition of `__float22bfloat162_rn(HIP_vector_type<float, 2u>)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0xa0): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-

We need a patch to fix the compilation issue before the next ROCm release is available.
This pull request fixed the build issue for ROCm build.

[jamestwhedbee]

I am using ROCm 5.7 and before this PR, v0.3.0 of vLLM could not be built. Applying this patch fixes it for me! Just thought it was worth mentioning that this issue was not limited to ROCm 6.0

[hongxiayang]

I am using ROCm 5.7 and before this PR, v0.3.0 of vLLM could not be built. Applying this patch fixes it for me! Just thought it was worth mentioning that this issue was not limited to ROCm 6.0

Thanks for your comment. I will update to apply the patch regardless the version.

[zhuohan123]

LGTM! Thanks for the fix!

[fxmarty]

Thank you @hongxiayang