Fix compile error when using rocm

[zhaoyang-star]

fix #2646

[zhuohan123]

LGTM! Thanks for the fix!

[hongxiayang]

Thanks for the effort.
I still got error. Can you continue working on fixing below errors?

g++ -pthread -B /opt/conda/envs/py_3.8/compiler_compat -Wl,--sysroot=/ -pthread -shared -B /opt/conda/envs/py_3.8/compiler_compat -L/opt/conda/envs/py_3.8/lib -Wl,-rpath=/opt/conda/envs/py_3.8/lib -Wl,--no-as-needed -Wl,--sysroot=/ /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/activation_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/hip_utils_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/layernorm_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/moe_align_block_size_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/pos_encoding_kernels.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/pybind.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/quantization/gptq/q_gemm.o /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/quantization/squeezellm/quant_hip_kernel.o -L/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib -L/opt/rocm/lib -L/opt/rocm/hip/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lamdhip64 -lc10_hip -ltorch_hip -o build/lib.linux-x86_64-cpython-38/vllm/_C.cpython-38-x86_64-linux-gnu.so
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__float2bfloat16(float)':
cache_kernels.hip:(.text+0x0): multiple definition of `__float2bfloat16(float)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x0): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__bfloat1622float2(__hip_bfloat162)':
cache_kernels.hip:(.text+0x40): multiple definition of `__bfloat1622float2(__hip_bfloat162)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x40): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__double2bfloat16(double)':
cache_kernels.hip:(.text+0x60): multiple definition of `__double2bfloat16(double)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x60): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__float22bfloat162_rn(HIP_vector_type<float, 2u>)':
cache_kernels.hip:(.text+0xa0): multiple definition of `__float22bfloat162_rn(HIP_vector_type<float, 2u>)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0xa0): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__high2float(__hip_bfloat162)':
cache_kernels.hip:(.text+0x110): multiple definition of `__high2float(__hip_bfloat162)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x110): first defined here
/opt/conda/envs/py_3.8/compiler_compat/ld: /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/cache_kernels.o: in function `__low2float(__hip_bfloat162)':
cache_kernels.hip:(.text+0x120): multiple definition of `__low2float(__hip_bfloat162)'; /app/vllm/build/temp.linux-x86_64-cpython-38/csrc/attention/attention_kernels.o:attention_kernels.hip:(.text+0x120): first defined here
collect2: error: ld returned 1 exit status
error: command '/usr/bin/g++' failed with exit code 1

[jamestwhedbee]

I am also still running into this

[hongxiayang]

My pull request to fix this: #2790