fix ROCm/HIP support for large memory allocation

[YangWang92]

Due to internal constraints in ROCm/HIP, virtual-memory allocation must be performed chunk-wise for large memory allocation. In addition, avoid using ROCm 6.4.0, an internal bug prevents memory from being released. This commit follows the approach introduced in vllm-project/vllm#12695.

EDIT by tom to make github happy:
Close #9

[fzyzcjy]

Great job! Firstly one thought

[fzyzcjy]

wondering is it possible we unify the CUDA and HIP versions to avoid code duplication

[YangWang92]

It is possbile to reuse CUDA and HIP versions, but current HIP version requires the chunk-wise allocation. Should I merge them?

[YangWang92]

Here is the example from vllm's pull https://github.com/vllm-project/vllm/pull/12695/files#diff-80ab2c32370c400e78803f0ea6c73ec130cf9fb6050ec2728de56bea731cf880 https://github.com/vllm-project/vllm/pull/12695/files#diff-b8401ec6ffbf88a5594fa627fddbf2e3ca5742318c6882520f99d3fc211cbd3e

[fzyzcjy]

As is discussed in DM, wondering whether it is possible to make a layered cake - one low-level layer that provides a unified API and hides diffs from CUDA and HIP, and a high-level layer that consumes this