Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docker/Dockerfile.npu
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
ARG VLLM_ASCEND_IMAGE=quay.io/atlas-ci/vllm-ascend
ARG VLLM_ASCEND_VERSION=v0.31.0
ARG VLLM_ASCEND_VERSION=v0.32.0
ARG VLLM_ASCEND_DEVICE_SUFFIX=
FROM ${VLLM_ASCEND_IMAGE}:${VLLM_ASCEND_VERSION}${VLLM_ASCEND_DEVICE_SUFFIX}

ARG APP_DIR=/vllm-workspace/vllm-omni
WORKDIR ${APP_DIR}
COPY . .

RUN apt-get update && \
apt-get install -y --no-install-recommends libjemalloc2 && \
rm -rf /var/lib/apt/lists/*

# Avoid glibc heap corruption at Python shutdown with the NPU runtime.
ENV LD_PRELOAD=libjemalloc.so.2

RUN export PIP_EXTRA_INDEX_URL=https://mirrors.huaweicloud.com/ascend/repos/pypi && \
source /usr/local/Ascend/ascend-toolkit/set_env.sh && \
source /usr/local/Ascend/nnal/atb/set_env.sh && \
Expand Down
7 changes: 6 additions & 1 deletion docker/Dockerfile.npu.ci
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ARG VLLM_ASCEND_IMAGE=quay.io/atlas-ci/vllm-ascend
ARG VLLM_ASCEND_VERSION=v0.31.0
ARG VLLM_ASCEND_VERSION=v0.32.0
ARG VLLM_ASCEND_DEVICE_SUFFIX=
FROM ${VLLM_ASCEND_IMAGE}:${VLLM_ASCEND_VERSION}${VLLM_ASCEND_DEVICE_SUFFIX}

Expand All @@ -14,6 +14,7 @@ RUN apt-get update && \
libgl1 \
libglib2.0-0 \
libgomp1 \
libjemalloc2 \
libsm6 \
libxext6 \
libxrender1 \
Expand All @@ -22,13 +23,17 @@ RUN apt-get update && \
rm -rf /var/cache/apt/* && \
rm -rf /var/lib/apt/lists/*

# Avoid glibc heap corruption at Python shutdown with the NPU runtime.
ENV LD_PRELOAD=libjemalloc.so.2

RUN python3 -m pip install -U pip && \
python3 -m pip install -v \
"mindiesd==3.1.0" \
"pytest>=7.0.0" \
"pytest-asyncio>=0.21.0" \
"pytest-cov>=4.0.0" \
"pytest-mock>=3.10.0" \
"pytest-xdist==3.8.0" \
"openpyxl>=3.0.0" \
"psutil>=7.2.0" \
"soundfile>=0.13.1" \
Expand Down
10 changes: 5 additions & 5 deletions docs/getting_started/installation/npu/npu.inc.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,25 +10,25 @@ The recommended way to use vLLM-Omni on NPU is through the vLLM-Ascend pre-built
=== "A2"

```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.31.0
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.30.0
```

=== "A3"

```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.31.0-a3
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.30.0-a3
```

=== "A5"

```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.31.0-a5
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.30.0-a5
```

=== "310P (Experimental)"

```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.31.0-310p
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.30.0-310p
```

```bash
Expand Down Expand Up @@ -73,7 +73,7 @@ We are keeping [issue #886](https://github.com/vllm-project/vllm-omni/issues/886

You can also install vLLM-Omni from source if you want to use the latest features
or bug fixes. This alignment checkout targets vLLM 0.32.0 and requires a
matching vLLM-Ascend runtime validated on your NPU. The v0.31.0 Ascend images
matching vLLM-Ascend runtime validated on your NPU. The v0.30.0 Ascend images
above do not establish compatibility with this vLLM 0.32.0 checkout. Check
[issue #886](https://github.com/vllm-project/vllm-omni/issues/886) for aligned
versions and NPU validation before selecting a development image.
Expand Down
1 change: 1 addition & 0 deletions tests/platforms/npu/test_contiguous_kv_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ def test_contiguous_kv_cache_layout_v2(monkeypatch: pytest.MonkeyPatch, padded:
assert original(None, None, spec) is (not padded)
config = make_kv_cache_config(name, spec)
vllm_config = MagicMock()
vllm_config.parallel_config.decode_context_parallel_size = 1
vllm_config.kv_transfer_config = None
vllm_config.quant_config = None
monkeypatch.setattr(attn_utils, "get_current_vllm_config", lambda: vllm_config)
Expand Down
Loading