Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions vllm_ascend/platform.py
Original file line number Diff line number Diff line change
Expand Up @@ -395,14 +395,15 @@ def check_and_update_config(cls, vllm_config: VllmConfig) -> None:
# For example: "page_size:1g" + ",expandable_segments:True".
# NOTE: `max_split_size_mb` or `garbage_collection_threshold` cannot
# be enabled together with `expandable_segments=True`.
if "expandable_segments" not in npu_alloc_configs and \
"max_split_size_mb" not in npu_alloc_configs and \
"garbage_collection_threshold" not in npu_alloc_configs:
if (
"expandable_segments" not in npu_alloc_configs
and "max_split_size_mb" not in npu_alloc_configs
and "garbage_collection_threshold" not in npu_alloc_configs
):
npu_alloc_configs += ",expandable_segments:True"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current logic for appending ,expandable_segments:True can lead to a malformed environment variable if PYTORCH_NPU_ALLOC_CONF is an empty string. In that case, npu_alloc_configs would become ',expandable_segments:True', which is likely incorrect due to the leading comma. It's better to handle the case where npu_alloc_configs is empty separately.

Suggested change
npu_alloc_configs += ",expandable_segments:True"
if npu_alloc_configs:
npu_alloc_configs += ",expandable_segments:True"
else:
npu_alloc_configs = "expandable_segments:True"

os.environ["PYTORCH_NPU_ALLOC_CONF"] = npu_alloc_configs
logger.info("Set PYTORCH_NPU_ALLOC_CONF=%s", npu_alloc_configs)


@classmethod
def import_kernels(cls) -> None:
# Directly importing vllm_ascend_C prevents ASCEND_RT_VISIBLE_DEVICES
Expand Down
Loading