[Misc] Avoid importing `nixl_ep` on every `vllm serve` config by NickLucche · Pull Request #50879 · vllm-project/vllm

NickLucche · 2026-08-03T14:22:36Z

Avoid importing nixl_ep all the time we boot up vllm. Instead, only do it lazily when needed (dpep configuration).

Right now if you do vllm serve ... import_utils will try to resolve nixl_ep, and in this case even log "unrelated" import failures

APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408] Module nixl_ep was found but failed to import
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408] Traceback (most recent call last):
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "/mnt/vast/home/nicolo.lucchesi/upstream/vllm/vllm/utils/import_utils.py", line 406, in _has_module
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]     importlib.import_module(module_name)
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "/mnt/vast/uv-pythons/pythons/cpython-3.12.13-linux-x86_64-gnu/lib/python3.12/importlib/__init__.py", line 90, in import_module
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]     return _bootstrap._gcd_import(name[level:], package, level)
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]   File "/mnt/vast/home/nicolo.lucchesi/upstream/vllm/.venv/lib/python3.12/site-packages/nixl_ep/__init__.py", line 23, in <module>
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408]     from . import nixl_ep_cpp as _nixl_ep_cpp
(APIServer pid=148679) WARNING 07-30 16:41:16 [import_utils.py:408] ImportError: libcudart.so.12: cannot open shared object file: No such file or directory

as this configuration can then go on and run just fine, as no DPEP is needed (I have no libcudart 12 on this machine, but it only matters if it's actually needed).

cc @yewentao256 @DarkLight1337

Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

yewentao256

LGTM, thanks for the work!

fxmarty-amd · 2026-08-05T09:38:16Z

@NickLucche FYI, this PR broke some CI tests on mi300, unexpectedly (and fixed by #51110):

e.g. VLLM_ROCM_USE_AITER=1 HIP_VISIBLE_DEVICES=1 pytest tests/quantization/test_online.py::test_online_quantization[True-fp8_per_tensor-None-Fp8PerTensorOnlineLinearMethod-Fp8PerTensorOnlineMoEMethod] -s -vvvvv

(EngineCore pid=43826) [aiter] import [module_quant] under /usr/local/lib/python3.12/dist-packages/aiter/jit/module_quant.so
(EngineCore pid=43826) [aiter] type hints mismatch, override to --> dynamic_per_tensor_quant(arg0: aiter_tensor_t, arg1: aiter_tensor_t, arg2: aiter_tensor_t) -> None
(EngineCore pid=43826) [aiter] import [module_moe_asm] under /usr/local/lib/python3.12/dist-packages/aiter/jit/module_moe_asm.so
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] EngineCore failed to start.
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] Traceback (most recent call last):
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/engine/core.py", line 1314, in run_engine_core
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/engine/core.py", line 1070, in __init__
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     super().__init__(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/engine/core.py", line 143, in __init__
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     kv_cache_config = self._initialize_kv_caches(vllm_config)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/engine/core.py", line 293, in _initialize_kv_caches
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     available_gpu_memory = self.model_executor.determine_available_memory()
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/executor/abstract.py", line 147, in determine_available_memory
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self.collective_rpc("determine_available_memory")
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/executor/uniproc_executor.py", line 92, in collective_rpc
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     result = run_method(self.driver_worker, method, args, kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/serial_utils.py", line 510, in run_method
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/gpu_worker.py", line 504, in determine_available_memory
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     self.model_runner.profile_run()
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/gpu/model_runner.py", line 737, in profile_run
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     hidden_states, sample_hidden_states = self._dummy_run(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                                           ^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/gpu/eplb_utils.py", line 38, in wrapper
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     result = fn(self, *args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]              ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/gpu/model_runner.py", line 631, in _dummy_run
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     self.execute_model(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return func(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/gpu/model_runner.py", line 1441, in execute_model
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     model_output = self.model(**model_inputs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self._call_impl(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1790, in _call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return forward_call(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/models/granitemoe.py", line 525, in forward
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     hidden_states = self.model(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                     ^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/compilation/decorators.py", line 507, in __call__
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self.forward(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/models/granitemoe.py", line 332, in forward
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     hidden_states = layer(positions, hidden_states)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self._call_impl(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1790, in _call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return forward_call(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/models/granitemoe.py", line 279, in forward
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     hidden_states = self.block_sparse_moe(hidden_states)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self._call_impl(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1790, in _call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return forward_call(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/models/granitemoe.py", line 127, in forward
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     final_hidden_states = self.experts(hidden_states, router_logits)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self._call_impl(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1790, in _call_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return forward_call(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/runner/moe_runner.py", line 721, in forward
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     result = self._forward_entry(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]              ^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1269, in __call__
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self._op(*args, **kwargs)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/runner/moe_runner.py", line 127, in _moe_forward
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return layer._forward_impl(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/runner/moe_runner.py", line 881, in _forward_impl
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     shared_output, hidden_states = self._apply_quant_method(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/runner/moe_runner.py", line 610, in _apply_quant_method
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     fused_out = self.routed_experts.forward_modular(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/routed_experts.py", line 1227, in forward_modular
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self.quant_method.apply(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/quantization/online/moe_base.py", line 151, in apply
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self.moe_kernel.apply(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/modular_kernel.py", line 1714, in apply
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     return self.impl.apply(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]            ^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/modular_kernel.py", line 1494, in apply
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     fused_out = self._fused_experts(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                 ^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/modular_kernel.py", line 1308, in _fused_experts
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     workspace13, workspace2, fused_out = self._allocate_buffers(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                                          ^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/model_executor/layers/fused_moe/modular_kernel.py", line 1167, in _allocate_buffers
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     common_workspace, workspace2 = current_workspace_manager().get_simultaneous(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/workspace.py", line 110, in get_simultaneous
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     current_workspace = self._ensure_workspace_size(total_bytes)
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]   File "/felmarty/repos/vllm/vllm/v1/worker/workspace.py", line 176, in _ensure_workspace_size
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]     self._current_workspaces[ubatch_id] = torch.empty(
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345]                                           ^^^^^^^^^^^^
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] torch.AcceleratorError: CUDA error: invalid configuration argument
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] Search for `hipErrorInvalidConfiguration' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] For debugging consider passing AMD_SERIALIZE_KERNEL=3
(EngineCore pid=43826) ERROR 08-05 09:30:24 [core.py:1345] Device-side assertion tracking was not enabled by user.

fxmarty-amd · 2026-08-05T12:18:27Z

Nothing is wrong per-se here: #51151

Surprising the CI did not catch this though, are there no ROCm + AITER tests running on PRs?

AndreasKaratzas · 2026-08-05T16:00:30Z

@fxmarty-amd CI caught the issue and the AMD test that was supposed to fail indeed failed.

AndreasKaratzas · 2026-08-05T16:05:11Z

@NickLucche we ll need to revert this PR. Cause it's breaking ROCm.
cc @dllehr-amd

…50879) (#51176) Signed-off-by: Felix Marty <Felix.Marty@amd.com>

init

ea07d12

Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>

NickLucche requested review from mgoin, pavanimajety and zyongye as code owners August 3, 2026 14:22

claude Bot reviewed Aug 3, 2026

View reviewed changes

NickLucche added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 3, 2026

mergify Bot added the kv-connector label Aug 3, 2026

tlrmchlsmth approved these changes Aug 3, 2026

View reviewed changes

yewentao256 approved these changes Aug 3, 2026

View reviewed changes

njhill enabled auto-merge (squash) August 3, 2026 21:32

Merge branch 'main' into do-not-import-nixl-all-the-times

514918c

vllm-bot merged commit 7b50d2c into vllm-project:main Aug 4, 2026
104 of 108 checks passed

This was referenced Aug 5, 2026

[ROCm][Bugfix] Restore guarded eager Mori EP import #51110

Closed

fix(jit): isolate AITER extensions from HIP interposers ROCm/aiter#4566

Open

fxmarty-amd mentioned this pull request Aug 5, 2026

[Bug][Regression] #50879 causes wrongful hipMalloc to be loaded on ROCm due to tilelang load, causing tests and allocation failures #51151

Closed

1 task

jimmy-adams mentioned this pull request Aug 5, 2026

Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel #47972

Merged

fxmarty-amd mentioned this pull request Aug 5, 2026

[ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang #51159

Merged

fxmarty-amd mentioned this pull request Aug 5, 2026

Revert [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) #51176

Merged

vllm-bot pushed a commit that referenced this pull request Aug 5, 2026

Revert [Misc] Avoid importing nixl_ep on every vllm serve config (#…

bc37fc9

…50879) (#51176) Signed-off-by: Felix Marty <Felix.Marty@amd.com>

JohnQinAMD mentioned this pull request Aug 6, 2026

[JIT] AITER extensions can bind HIP symbols from an earlier global TileLang stub ROCm/aiter#4585

Open

lazypool mentioned this pull request Aug 6, 2026

[Feature]: vllm.utils.import_util._has_module should not import modules #51162

Open

1 task

Navigation Menu

Uh oh!

Uh oh!

[Misc] Avoid importing `nixl_ep` on every `vllm serve` config - #50879

[Misc] Avoid importing `nixl_ep` on every `vllm serve` config#50879
vllm-bot merged 2 commits into
vllm-project:mainfrom
mistralai:do-not-import-nixl-all-the-times

NickLucche commented Aug 3, 2026

Uh oh!

claude Bot left a comment

Uh oh!

yewentao256 left a comment

Uh oh!

Uh oh!

fxmarty-amd commented Aug 5, 2026 •

edited

Loading

Uh oh!

fxmarty-amd commented Aug 5, 2026

Uh oh!

AndreasKaratzas commented Aug 5, 2026

Uh oh!

AndreasKaratzas commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Uh oh!

Conversation

NickLucche commented Aug 3, 2026

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

yewentao256 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fxmarty-amd commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fxmarty-amd commented Aug 5, 2026

Uh oh!

AndreasKaratzas commented Aug 5, 2026

Uh oh!

AndreasKaratzas commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

fxmarty-amd commented Aug 5, 2026 •

edited

Loading