[Bug]:

[cheng358]

OpenVINO Version

AttributeError: module 'torch._C' has no attribute 'OutOfMemoryError'

Operating System

Ubuntu 18.04 (LTS)

Device used for inference

CPU

Framework

None

Model used

No response

Issue description

I install vLLM with OpenVINO backend use cmd : PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu" VLLM_TARGET_DEVICE=openvino python -m pip install -v .
follow error occurred：
Traceback (most recent call last):
File "/mnt/new_afs/demo/vllm_vino/vino/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in
main()
File "/mnt/new_afs/demo/vllm_vino/vino/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/mnt/new_afs/demo/vllm_vino/vino/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-en0zck1y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
File "/tmp/pip-build-env-en0zck1y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-en0zck1y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in run_setup
exec(code, locals())
File "", line 12, in
File "/tmp/pip-build-env-en0zck1y/overlay/lib/python3.10/site-packages/torch/init.py", line 1954, in
_C._initExtension(_manager_path())
File "/tmp/pip-build-env-en0zck1y/overlay/lib/python3.10/site-packages/torch/cuda/init.py", line 272, in
OutOfMemoryError = torch._C.OutOfMemoryError
AttributeError: module 'torch._C' has no attribute 'OutOfMemoryError'
error: subprocess-exited-with-error

Step-by-step reproduction

No response

Relevant log output

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[Aznie-Intel]

To resolve the issue you're facing while installing vLLM with OpenVINO, follow these steps:

1. Check PyTorch Compatibility: Ensure your PyTorch version is compatible with both vLLM and OpenVINO. You may need to install a specific version like torch==1.13.1.
2. Check Dependencies: Run pip check to see if there are any dependency issues.
3. Verify OpenVINO Installation: Make sure OpenVINO is installed and set up properly. Run the setup script (source /opt/intel/openvino/bin/setupvars.sh) if necessary.
4. Reinstall PyTorch and vLLM: Uninstall and reinstall both PyTorch and vLLM to fix any potential issues with the installations.
5. Set CUDA Environment Variable: If you’re using only the CPU or OpenVINO, ensure CUDA is not being used by setting export CUDA_VISIBLE_DEVICES="".
6. Verbose Logging: Run the installation with pip install -v . to gather more information about the error.

Let me know if the problem persists after trying these steps.