Cannot pip install v0.18 on x86_64 arch with OSX 14 and Python 3.9

[nicklofaso]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for main branch).

Steps to reproduce the issue

I pip installed v0.18 Open3D:

python3.9 -m pip install open3d==0.18.0

Then, I tried to print out open3d version

open3d --version

Then, I inspected pybind.cpython-39-darwin.so

> file ${WORKSPACE}/env/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so

${WORKSPACE}/env/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so: Mach-O 64-bit bundle arm64

Error message

$ open3d --version
Traceback (most recent call last):
  File "${WORKSPACE}/env/bin/open3d", line 5, in <module>
    from open3d.tools.cli import main
  File "${WORKSPACE}/env/lib/python3.9/site-packages/open3d/__init__.py", line 93, in <module>
    from open3d.cpu.pybind import (core, camera, data, geometry, io, pipelines,
ImportError: dlopen(${WORKSPACE}/env/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so, 0x0002): tried: '${WORKSPACE}/env/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS${WORKSPACE}/env/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so' (no such file), '${WORKSPACE}/env/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64'))

### Open3D, Python and System information

```markdown
- Operating system: macOS 14.2.1
- Python version: Python 3.9
- Open3D version: 0.18
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
- Compiler version (if built from source): n/a

Additional information

Pip logs showed that it downloaded open3d-0.18.0-cp39-cp39-macosx_13_0_universal2.whl from pypi. Based on my inspection of pybind.cpython-39-darwin.so it seems like the wheel incorrectly installed the arm64 version or it does not contain the x86_64 version of open3d?

[tavurth]

Confirming this issue, just saw the same thing myself

ImportError: dlopen(/usr/local/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so, 0x0002): 

tried: '/usr/local/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so' 
(mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64')), 

'/System/Volumes/Preboot/Cryptexes/OS/usr/local/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so'
(no such file), '/usr/local/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so' 
(mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64'))

When downloading the .whl file directly it will not install:

▶ pip install open3d-0.18.0-cp310-cp310-macosx_11_0_x86_64.whl

DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621

ERROR: open3d-0.18.0-cp310-cp310-macosx_11_0_x86_64.whl is not a supported wheel on this platform.

[sitic]

Do the latest development wheels work?

[tavurth]

https://storage.googleapis.com/open3d-releases/python-wheels/open3d-0.18.0+0cf605f-cp39-cp39-macosx_11_0_x86_64.whl

They do not exist @sitic

[sitic]

Hmm, the docs seem to be stuck on an old commit (0cf605f) for which the wheels got deleted. Here is the working link for the latest commit (aba7214):

https://storage.googleapis.com/open3d-releases/python-wheels/open3d-0.18.0+aba7214-cp39-cp39-macosx_11_0_x86_64.whl

Created #6694 for the docs issue.

[tavurth]

That works ok!

▶ pip install open3d-0.18.0+aba7214-cp39-cp39-macosx_11_0_x86_64.whl DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621 Processing ./open3d-0.18.0+aba7214-cp39-cp39-macosx_11_0_x86_64.whl Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (1.20.1) Requirement already satisfied: dash>=2.6.0 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (2.9.2) Requirement already satisfied: werkzeug>=2.2.3 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (2.2.3) Requirement already satisfied: nbformat>=5.7.0 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (5.7.0) Requirement already satisfied: configargparse in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (1.5.3) Requirement already satisfied: addict in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (2.4.0) Requirement already satisfied: pillow>=9.3.0 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (9.5.0) Requirement already satisfied: matplotlib>=3 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (3.3.4) Requirement already satisfied: pandas>=1.0 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (1.2.2) Requirement already satisfied: pyyaml>=5.4.1 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (6.0) Requirement already satisfied: scikit-learn>=0.21 in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (1.2.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (4.65.0) Requirement already satisfied: pyquaternion in /usr/local/lib/python3.9/site-packages (from open3d==0.18.0+aba7214) (0.9.9) Requirement already satisfied: Flask>=1.0.4 in /usr/local/lib/python3.9/site-packages (from dash>=2.6.0->open3d==0.18.0+aba7214) (2.2.3) Requirement already satisfied: plotly>=5.0.0 in /usr/local/lib/python3.9/site-packages (from dash>=2.6.0->open3d==0.18.0+aba7214) (5.14.0) Requirement already satisfied: dash-html-components==2.0.0 in /usr/local/lib/python3.9/site-packages (from dash>=2.6.0->open3d==0.18.0+aba7214) (2.0.0) Requirement already satisfied: dash-core-components==2.0.0 in /usr/local/lib/python3.9/site-packages (from dash>=2.6.0->open3d==0.18.0+aba7214) (2.0.0) Requirement already satisfied: dash-table==5.0.0 in /usr/local/lib/python3.9/site-packages (from dash>=2.6.0->open3d==0.18.0+aba7214) (5.0.0) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/site-packages (from matplotlib>=3->open3d==0.18.0+aba7214) (0.10.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/site-packages (from matplotlib>=3->open3d==0.18.0+aba7214) (1.3.1) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /Users/will/Library/Python/3.9/lib/python/site-packages (from matplotlib>=3->open3d==0.18.0+aba7214) (2.4.7) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.9/site-packages (from matplotlib>=3->open3d==0.18.0+aba7214) (2.8.1) Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.9/site-packages (from nbformat>=5.7.0->open3d==0.18.0+aba7214) (2.16.3) Requirement already satisfied: jsonschema>=2.6 in /Users/will/Library/Python/3.9/lib/python/site-packages (from nbformat>=5.7.0->open3d==0.18.0+aba7214) (3.2.0) Requirement already satisfied: jupyter-core in /usr/local/lib/python3.9/site-packages (from nbformat>=5.7.0->open3d==0.18.0+aba7214) (4.7.1) Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.9/site-packages (from nbformat>=5.7.0->open3d==0.18.0+aba7214) (5.9.0) Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.9/site-packages (from pandas>=1.0->open3d==0.18.0+aba7214) (2021.1) Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.9/site-packages (from scikit-learn>=0.21->open3d==0.18.0+aba7214) (1.10.1) Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.9/site-packages (from scikit-learn>=0.21->open3d==0.18.0+aba7214) (1.2.0) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.9/site-packages (from scikit-learn>=0.21->open3d==0.18.0+aba7214) (2.1.0) Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.9/site-packages (from werkzeug>=2.2.3->open3d==0.18.0+aba7214) (2.1.2) Requirement already satisfied: six in /usr/local/lib/python3.9/site-packages (from cycler>=0.10->matplotlib>=3->open3d==0.18.0+aba7214) (1.15.0) Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.9/site-packages (from Flask>=1.0.4->dash>=2.6.0->open3d==0.18.0+aba7214) (3.1.2) Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.9/site-packages (from Flask>=1.0.4->dash>=2.6.0->open3d==0.18.0+aba7214) (2.1.2) Requirement already satisfied: click>=8.0 in /usr/local/lib/python3.9/site-packages (from Flask>=1.0.4->dash>=2.6.0->open3d==0.18.0+aba7214) (8.0.1) Requirement already satisfied: importlib-metadata>=3.6.0 in /usr/local/lib/python3.9/site-packages (from Flask>=1.0.4->dash>=2.6.0->open3d==0.18.0+aba7214) (6.1.0) Requirement already satisfied: attrs>=17.4.0 in /Users/will/Library/Python/3.9/lib/python/site-packages (from jsonschema>=2.6->nbformat>=5.7.0->open3d==0.18.0+aba7214) (20.3.0) Requirement already satisfied: pyrsistent>=0.14.0 in /Users/will/Library/Python/3.9/lib/python/site-packages (from jsonschema>=2.6->nbformat>=5.7.0->open3d==0.18.0+aba7214) (0.17.3) Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (from jsonschema>=2.6->nbformat>=5.7.0->open3d==0.18.0+aba7214) (65.6.3) Requirement already satisfied: tenacity>=6.2.0 in /usr/local/lib/python3.9/site-packages (from plotly>=5.0.0->dash>=2.6.0->open3d==0.18.0+aba7214) (8.2.2) Requirement already satisfied: packaging in /usr/local/lib/python3.9/site-packages (from plotly>=5.0.0->dash>=2.6.0->open3d==0.18.0+aba7214) (23.0) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/site-packages (from importlib-metadata>=3.6.0->Flask>=1.0.4->dash>=2.6.0->open3d==0.18.0+aba7214) (3.15.0) Installing collected packages: open3d Attempting uninstall: open3d Found existing installation: open3d 0.17.0 Uninstalling open3d-0.17.0: Successfully uninstalled open3d-0.17.0

I guess in that case this could be related to the other issue, as pip before was not finding anything to download 🤔
Or perhaps it downloaded it, but did not link to the correct location.

tried: '/usr/local/lib/python3.9/site-packages/open3d/cpu/pybind.cpython-39-darwin.so' 
(mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64')), 

[rossng]

I also encountered this issue. Currently-published wheels with universal2 in their platform tag do not contain a universal version of the pybind.cpython-3*-darwin.so library.

open3d-0.18.0-cp39-cp39-macosx_13_0_universal2/open3d/cpu $ file -b pybind.cpython-39-darwin.so              
Mach-O 64-bit bundle arm64

open3d-0.18.0-cp311-cp311-macosx_10_15_universal2/open3d/cpu $ file -b pybind.cpython-311-darwin.so
Mach-O 64-bit bundle x86_64

I checked with the latest universal development wheel for Python 3.11 too, same problem:

open3d-0.18.0+aba7214-cp311-cp311-macosx_10_15_universal2/open3d/cpu $ file -b pybind.cpython-311-darwin.so
Mach-O 64-bit bundle x86_64

I poked around and this seems to apply to all the other dylibs in the wheels too.

Just to compare, here's what the output for a universal .so should look like:

PyNaCl-1.5.0-cp36-abi3-macosx_10_10_universal2/nacl $ file -b _sodium.abi3.so
Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64] [arm64:Mach-O 64-bit bundle arm64]
_sodium.abi3.so (for architecture x86_64):      Mach-O 64-bit bundle x86_64
_sodium.abi3.so (for architecture arm64):       Mach-O 64-bit bundle arm64

[rossng]

Hmm, it might just be a case of setting something like -DCMAKE_OSX_ARCHITECTURES:STRING="x86_64;arm64" when compiling. Hopefully I'll get a chance to test this tomorrow.

[sitic]

The macOS dev wheels are indeed only build on x64 and incorrectly named. I plan to fix this in

• Add macOS arm64 CI runner, fix macOS arm64 builds #6695

The arm64 macOS wheels on PyPi are currently built manually. The artifacts of that PR CI job has the most recent open3d_macosx_ARM64_wheels.

[rossng]

Thanks for working on this! I had been wondering how the arm64 wheels were being built for distribution.

I had a look at the artifacts from that CI job and it looks like the libs in the universal2 wheels are built for arm64 only. Perhaps the output names should then change from (for example) open3d-0.18.0+48ca4db-cp311-cp311-macosx_11_0_universal2.whl to open3d-0.18.0+48ca4db-cp311-cp311-macosx_11_0_arm64.whl

[JennySeidenschwarz]

Hey! I cant install v0.18 on mac since it is apparently not available

python -m pip install open3d==0.18.0 
ERROR: Could not find a version that satisfies the requirement open3d==0.18.0 (from versions: 0.16.0, 0.16.1)
ERROR: No matching distribution found for open3d==0.18.0

I guess this might be the same issue?

macOS: 14.4.1
Python: 3.10.13
Processor: X86_64

Thanks!!

[csyhping]

Hey! I cant install v0.18 on mac since it is apparently not available

python -m pip install open3d==0.18.0 
ERROR: Could not find a version that satisfies the requirement open3d==0.18.0 (from versions: 0.16.0, 0.16.1)
ERROR: No matching distribution found for open3d==0.18.0

I guess this might be the same issue?

macOS: 14.4.1 Python: 3.10.13 Processor: X86_64

Thanks!!

same problem

[jloveric]

Seeing this issue on ubuntu 24.04 which uses python12.