BlenderBIM build from source on macos (arm64) with make dist exits with Error

[dirkolbrich]

Building the BlenderBim Add-on from source according to the documentation at https://blenderbim.org/docs/devs/installation.html#building-from-source exits with an Error.

System information:

➜  ~ sw_vers
ProductName:	macOS
ProductVersion:	12.5.1
BuildVersion:	21G83
➜  ~ arch
arm64
➜  ~ which python
python not found
➜  ~ which python3
/opt/homebrew/bin/python3
➜  ~ python3 --version
Python 3.10.6

Starting the build from source with the command:

make dist PLATFORM=macos PYVERSION=py310

wrong python command

First try gives an error due to wrong command references in the current /src/blenderbim/Makefile:

# Provides BIMTester functionality
cd dist/working && python -m venv env
/bin/sh: python: command not found
make: *** [dist] Error 127

The Makefile refers in 3 occurrences to a python command, instead of a python3 command. python3 is the default command name for either the system provided as well as the homebrew installed Python v3. This can easily fixed:

diff --git a/src/blenderbim/Makefile b/src/blenderbim/Makefile
index c9dd15b7..dac9b774 100644
--- a/src/blenderbim/Makefile
+++ b/src/blenderbim/Makefile
@@ -138,7 +138,7 @@ endif
        # Provides IFCClash functionality
        cp -r dist/working/IfcOpenShell-0.7.0/src/ifcclash/ifcclash dist/blenderbim/libs/site/packages/
        # Provides BIMTester functionality
-       cd dist/working && python -m venv env
+       cd dist/working && python3 -m venv env
        cd dist/working && . env/bin/activate && pip install pybabel
        cd dist/working && . env/bin/activate && pip install babel
        cd dist/working && . env/bin/activate && ./env/bin/pybabel compile -d ./IfcOpenShell-0.7.0/src/ifcbimtester/bimtester/locale/
@@ -165,7 +165,7 @@ endif
        mkdir dist/working
        cd dist/working && wget https://files.pythonhosted.org/packages/d6/fd/eb8c212053addd941cc90baac307c00ac246ac3fce7166b86434c6eae963/pystache-0.5.4.tar.gz
        cd dist/working && tar -xzvf pystache*
-       cd dist/working/pystache-0.5.4/ && python setup.py build && cp -r build/lib/pystache ../../blenderbim/libs/site/packages/
+       cd dist/working/pystache-0.5.4/ && python3 setup.py build && cp -r build/lib/pystache ../../blenderbim/libs/site/packages/
        rm -rf dist/working

        # Provides SVG export in construction documentation
@@ -283,7 +283,7 @@ endif

        # Required by IFCCityJSON
        mkdir dist/working
-       cd dist/working && python -m venv env
+       cd dist/working && python3 -m venv env
        cd dist/working && mkdir site-packages
        cd dist/working && . env/bin/activate && pip install cjio --target=./site-packages
        cd dist/working/site-packages/ && rm -r *dist-info*

pystache error

Second try after fixing the correct python command errors on a pystache type error:

x pystache-0.5.4/tox.ini
cd dist/working/pystache-0.5.4/ && python3 setup.py build && cp -r build/lib/pystache ../../blenderbim/libs/site/packages/
pystache: using: version '63.4.3' of <module 'setuptools' from '/opt/homebrew/lib/python3.10/site-packages/setuptools/__init__.py'>
Warning: 'classifiers' should be a list, got type 'tuple'
error in pystache setup command: use_2to3 is invalid.
make: *** [dist] Error 1

And indeed, pistache v0.5.4 uses a tuple in it's classifiers, see https://github.com/defunkt/pystache/blob/17a5dfdcd56eb76af731d141de395a7632a905b8/setup.py#L129-L143.
This can be fixed by simply updating pystache to v0.6.0 (which doesn't use Classifiers anymore):

diff --git a/src/blenderbim/Makefile b/src/blenderbim/Makefile
index c9dd15b7..e1bc0f73 100644
--- a/src/blenderbim/Makefile
+++ b/src/blenderbim/Makefile
@@ -163,9 +163,9 @@ endif

        # Provides Mustache templating in construction documentation
        mkdir dist/working
-       cd dist/working && wget https://files.pythonhosted.org/packages/d6/fd/eb8c212053addd941cc90baac307c00ac246ac3fce7166b86434c6eae963/pystache-0.5.4.tar.gz
+       cd dist/working && wget https://files.pythonhosted.org/packages/3f/e7/8750ba6c6101d6aa5ceeb20c013adf2c6f3554a12c71d75654b468404bfa/pystache-0.6.0.tar.gz
        cd dist/working && tar -xzvf pystache*
-       cd dist/working/pystache-0.5.4/ && python setup.py build && cp -r build/lib/pystache ../../blenderbim/libs/site/packages/
+       cd dist/working/pystache-0.6.0/ && python3 setup.py build && cp -r build/lib/pystache ../../blenderbim/libs/site/packages/
        rm -rf dist/working

        # Provides SVG export in construction documentation

patch rejected

The next error is above my mental pay grade, as it deals with file paths.

cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/model_core.py < model_core.patch
patch: **** rejecting file name with ".." component: ../blenderbim/libs/site/packages/behave/model_core.py
make: *** [dist] Error 2

Any idea on that?

[dirkolbrich]

Note: I don't know if changing the python command to python3 break on Linux/Win.

[Moult]

The python vs python3 is a packaging preference, so happy for an if statement to run python3 on Mac, but leave it as python elsewhere. It'd definitely break windows if you changed it globally.

The pystache error looks suspicious as the builds haven't failed. I'll take a closer look tomorrow.

.. Means parent directory. Something seems super suspicious if patch is treating it as a filename. Can you try quoting the entire filepath and or checking if your editor has changed tabs to spaces or something.

Are you also assuming this will give you a working Mac ARM build? Because it won't, its just bundling prebuilt stuff and only devs who want to package a blender add-on would be interested in it (which is almost no one but me?)

[dirkolbrich]

Nope, quoting the file path either with single or with double quotation marks does not solve the error:

diff --git a/src/blenderbim/Makefile b/src/blenderbim/Makefile
index c9dd15b7..b2292864 100644
--- a/src/blenderbim/Makefile
+++ b/src/blenderbim/Makefile
@@ -391,11 +391,11 @@ endif
        # See bug #1294
        cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/model_core.patch
        cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/runner_util.patch
-       cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/model_core.py < model_core.patch
-       cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/runner_util.py < runner_util.patch
+       cd dist/working/ && patch '../blenderbim/libs/site/packages/behave/model_core.py' < model_core.patch
+       cd dist/working/ && patch '../blenderbim/libs/site/packages/behave/runner_util.py' < runner_util.patch
        # See bug #2159
        cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/junit.patch
-       cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/reporter/junit.py < junit.patch
+       cd dist/working/ && patch '../blenderbim/libs/site/packages/behave/reporter/junit.py' < junit.patch
        rm -rf dist/working

error:

cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/model_core.patch
--2022-09-06 15:46:48--  https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/model_core.patch
Auflösen des Hostnamens raw.githubusercontent.com (raw.githubusercontent.com)… 2606:50c0:8001::154, 2606:50c0:8002::154, 2606:50c0:8003::154, ...
Verbindungsaufbau zu raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8001::154|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: 708 [text/plain]
Wird in »model_core.patch« gespeichert.

model_core.patch                             100%[===========================================================================================>]     708  --.-KB/s    in 0s

2022-09-06 15:46:48 (75,0 MB/s) - »model_core.patch« gespeichert [708/708]

cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/runner_util.patch
--2022-09-06 15:46:48--  https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/runner_util.patch
Auflösen des Hostnamens raw.githubusercontent.com (raw.githubusercontent.com)… 2606:50c0:8002::154, 2606:50c0:8003::154, 2606:50c0:8000::154, ...
Verbindungsaufbau zu raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8002::154|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: 622 [text/plain]
Wird in »runner_util.patch« gespeichert.

runner_util.patch                            100%[===========================================================================================>]     622  --.-KB/s    in 0s

2022-09-06 15:46:48 (65,9 MB/s) - »runner_util.patch« gespeichert [622/622]

cd dist/working/ && patch '../blenderbim/libs/site/packages/behave/model_core.py' < model_core.patch
patch: **** rejecting file name with ".." component: ../blenderbim/libs/site/packages/behave/model_core.py
make: *** [dist] Error 2

checking if your editor has changed tabs to spaces or something.

I'm using VS Code. The editor did not changed the first tab in the corresponding line of the source code (apart from my above stated changes to fix errors).

Are you also assuming this will give you a working Mac ARM build? Because it won't, its just bundling prebuilt stuff and only devs who want to package a blender add-on would be interested in it (which is almost no one but me?)

Yes indeed, I did assumed that, as the Heading in the documentation says "Building from source".

[dirkolbrich]

Fixed the patch: **** rejecting file name with ".." component error by using the -d option of patch, that is first switching to the directory and then patching the specified file:

diff --git a/src/blenderbim/Makefile b/src/blenderbim/Makefile
index c9dd15b7..a5f801e0 100644
--- a/src/blenderbim/Makefile
+++ b/src/blenderbim/Makefile
@@ -391,11 +391,11 @@ endif
        # See bug #1294
        cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/model_core.patch
        cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/runner_util.patch
-       cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/model_core.py < model_core.patch
-       cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/runner_util.py < runner_util.patch
+       cd dist/working/ && patch -d ../blenderbim/libs/site/packages/behave model_core.py < model_core.patch
+       cd dist/working/ && patch -d ../blenderbim/libs/site/packages/behave runner_util.py < runner_util.patch
        # See bug #2159
        cd dist/working/ && wget https://raw.githubusercontent.com/IfcOpenShell/IfcOpenShell/v0.7.0/src/ifcbimtester/patch/junit.patch
-       cd dist/working/ && patch ../blenderbim/libs/site/packages/behave/reporter/junit.py < junit.patch
+       cd dist/working/ && patch -d ../blenderbim/libs/site/packages/behave/reporter junit.py < junit.patch
        rm -rf dist/working

The build now runs nearly through, but gives an error at the near end:

rm -rf dist/working
cd dist/blenderbim && sed -i "s/999999/`date '+%y%m%d'`/" __init__.py
sed: 1: "__init__.py": invalid command code _
make: *** [dist] Error 1

The corresponding source code is:

cd dist/blenderbim && sed -i "s/999999/$(VERSION)/" __init__.py

Edit: It seems the $(VERSION) variable is somehow messed up. Got nothing to do with the $(VERSION) variable. Instead sed doesn't recognize __init__,py as a file name.

[dirkolbrich]

Duuuuhhhhhhhh!

sed on macOS does not work as sed on Linux, see https://stackoverflow.com/a/19457213/3358138

diff --git a/src/blenderbim/Makefile b/src/blenderbim/Makefile
index c9dd15b7..26432bf1 100644
--- a/src/blenderbim/Makefile
+++ b/src/blenderbim/Makefile
@@ -523,7 +523,7 @@ endif
        cd dist/working/ && cp -r pkg_resources ../blenderbim/libs/site/packages/
        rm -rf dist/working

-       cd dist/blenderbim && sed -i "s/999999/$(VERSION)/" __init__.py
+       cd dist/blenderbim && sed -i '' -e "s/999999/$(VERSION)/" __init__.py
        cd dist && zip -r blenderbim-$(VERSION)-$(PYVERSION)-$(PLATFORM).zip ./*
        rm -rf dist/blenderbim

[Moult]

Cool, ive been travelling and not able to review these changes on a computer, but I'll be sure to include them! Thanks for helping!

I'll also rename the docs to be "Bundling from source" to better represent what its doing. The first couple paragraphs in that section explain how its not compilation.

Have you seen the Mac compilation instructions ? https://blenderbim.org/docs-python/ifcopenshell/installation.html#compiling-on-macos there have also been success and question issues here you can search in the past about arm if you run into issues.

[Moult]

Looking again at the Github actions output on a larger screen I see the actual error is not the list/tuple classifiers, it's use_2to3, and the problem is upstream and thanks to you I've now learned the original pystache is dead. Upgrading to the fork as you've proposed is the perfect solution!