Setup.sh is broken on linux. And lots of things are broken on linux-aarch64

[fake-name]

Describe the bug

Some of the command escape sequencence in setup.sh are broken. Instead of trying to install cmake and libarrow-dev, it tries (and fails) to install a single package named 'cmake libarrow-dev'.

To Reproduce
Steps to reproduce the behavior:

1. Clone repository
2. Try to run setup.sh or setup_dev.sh

Expected behavior
I'd expect the setup script to set things up.

Screenshots
N/A

Backtrace

+ sudo apt-get update
Hit:1 http://packages.ros.org/ros2/ubuntu focal InRelease
Hit:2 https://repo.download.nvidia.com/jetson/common r35.3 InRelease
Hit:3 https://apache.jfrog.io/artifactory/arrow/ubuntu focal InRelease
Hit:4 http://ports.ubuntu.com/ubuntu-ports focal InRelease
Hit:5 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease
Hit:6 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal InRelease 
Hit:8 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease
Hit:9 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease 
Fetched 15.5 kB in 2s (9,739 B/s)
Reading package lists... Done 
+ sudo apt-get -y install 'cmake libarrow-dev'
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package cmake libarrow-dev

Note that manually executing sudo apt-get -y install 'cmake libarrow-dev' fails. Instead, executing sudo apt-get -y install cmake libarrow-dev works as expected.

Desktop (please complete the following information):

client@box:~/rerun_build/rerun$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
client@box:~/rerun_build/rerun$ uname -a
Linux box 5.10.104-tegra #1 SMP PREEMPT Fri Jul 21 13:41:19 EDT 2023 aarch64 aarch64 aarch64 GNU/Linux
client@box:~/rerun_build/rerun$ bash --version
GNU bash, version 5.0.17(1)-release (aarch64-unknown-linux-gnu)
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
client@box:~/rerun_build/rerun$

Rerun version
Current in git, commit dab0322

Other notes

I'm only bothering with building from source because there are no Aarch64 wheels. All the documentation just says "install from pypi", except there's no source-only wheel, and the pre-baked wheels don't work on aarch64.

[fake-name]

Changing https://github.com/rerun-io/rerun/blob/main/scripts/setup.sh#L38 to double quotes, and removing the quotes entirely from https://github.com/rerun-io/rerun/blob/main/scripts/setup.sh#L46 appear to fix the issue.

[fake-name]

This appears to have been added in #2901. Specifically 1c255e7

[fake-name]

Ok, more stuff. In setup_dev.sh, it tries to install a package named flatbuffers. That package does not exist on ubuntu. Instead I think it's trying to install libflatbuffers-dev.

Changing https://github.com/rerun-io/rerun/blob/main/scripts/setup_dev.sh#L19 from

packagesNeeded='pngcrush pipx clang-format flatbuffers'

to

packagesNeeded='pngcrush pipx clang-format libflatbuffers-dev'

resolves that issue. Note that I also had to install python3.8-venv. That might be safe to assume it's installed, but it should probably be mentioned somewhere. Using the more generic python3-venv is a better idea.

[fake-name]

More notes.

The BUILD.md document comments:

Now you can run the python examples from the repository, given that you're still in the virtual environment.

python examples/python/car/main.py

Except that that file tries to import cv2, and nothing installs OpenCV so the "check if your build is installed" command also does not work.

[fake-name]

More things. So I have almost zero rust experience, and I'm trying to get this package working just based on BUILD.md.

Something in pixi is broken on this particular platform. I'm not sure how it even got installed, but:

(venv) 1 client@box:~/rerun_build/rerun$ pixi run cpp-test
  × The current platform does not satisfy the minimal virtual package requirements
  ╰─▶ the project does not support 'linux-aarch64'

Pixi is YET ANOTHER tool that had the wonderful kindness to chose a COMPLETLY ungooglable name. Urgh. I'm assuming the issue here is the lack of the appropriate target in pixi.toml, but "pixi targets" returns lots of results about a skin creme being sold at Target (the department store) called Pixi.

Projects need to google the name they want to use, and if there are more then maybe like 500 results, NOT USE THE FREAKING NAME.

[fake-name]

OK, patching pixi.toml for the proper platform, now I get to find all the dependencies that don't work on linux-aarch64

So far:

meilisearch
taplo
binaryen

So far, I've just commented them out in the toml file. So far, things work without them, though I expect things to be broken later on.

[abey79]

@fake-name as part of #5511, I'm fixing pixi.toml. You may use that version if you want.

• binaryen was recently updated to support linux-arm64 (needs v117)
• I've moved taplo behind a feature, so it's not required unless you want to use the tasks that depend on it
• I've moved meilisearch behind a target, as it used only on our ci on a specific runner

Note that I also ran into nox being flagged as arch-dependent despite being pure python. I've moved it to [pypi-dependencies] (requires a recent version of pixi).

[abey79]

Also note that our dev builds now have linux-aarch64 CLI builds and C++ SDK: https://github.com/rerun-io/rerun/releases/tag/prerelease

Wheels and lower glibc version requirement for the above will be available once #5511 is merged (which I hope is soon).

[fake-name]

Sweet! Thanks. Sorry about my long rambling blathering.

[fake-name]

Note that that branch still has the setup.sh/setup_dev.sh issues.

I'm quite curious what platform it does work on. I'm stuck on this particular 20.04 release as it's the only distro supported by nvidia on the embedded device I'm working with.

[abey79]

Yes, our setup*.sh are in a really dire state and we need to fix that. #5511 doesn't aim to fix that though.

[abey79]

Ideally, we aim to remove them altogether in favour of pixi:

• Replace our setup*.sh files with pixi #4170