Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARM binaries do not work on 64-bit ARM ubuntu (Raspberry Pi 4) #3383

Closed
gclawes opened this issue Feb 26, 2020 · 4 comments
Closed

ARM binaries do not work on 64-bit ARM ubuntu (Raspberry Pi 4) #3383

gclawes opened this issue Feb 26, 2020 · 4 comments

Comments

@gclawes
Copy link

gclawes commented Feb 26, 2020

What happened: ARM binaries do not run on 64-bit Ubuntu running on a Raspberry Pi 4

What you expected to happen: Binaries can be exectued

How to reproduce it (as minimally and precisely as possible):

ubuntu@pi01:~/teleport$ ls -lah
total 95M
drwxrwxr-x  3 ubuntu ubuntu 4.0K Feb 21 15:35 .
drwxr-xr-x  6 ubuntu ubuntu 4.0K Feb 26 00:30 ..
-rw-rw-r--  1 ubuntu ubuntu  60K Feb 21 15:35 CHANGELOG.md
-rw-rw-r--  1 ubuntu ubuntu 8.1K Feb 21 15:35 README.md
-rw-rw-r--  1 ubuntu ubuntu    7 Feb 21 15:35 VERSION
drwxrwxr-x 12 ubuntu ubuntu 4.0K Feb 21 15:35 examples
-rwxrwxr-x  1 ubuntu ubuntu  483 Feb 21 15:35 install
-rwxrwxr-x  1 ubuntu ubuntu  32M Feb 21 15:35 tctl
-rwxrwxr-x  1 ubuntu ubuntu  42M Feb 21 15:35 teleport
-rwxrwxr-x  1 ubuntu ubuntu  22M Feb 21 15:35 tsh
ubuntu@pi01:~/teleport$ cat VERSION
v4.2.3
ubuntu@pi01:~/teleport$ ./tsh
-bash: ./tsh: No such file or directory
ubuntu@pi01:~/teleport$ file tsh
tsh: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, Go BuildID=loIAEgzMzAh5YmfjehQr/m8eZoEzX4mavUHlw9Dvh/TsBtUbDo5EEM3df4dbY3/H-fKAJwbrHQAi_-y08vR, BuildID[sha1]=160a0d8fc7ddbff40885461220b3e379de40a4cc, stripped

Environment:

  • Teleport version (use teleport version): v4.2.3
  • Tsh version (use tsh version): v4.2.3
  • OS (e.g. from /etc/os-release):
ubuntu@pi01:~/teleport$ uname -a
Linux pi01.pico 5.3.0-1018-raspi2 #20-Ubuntu SMP Mon Feb 3 19:45:46 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@pi01:~/teleport$ cat /etc/os-release
NAME="Ubuntu"
VERSION="19.10 (Eoan Ermine)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 19.10"
VERSION_ID="19.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=eoan
UBUNTU_CODENAME=eoan
ubuntu@pi01:~/teleport$ cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 1
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 2
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 3
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

Hardware        : BCM2835
Revision        : c03112
Serial          : 10000000fc316a97
Model           : Raspberry Pi 4 Model B Rev 1.2

NOTE: This is related to my personal use of Teleport Community Edition, not my employer's use of Teleport Enterprise.

@webvictim
Copy link
Contributor

webvictim commented Feb 26, 2020

Interesting. It looks like Ubuntu's support for 64-bit on the Pi 4 is quite recent. I'm running Raspbian buster on my own personal Pi 4 and Teleport works fine.

gus@artemis:~ $ uname -a
Linux artemis 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux
gus@artemis:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
gus@artemis:~ $ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 1
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 2
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 3
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

Hardware	: BCM2835
Revision	: b03111
Serial		: 10000000b0a6b070
Model		: Raspberry Pi 4 Model B Rev 1.1
gus@artemis:~ $ which tsh
/usr/local/bin/tsh
gus@artemis:~ $ tsh version
Teleport v4.2.0 git:v4.2.0-0-g3090806 go1.13.2

Our ARM build boxes are also armv7l or armhf (i.e. 32-bit), so it's unlikely that we'll have 64-bit ARM builds regularly available until we set up a 64-bit ARM build box.

I did a quick check on an AWS a1.large box running 64-bit Ubuntu and it does appear that you can build Teleport from source without any trouble, so that might be an option in the meantime. These are the steps I used:

sudo apt-get -y update && sudo apt-get -y install zip build-essential
wget https://dl.google.com/go/go1.13.2.linux-arm64.tar.gz
tar xvzf go1.13.2.linux-arm64.tar.gz
sudo mv go /usr/local/
echo "export PATH=/usr/local/go/bin:$PATH" >> ~/.bashrc
source ~/.bashrc
go get github.com/gravitational/teleport
cd ~/go/src/github.com/gravitational/teleport
git checkout tags/v4.2.3
make release

This puts teleport-v4.2.3-linux-arm64-bin.tar.gz in the ~/go/src/github.com/gravitational/teleport directory and then you can use this as desired.

@gclawes
Copy link
Author

gclawes commented Feb 26, 2020

Raspbian is still a 32-bit userland, with the option to enable a 64-bit kernel on the Pi 4. Ubuntu is 64-bit userland and kernel.

@gclawes
Copy link
Author

gclawes commented Sep 4, 2020

This is an issue on Manjaro Linux aarch64 as well

[gclawes@just-read-the-instructions teleport]$ cat /etc/os-release 
NAME="Manjaro-ARM"
ID=manjaro-arm
ID_LIKE=manjaro,arch
PRETTY_NAME="Manjaro ARM"
ANSI_COLOR="1;32"
HOME_URL="https://www.manjaro.org/"
SUPPORT_URL="https://forum.manjaro.org/c/manjaro-arm/"

[gclawes@just-read-the-instructions teleport]$ uname -a
Linux just-read-the-instructions 5.8.3-2-MANJARO-ARM #1 SMP Sat Aug 22 21:00:07 CEST 2020 aarch64 GNU/Linux
[gclawes@just-read-the-instructions teleport]$ file ./tsh
./tsh: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, Go BuildID=crhkFgAkhYoa_WcHNS2a/5xOrYJ9gAKmQ8UCmyzCX/BSfVolzsaA0q1gVvxNld/dWCl6K0GbCVKlg5avZVm, BuildID[sha1]=0183bfa0b1bf1c0cc18cfc845c5f31ebee36bfd7, stripped
[gclawes@just-read-the-instructions teleport]$ ./tsh
bash: ./tsh: No such file or directory

@awly
Copy link
Contributor

awly commented Sep 8, 2020

#1812 is the same issue and was reported earlier.
Let's keep all the discussion there.

We need to set up dedicated aarch64 build servers (for linux, mac and maybe windows).

@awly awly closed this as completed Sep 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants