Initial aarch64 support, cross arch stateless images with qemu-user-static, grub2 for x86

[Obihoernchen]

The following PR adds three features:

• initial aarch64 support for xCAT using grub2
• grub2 support for x86 arch
• cross arch stateless image generation with qemu-user-static allowing you to build ppc64 or aarch64 stateless images on x86 hosts

It's not 100% ready yet and at the moment I've only tested EL8, but it should be good enough for some PoC tests :)

Status

• stateful aarch64 images
• stateless aarch64 images
• Fix aarch64 install dependencies
• activate xCAT-genesis-scripts-aarch64 build + build xCAT-genesis-base-aarch64 + add dep to xCAT.spec (optional)
• grub2 support for x86 arch
• cross arch stateless image generation with qemu-user-static
• Man pages
• Documentation

Emulate aarch64 on x86_64 with QEMU for testing

• Install qemu-system-aarch64
• Create VM with:
  • arch: aarch64
  • machine type: virt
  • cpu model: max
  • emulator: /usr/bin/qemu-system-aarch64
  • firmware: UEFI aarch64: /usr/share/edk2/aarch64/QEMU_EFI-silent-pflash.raw

Note: tested with Fedora 36

Refs:

• https://www.qemu.org/docs/master/system/arm/virt.html
• https://www.qemu.org/docs/master/system/target-arm.html

xCAT RPMs for aarch64

• xcat-core packages are builded for aarch64 with: bf57a2b13ef5978a3e05076b195f01dc3f2ddb35
• Missing dependencies:
  • ipmitool-xcat
  • goconserver [dep, SOL] --> built
  • perl-IO-Tty.x86_64 [dep] --> perl-IO-Tty.aarch64 in powertools repo

Build goconserver for aarch64

dnf install golang
git clone https://github.com/xcat2/goconserver.git
cd goconserver
go env -w GO111MODULE=auto
make deps
make rpm
ls -l build/*.rpm

grub2 boot preparation

• Download grub2 efi for x86 and aarch64 from any EL8 mirror. For instance:
  • https://ftp.fau.de/almalinux/8/BaseOS/aarch64/os/EFI/BOOT/grubaa64.efi
  • https://ftp.fau.de/almalinux/8/BaseOS/x86_64/os/EFI/BOOT/grubx64.efi
• Copy to /tftpboot/boot/grub2/grub2.aarch64 and /tftpboot/boot/grub2/grub2.x86_64

qemu-user-static for cross arch stateless images

• Get qemu-user-static binary for aarch64 and copy to: /usr/bin/qemu-aarch64-static with 755 permissions
• Configure /etc/binfmt.d/qemu-aarch64-static.conf accordingly:

:qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64-static:F

• Restart binfmt service: systemctl restart systemd-binfmt

Misc

[root@xcat-aarch64 grub2]# lsdef -t osimage rocky8-aarch64-netboot-compute
Object name: rocky8-aarch64-netboot-compute
    exlist=/opt/xcat/share/xcat/netboot/rocky/compute.rocky8.x86_64.exlist
    imagetype=linux
    osarch=aarch64
    osdistroname=rocky8-aarch64
    osname=Linux
    osvers=rocky8
    otherpkgdir=/install/post/otherpkgs/rocky8/aarch64
    permission=755
    pkgdir=/install/rocky8/aarch64
    pkglist=/opt/xcat/share/xcat/netboot/rocky/compute.rocky8.x86_64.pkglist
    postinstall=/opt/xcat/share/xcat/netboot/rocky/compute.rocky8.x86_64.postinstall
    profile=compute
    provmethod=netboot
    rootimgdir=/install/netboot/rocky8/aarch64/compute

[Obihoernchen]

Works fine with el9, too.
Feel free to test it. You don't need an arm management node for testing.
Just install grub2 and qemu-user-static on your management node as described above.

[Obihoernchen]

Note: you might have to manually edit generated grub configs because ARM servers might use serial port ttyAMA0 and xCAT will always use the string "ttyS" + nodehm.serialport

[Obihoernchen]

@ocfmatt please take a look at it. Especially the documentation I have written.

[ocfmatt]

The changes look good to me. I like the use of qemu for cross architecture compiling.