Support for zstd compressed kernel and modules

[arnaldo2792]

Issue number:
#1614

Description of changes:

47032848 packages: add support for ZSTD compression in kernel 5.10
3c3a6399 packages: add support for ZSTD compression in kernel 5.4
9e5ec8eb packages: add support to kmod for zstd compression
85aa7301 packages: add libzstd

With these changes, kernel x86_64 images and modules for both arches are compressed using the Zstandard compression algorithm.

Testing done:

aws-dev, aws-k8s-1.19, aws-k8s-1.20, aws-ecs-1 (x86_64, aarch64):

• Hosts actually boot
• Launch pod/task/container
• systemctl status OK
• journalctl -p3 OK
• modinfo fuse shows module's name with extension .ko.zst
• Only x86_64: check kernel image is compressed with a modified version of this script
• Only k8s: Install falco without ebpf.enabled, this will force compiling and loading the falco driver at runtime.
• Compile kernel module with new kmod kit, with Dockerfile:

FROM amazonlinux:2 as build

WORKDIR /tmp
# Install dependencies and download sources
RUN \
  ulimit -n 1024; yum -y install \
    bc bzip2 cmake3 curl diffutils dwarves elfutils-devel \
    findutils gcc gcc-c++ git kmod make tar ncurses-devel \
    patch xz && \
  git clone https://github.com/falcosecurity/falco.git && \
  curl -OJL https://github.com/openzfs/zfs/releases/download/zfs-2.0.4/zfs-2.0.4.tar.gz

# Add kmod kit in its own layer
FROM build as kmod-kit
ARG VARIANT="aws-dev"
ARG ARCH="x86_64"
ARG VERSION="1.1.4"
ARG KIT="${VARIANT}-${ARCH}-kmod-kit-v${VERSION}"
COPY ./${KIT}.tar.xz /tmp
RUN tar xf ${KIT}.tar.xz

# Compile falco driver
FROM kmod-kit as falco-build
ARG VARIANT="aws-dev"
ARG ARCH="x86_64"
ARG VERSION="1.1.4"
ARG KIT="${VARIANT}-${ARCH}-kmod-kit-v${VERSION}"
ARG KERNELDIR="/tmp/${KIT}/kernel-devel"
ARG CROSS_COMPILE="${ARCH}-bottlerocket-linux-musl-"
ARG INSTALL_MOD_STRIP=1
RUN \
  export PATH="/tmp/${KIT}/toolchain/usr/bin:${PATH}" && \
  mkdir -p falco/build && \
  cd falco/build && \
  cmake3 -DUSE_BUNDLED_DEPS=ON .. && \
  make driver -j8
RUN test -f /tmp/falco/build/driver/falco.ko

# Compile zfs kernel module
FROM kmod-kit as zfs-build
ARG VARIANT="aws-dev"
ARG ARCH="x86_64"
ARG VERSION="1.1.4"
ARG KIT="${VARIANT}-${ARCH}-kmod-kit-v${VERSION}"
ARG KERNELDIR="/tmp/${KIT}/kernel-devel"
ARG CROSS_COMPILE="${ARCH}-bottlerocket-linux-musl-"
ARG INSTALL_MOD_STRIP=1
RUN \
  export PATH="/tmp/${KIT}/toolchain/usr/bin:${PATH}" && \
  tar xf zfs-2.0.4.tar.gz && \
  cd zfs-2.0.4 && \
  ./configure \
    --with-config=kernel \
    --with-linux="${KERNELDIR}" \
    --with-linux-obj="${KERNELDIR}" && \
  make -C module modules -j8
RUN test -f  /tmp/zfs-2.0.4/module/avl/zavl.ko && \
    test -f /tmp/zfs-2.0.4/module/icp/icp.ko && \
    test -f /tmp/zfs-2.0.4/module/lua/zlua.ko && \
    test -f /tmp/zfs-2.0.4/module/nvpair/znvpair.ko && \
    test -f /tmp/zfs-2.0.4/module/spl/spl.ko && \
    test -f /tmp/zfs-2.0.4/module/unicode/zunicode.ko && \
    test -f /tmp/zfs-2.0.4/module/zcommon/zcommon.ko && \
    test -f /tmp/zfs-2.0.4/module/zfs/zfs.ko

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[arnaldo2792]

Force push includes:

• Fix the libzstd license in spec
• Add Sign-off-by legend in patches that required manual intervention due to merge conflicts
• Add cherry-pick legend in patches so that we know from which commit we took the patch
• Removed some duplicated lines from 2001-kbuild-add-support-for-zstd-compressed-modules.patch, which are already present in 2000-kbuild-move-module-strip-compression-code-into-scrip.patch, but in the previous version they got carried over twice.

[bcressey]

nit:

Suggested change

	%description devel
	%description devel
	%{summary}.

[arnaldo2792]

Forced pushed includes updated description for the libzstd-devel package.

[webern]

👍