Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: build nvidia images in main repo #319

Merged
merged 14 commits into from
Aug 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 59 additions & 13 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,28 +12,38 @@ env:
jobs:
push-ghcr:
name: Build and push image
runs-on: ubuntu-22.04
runs-on: buildjet-2vcpu-ubuntu-2204
permissions:
contents: read
packages: write
id-token: write
strategy:
fail-fast: false
matrix:
image_flavor: [main, nvidia]
image_name: [silverblue, kinoite, vauxite, sericea, base, lxqt, mate]
major_version: [37, 38]
nvidia_version: [0, 470, 535]
include:
- major_version: 37
is_latest_version: false
is_stable_version: true
- major_version: 38
is_latest_version: true
is_stable_version: true
- nvidia_version: 535
is_latest_nvidia: true
exclude:
# There is no Fedora 37 version of sericea
# When F38 is added, sericea will automatically be built too
- image_name: sericea
major_version: 37
- image_flavor: main
nvidia_version: 470
- image_flavor: main
nvidia_version: 535
- image_flavor: nvidia
nvidia_version: 0
steps:
# Checkout push-to-registry action GitHub repository
- name: Checkout Push to Registry action
Expand All @@ -54,24 +64,46 @@ jobs:
run: |
# Generate a timestamp for creating an image version history
TIMESTAMP="$(date +%Y%m%d)"
MAJOR_VERSION="${{ matrix.major_version }}"
VARIANT="${{ 'nvidia' == matrix.image_flavor && format('{0}-{1}', matrix.major_version, matrix.nvidia_version) || format('{0}', matrix.major_version) }}"

COMMIT_TAGS=()
BUILD_TAGS=()

# Have tags for tracking builds during pull request
SHA_SHORT="${GITHUB_SHA::7}"
COMMIT_TAGS+=("pr-${{ github.event.number }}-${MAJOR_VERSION}")
COMMIT_TAGS+=("${SHA_SHORT}-${MAJOR_VERSION}")
COMMIT_TAGS+=("pr-${{ github.event.number }}-${VARIANT}")
COMMIT_TAGS+=("${SHA_SHORT}-${VARIANT}")

if [[ "${{ matrix.is_latest_version }}" == "true" ]] && \
[[ "${{ matrix.is_stable_version }}" == "true" ]]; then
COMMIT_TAGS+=("pr-${{ github.event.number }}")
COMMIT_TAGS+=("${SHA_SHORT}")
if [[ "${{ matrix.image_flavor }}" == "main" ]] || \
[[ "${{ matrix.is_latest_nvidia }}" == "true" ]]; then
# assume matrix is honestis_latest_nvidia==true only when image_flavor==nvidia
COMMIT_TAGS+=("pr-${{ github.event.number }}")
COMMIT_TAGS+=("${SHA_SHORT}")
fi
fi

BUILD_TAGS=("${MAJOR_VERSION}" "${MAJOR_VERSION}-${TIMESTAMP}")
BUILD_TAGS=("${VARIANT}")

if [[ "${{ matrix.is_latest_nvidia }}" == "true" ]]; then
BUILD_TAGS+=("${{ matrix.major_version }}-current")
BUILD_TAGS+=("${{ matrix.major_version }}")
fi

# Append matching timestamp tags to keep a version history
for TAG in "${BUILD_TAGS[@]}"; do
BUILD_TAGS+=("${TAG}-${TIMESTAMP}")
done

if [[ "${{ matrix.is_latest_version }}" == "true" ]] && \
[[ "${{ matrix.is_stable_version }}" == "true" ]]; then
BUILD_TAGS+=("latest")
if [[ "${{ matrix.image_flavor }}" == "main" ]] || \
[[ "${{ matrix.is_latest_nvidia }}" == "true" ]]; then
# assume matrix is honest: is_latest_nvidia==true only when image_flavor==nvidia
BUILD_TAGS+=("${TIMESTAMP}")
BUILD_TAGS+=("latest")
fi
fi

if [[ "${{ github.event_name }}" == "pull_request" ]]; then
Expand All @@ -83,10 +115,12 @@ jobs:
else
alias_tags=("${BUILD_TAGS[@]}")
fi

echo "Generated the following build tags: "
for TAG in "${BUILD_TAGS[@]}"; do
echo "${TAG}"
done

echo "alias_tags=${alias_tags[*]}" >> $GITHUB_OUTPUT

- name: Get current version
Expand All @@ -103,10 +137,10 @@ jobs:
images: |
${{ env.IMAGE_NAME }}
labels: |
org.opencontainers.image.title=${{ env.IMAGE_NAME }}
org.opencontainers.image.title=${{ matrix.image_name }}-${{ matrix.image_flavor }}
org.opencontainers.image.version=${{ steps.labels.outputs.VERSION }}
org.opencontainers.image.description=A base ${{ env.IMAGE_NAME }} image with batteries included
io.artifacthub.package.readme-url=https://raw.githubusercontent.com/ublue-os/main/main/README.md
org.opencontainers.image.description=A base Universal Blue ${{ matrix.image_name }} image with batteries included${{ 'nvidia' == matrix.image_flavor && ' and Nvidia drivers added' || '' }}
io.artifacthub.package.readme-url=https://raw.githubusercontent.com/${{ github.repository }}/main/README.md
io.artifacthub.package.logo-url=https://avatars.githubusercontent.com/u/120078124?s=200&v=4

# Build image using Buildah action
Expand All @@ -116,15 +150,18 @@ jobs:
with:
containerfiles: |
./Containerfile
image: ${{ env.IMAGE_NAME }}
image: ${{ matrix.image_name }}-${{ matrix.image_flavor }}
tags: |
${{ steps.generate-tags.outputs.alias_tags }}
build-args: |
IMAGE_NAME=${{ matrix.image_name }}
SOURCE_IMAGE=${{ env.SOURCE_IMAGE }}
FEDORA_MAJOR_VERSION=${{ matrix.major_version }}
NVIDIA_MAJOR_VERSION=${{ matrix.nvidia_version }}
labels: ${{ steps.meta.outputs.labels }}
oci: false
extra-args: |
--target=${{ matrix.image_flavor }}

# Workaround bug where capital letters in your GitHub username make it impossible to push to GHCR.
# https://github.com/macbre/push-to-ghcr/issues/12
Expand Down Expand Up @@ -166,7 +203,8 @@ jobs:
- name: Sign container image
if: github.event_name != 'pull_request'
run: |
cosign sign -y --key env://COSIGN_PRIVATE_KEY ${{ steps.registry_case.outputs.lowercase }}/${{ env.IMAGE_NAME }}@${TAGS}
cosign sign -y --key env://COSIGN_PRIVATE_KEY ${{ steps.registry_case.outputs.lowercase }}/${{ steps.build_image.outputs.image }}@${TAGS}
tags: ${{ steps.build_image.outputs.tags }}
env:
TAGS: ${{ steps.push.outputs.digest }}
COSIGN_EXPERIMENTAL: false
Expand All @@ -177,3 +215,11 @@ jobs:
run: |
echo "${{ toJSON(steps.push.outputs) }}"

check:
name: Check all builds successful
runs-on: ubuntu-latest
needs: [push-ghcr]
steps:
- name: Exit
shell: bash
run: exit 0
35 changes: 26 additions & 9 deletions Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,38 @@ ARG SOURCE_IMAGE="${SOURCE_IMAGE:-silverblue}"
ARG BASE_IMAGE="quay.io/fedora-ostree-desktops/${SOURCE_IMAGE}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-37}"

FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION} AS builder

ARG IMAGE_NAME="${IMAGE_NAME}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION}"
FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION} AS main
ARG IMAGE_NAME="${IMAGE_NAME:-silverblue}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-37}"

ADD github-release-install.sh /tmp/github-release-install.sh
ADD build.sh /tmp/build.sh
ADD post-install.sh /tmp/post-install.sh
ADD packages.json /tmp/packages.json
COPY github-release-install.sh /tmp/github-release-install.sh
COPY main-install.sh /tmp/main-install.sh
COPY main-post-install.sh /tmp/main-post-install.sh
COPY main-packages.json /tmp/main-packages.json

COPY --from=ghcr.io/ublue-os/config:latest /rpms /tmp/rpms
COPY --from=ghcr.io/ublue-os/akmods:${FEDORA_MAJOR_VERSION} /rpms /tmp/akmods-rpms

RUN /tmp/build.sh
RUN /tmp/post-install.sh
RUN /tmp/main-install.sh
RUN /tmp/main-post-install.sh
RUN rm -rf /tmp/* /var/*
RUN ostree container commit
RUN mkdir -p /var/tmp && chmod -R 1777 /var/tmp


FROM main AS nvidia
ARG IMAGE_NAME="${IMAGE_NAME:-silverblue}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-37}"
ARG NVIDIA_MAJOR_VERSION="${NVIDIA_MAJOR_VERSION:-535}"

COPY nvidia-install.sh /tmp/nvidia-install.sh
COPY nvidia-post-install.sh /tmp/nvidia-post-install.sh

COPY --from=ghcr.io/ublue-os/akmods-nvidia:${FEDORA_MAJOR_VERSION}-${NVIDIA_MAJOR_VERSION} /rpms /tmp/akmods-rpms

RUN /tmp/nvidia-install.sh
RUN /tmp/nvidia-post-install.sh
RUN rm -rf /tmp/* /var/*
RUN ostree container commit
RUN mkdir -p /var/tmp && chmod -R 1777 /tmp /var/tmp
2 changes: 1 addition & 1 deletion github-release-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ fi
set -ouex pipefail

API="https://api.github.com/repos/${ORG_PROJ}/releases/latest"
RPM_URLS=$(curl -sL ${API} \
RPM_URLS=$(curl --retry 3 --retry-delay 0 --retry-all-errors -sL ${API} \
| jq \
-r \
--arg arch_filter "${ARCH_FILTER}" \
Expand Down
4 changes: 2 additions & 2 deletions build.sh → main-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ RELEASE="$(rpm -E %fedora)"

INCLUDED_PACKAGES=($(jq -r "[(.all.include | (.all, select(.\"$IMAGE_NAME\" != null).\"$IMAGE_NAME\")[]), \
(select(.\"$FEDORA_MAJOR_VERSION\" != null).\"$FEDORA_MAJOR_VERSION\".include | (.all, select(.\"$IMAGE_NAME\" != null).\"$IMAGE_NAME\")[])] \
| sort | unique[]" /tmp/packages.json))
| sort | unique[]" /tmp/main-packages.json))
EXCLUDED_PACKAGES=($(jq -r "[(.all.exclude | (.all, select(.\"$IMAGE_NAME\" != null).\"$IMAGE_NAME\")[]), \
(select(.\"$FEDORA_MAJOR_VERSION\" != null).\"$FEDORA_MAJOR_VERSION\".exclude | (.all, select(.\"$IMAGE_NAME\" != null).\"$IMAGE_NAME\")[])] \
| sort | unique[]" /tmp/packages.json))
| sort | unique[]" /tmp/main-packages.json))


if [[ "${#EXCLUDED_PACKAGES[@]}" -gt 0 ]]; then
Expand Down
File renamed without changes.
File renamed without changes.
23 changes: 23 additions & 0 deletions nvidia-install.sh
Copy link
Member

@p5 p5 Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Unrelated)
It might be a good idea to bundle these Nvidia install and post-install scripts into the akmods image instead of requiring downstream consumers to copy, paste and keep them maintained.

We can keep them up-to-date in the akmods repo, then if people are consuming the akmods repo rather than main images, they have the scripts bundled with the image to call.

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/sh

set -ouex pipefail

sed -i 's@enabled=1@enabled=0@g' /etc/yum.repos.d/fedora-{cisco-openh264,modular,updates-modular}.repo

rpm-ostree install \
/tmp/akmods-rpms/ublue-os/ublue-os-nvidia-addons-*.rpm

source /tmp/akmods-rpms/kmods/nvidia-vars.${NVIDIA_MAJOR_VERSION}

if [[ "${IMAGE_NAME}" == "kinoite" ]]; then
VARIANT_PKGS="supergfxctl-plasmoid"
elif [[ "${IMAGE_NAME}" == "silverblue" ]]; then
VARIANT_PKGS="gnome-shell-extension-supergfxctl-gex"
else
VARIANT_PKGS=""
fi

rpm-ostree install \
xorg-x11-drv-${NVIDIA_PACKAGE_NAME}-{,cuda-,devel-,kmodsrc-,power-}${NVIDIA_FULL_VERSION} \
nvidia-container-toolkit nvidia-vaapi-driver supergfxctl ${VARIANT_PKGS} \
/tmp/akmods-rpms/kmods/kmod-${NVIDIA_PACKAGE_NAME}-${KERNEL_VERSION}-${NVIDIA_AKMOD_VERSION}.fc${RELEASE}.rpm
17 changes: 17 additions & 0 deletions nvidia-post-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/sh

set -ouex pipefail

sed -i 's@enabled=1@enabled=0@g' /etc/yum.repos.d/{eyecantcu-supergfxctl,nvidia-container-runtime}.repo

mv /etc/nvidia-container-runtime/config.toml{,.orig}
cp /etc/nvidia-container-runtime/config{-rootless,}.toml

semodule --verbose --install /usr/share/selinux/packages/nvidia-container.pp
ln -s /usr/bin/ld.bfd /etc/alternatives/ld
ln -s /etc/alternatives/ld /usr/bin/ld

if [[ "${IMAGE_NAME}" == "sericea" ]]; then
mv /etc/sway/environment{,.orig}
install -Dm644 /usr/share/ublue-os/etc/sway/environment /etc/sway/environment
fi
Loading