release, rpm2img, partyplanner: label BOTTLEROCKET-DATA during firstboot

Changes partitioning to append a fallback data partition to the
os image. This is only applicable with the "split" partitioning plan.
Nothing changes for the "unified" partitioning plan.

We add three new services that run as part of 'local-fs.target' and
before 'local.mount' and 'repart-local.service':
  * 'label-data-alt.service'
  * 'label-data-pref.service'
  * 'formalize-data-part.service'

'label-data-alt' and 'label-data-pref' will "compete" and both try to
label 'BOTTLEROCKET-DATA' first with the partition they're each waiting
for.

'label-data-pref' waits for the "preferred" data partition that
either resides on a separate disk for "split" image configurations or on
the same OS image for "unified" image configurations. Once that device
is ready, we call 'systemd-repart' to relabel it as 'BOTTLEROCKET-DATA'
and grow it as much as possible.

'label-data-alt' first waits for a moderate amount of time before
kicking off. This gives 'label-data-pref' a chance to relabel the
"preferred" partition as 'BOTTLEROCKET-DATA'. If the "preferred" data
partition is not available for whatever reason, this service will call
'systemd-repart' to relabel the alternative/back-up data partition on
the OS image as 'BOTTLEROCKET-DATA' and grow the partition to fill the
remainder of the disk.

'formalize-data-part''s job is to wait for a partition labelled
'BOTTLEROCKET-DATA' to appear. Once it does, it forcibly stops and
masks the the other two service units.

All of this lets the host to boot if the "preferred" data partition
doesn't exist and the root filesystem disk has leftover extra space to
accommodate a reasonably-sized back-up data partition.

packages/release/finalize-data-label.service

@@ -0,0 +1,20 @@
+[Unit]
+Description=Finalize BOTTLEROCKET-DATA
+DefaultDependencies=no
+Conflicts=shutdown.target
+# Wait for partition labeled 'BOTTLEROCKET-DATA' to appear.
+Wants=dev-disk-by\x2dpartlabel-BOTTLEROCKET\x2dDATA.device
+After=dev-disk-by\x2dpartlabel-BOTTLEROCKET\x2dDATA.device
+
+[Service]
+Type=oneshot
+# Once we have the 'BOTTLEROCKET-DATA' partition, the 'label-data-*' services should never run again.
+ExecStart=/usr/bin/systemctl stop label-data-alternative --no-block
+ExecStart=/usr/bin/systemctl stop label-data-preferred --no-block
+ExecStart=/usr/bin/systemctl mask label-data-alternative --no-block
+ExecStart=/usr/bin/systemctl mask label-data-preferred --no-block
+RemainAfterExit=true
+StandardError=journal+console
+
+[Install]
+WantedBy=local-fs.target

packages/release/label-data-alternative.service

@@ -0,0 +1,21 @@
+[Unit]
+Description=Label alternative data partition
+DefaultDependencies=no
+Conflicts=shutdown.target
+# Only run this if a partition labeled 'BOTTLEROCKET-DATA' does not exist already.
+ConditionPathIsSymbolicLink=!/dev/disk/by-partlabel/BOTTLEROCKET-DATA
+# This is the partition GUID for the fallback data partition.
+Wants=dev-disk-by\x2dpartuuid-69040874\x2d417d\x2d4e26\x2da764\x2d7885f22007ea.device
+After=dev-disk-by\x2dpartuuid-69040874\x2d417d\x2d4e26\x2da764\x2d7885f22007ea.device
+
+[Service]
+Type=oneshot
+
+# Wait a bit to give 'label-data-partition.service' a chance to label and size the preferred data partition.
+# If 'BOTTLEROCKET-DATA' isn't present after this wait, then we try to label and size the alternative data partition.
+ExecStartPre=-/usr/bin/sleep 30
+# Label the partition as 'BOTTLEROCKET-DATA' and resize the partition, whether or not it resides on the same disk as /.
+ExecStart=-/usr/bin/systemd-repart --dry-run=no /dev/disk/by-partuuid/69040874-417d-4e26-a764-7885f22007ea
+
+[Install]
+WantedBy=local-fs.target

packages/release/label-data-preferred.service

@@ -0,0 +1,18 @@
+[Unit]
+Description=Label preferred data partition
+DefaultDependencies=no
+Conflicts=shutdown.target
+# Only run this if a partition labeled 'BOTTLEROCKET-DATA' does not exist already.
+ConditionPathIsSymbolicLink=!/dev/disk/by-partlabel/BOTTLEROCKET-DATA
+# This is the partition GUID for the preferred data partition.
+Wants=dev-disk-by\x2dpartuuid-5b94e8df\x2d28b8\x2d485c\x2d9d19\x2d362263b5944c.device
+After=dev-disk-by\x2dpartuuid-5b94e8df\x2d28b8\x2d485c\x2d9d19\x2d362263b5944c.device
+
+[Service]
+Type=oneshot
+
+# Label the partition as 'BOTTLEROCKET-DATA' and resize the partition, whether or not it resides on the same disk as /.
+ExecStart=-/usr/bin/systemd-repart --dry-run=no /dev/disk/by-partuuid/5b94e8df-28b8-485c-9d19-362263b5944c
+
+[Install]
+WantedBy=local-fs.target

packages/release/release-repart-local.conf

@@ -1,4 +1,7 @@
 [Partition]
+# We want to label this partition 'BOTTLEROCKET-DATA'
+Label=BOTTLEROCKET-DATA
+
 # This is the partition type UUID for BOTTLEROCKET-DATA, which will be resized
 # to fill the remaining sectors on the disk where it resides.
 Type=626f7474-6c65-6474-6861-726d61726b73

packages/release/release.spec

@@ -57,6 +57,9 @@ Source1043: repart-local.service
 Source1044: mask-local-mnt.service
 Source1045: mask-local-opt.service
 Source1046: mask-local-var.service
+Source1047: label-data-alternative.service
+Source1048: label-data-preferred.service
+Source1049: finalize-data-label.service
 
 # Services for kdump support
 Source1060: capture-kernel-dump.service
@@ -138,7 +141,8 @@ install -p -m 0644 \
   %{S:1001} %{S:1002} %{S:1003} %{S:1004} %{S:1005} %{S:1006} %{S:1007} \
   %{S:1008} %{S:1009} %{S:1010} %{S:1011} %{S:1012} %{S:1013} %{S:1015} \
   %{S:1040} %{S:1041} %{S:1042} %{S:1043} %{S:1044} %{S:1045} %{S:1046} \
-  %{S:1060} %{S:1061} %{S:1062} %{S:1080} %{S:1014} \
+  %{S:1047} %{S:1048} %{S:1049} %{S:1060} %{S:1061} %{S:1062} %{S:1080} \
+  %{S:1014} \
   %{buildroot}%{_cross_unitdir}
 
 install -d %{buildroot}%{_cross_unitdir}/systemd-tmpfiles-setup.service.d
@@ -219,6 +223,9 @@ ln -s preconfigured.target %{buildroot}%{_cross_unitdir}/default.target
 %{_cross_unitdir}/mask-local-opt.service
 %{_cross_unitdir}/mask-local-var.service
 %{_cross_unitdir}/root-.aws.mount
+%{_cross_unitdir}/label-data-alternative.service
+%{_cross_unitdir}/label-data-preferred.service
+%{_cross_unitdir}/finalize-data-label.service
 %dir %{_cross_unitdir}/systemd-tmpfiles-setup.service.d
 %{_cross_unitdir}/systemd-tmpfiles-setup.service.d/00-debug.conf
 %dir %{_cross_templatedir}

tools/partyplanner

@@ -2,7 +2,7 @@
 # shellcheck disable=SC2034  # Variables are used externally by rpm2img
 
 ###############################################################################
-# Section 1: partition type GUIDs
+# Section 1: partition type GUIDs and partition GUIDs
 
 # Define partition type GUIDs for all OS-managed partitions. This is required
 # for the boot partition, where we set gptprio bits in the GUID-specific use
@@ -40,6 +40,11 @@ EFI_SYSTEM_TYPECODE="C12A7328-F81F-11D2-BA4B-00A0C93EC93B"
 # for an alternate bank.
 EFI_BACKUP_TYPECODE="B39CE39C-0A00-B4AB-2D11-F18F8237A21C"
 
+# Define partition GUIDs for the preferred and fallback data partitions. We
+# use the GUID for determining which data partition to label and use at boot.
+BOTTLEROCKET_DATA_A_PARTGUID="5b94e8df-28b8-485c-9d19-362263b5944c"
+BOTTLEROCKET_DATA_B_PARTGUID="69040874-417d-4e26-a764-7885f22007ea"
+
 ###############################################################################
 # Section 2: fixed size partitions and reservations
 
@@ -62,6 +67,9 @@ OVERHEAD_MIB="$((GPT_MIB * 2 + BIOS_MIB))"
 # EFI partition size by taking space from the "reserved" area below.
 EFI_MIB="5" # one per bank
 
+# Allocate 1 MiB for the fallback data partition.
+DATA_B_MIB="1" # one per disk
+
 ###############################################################################
 # Section 3: variable sized partitions
 
@@ -102,7 +110,8 @@ PRIVATE_SCALE_FACTOR="24"
 #          | Hash partition B         5 MiB* |          500 MiB
 #          | Reserved partition B    10 MiB* |
 #          +---------------------------------+
-#          | Private partition       18 MiB* | (image size * 24 as MiB) - prelude
+#          | Private partition  (18/17) MiB* | (image size * 24 as MiB) - prelude - DATA-B size
+#          | Data partition B     (0/1) MiB  | Fallback data partition (only in split partition plan)
 # Postlude | GPT footer               1 MiB  | GPT is fixed, private partition grows.
 #          +---------------------------------+
 
@@ -139,6 +148,10 @@ set_partition_sizes() {
 
   # Private space scales per GiB, minus the BIOS and GPT partition overhead.
   private_mib=$((os_image_gib * PRIVATE_SCALE_FACTOR - OVERHEAD_MIB))
+  if [ "${partition_plan}" == "split" ] ; then
+    # We need 1 MiB of space for the fallback data partition.
+    private_mib=$((private_mib - DATA_B_MIB))
+  fi
 
   # Skip the GPT label at start of disk.
   local offset
@@ -176,16 +189,21 @@ set_partition_sizes() {
 
   case "${partition_plan}" in
     split)
+      # For the fallback data partition that lives on the OS image
+      pp_offset["DATA-B"]="${offset}"
+      pp_size["DATA-B"]="${DATA_B_MIB}"
+      ((offset += DATA_B_MIB))
+
       # For a split data image, the first and last MiB are reserved for the GPT
-      # labels, and the rest is for the "data" partition.
-      pp_size["DATA"]="$((data_image_gib * 1024 - GPT_MIB * 2))"
-      pp_offset["DATA"]="1"
+      # labels, and the rest is for the preferred data partition.
+      pp_size["DATA-A"]="$((data_image_gib * 1024 - GPT_MIB * 2))"
+      pp_offset["DATA-A"]="1"
       ;;
     unified)
       # For a unified image, we've already accounted for the GPT label space in
-      # the earlier calculations, so all the space is for the "data" partition.
-      pp_size["DATA"]="$((data_image_gib * 1024))"
-      pp_offset["DATA"]="${offset}"
+      # the earlier calculations, so all the space is for the data partition.
+      pp_size["DATA-A"]="$((data_image_gib * 1024))"
+      pp_offset["DATA-A"]="${offset}"
       ((offset += data_image_gib * 1024))
       ;;
     *)
@@ -200,9 +218,12 @@ set_partition_labels() {
   local -n pp_label
   pp_label="${1:?}"
   pp_label["BIOS"]="BIOS-BOOT"
-  pp_label["DATA"]="BOTTLEROCKET-DATA"
   pp_label["EFI-A"]="EFI-SYSTEM"
   pp_label["EFI-B"]="EFI-BACKUP"
+  # Empty label for the data partitions. We're labelling the data partition
+  # during boot.
+  pp_label["DATA-A"]=""
+  pp_label["DATA-B"]=""
   pp_label["PRIVATE"]="BOTTLEROCKET-PRIVATE"
   for part in BOOT ROOT HASH RESERVED ; do
     for bank in A B ; do
@@ -216,7 +237,8 @@ set_partition_types() {
   local -n pp_type
   pp_type="${1:?}"
   pp_type["BIOS"]="${BIOS_BOOT_TYPECODE}"
-  pp_type["DATA"]="${BOTTLEROCKET_DATA_TYPECODE}"
+  pp_type["DATA-A"]="${BOTTLEROCKET_DATA_TYPECODE}"
+  pp_type["DATA-B"]="${BOTTLEROCKET_DATA_TYPECODE}"
   pp_type["EFI-A"]="${EFI_SYSTEM_TYPECODE}"
   pp_type["EFI-B"]="${EFI_BACKUP_TYPECODE}"
   pp_type["PRIVATE"]="${BOTTLEROCKET_PRIVATE_TYPECODE}"
@@ -229,3 +251,16 @@ set_partition_types() {
     done
   done
 }
+
+# Populate the caller's table with GPT partition UUIDs for the preferred and
+# fallback data partitions.
+set_partition_uuids() {
+  local -n pp_uuid
+  pp_uuid="${1:?}"
+  local uuid
+  for bank in A B ; do
+    uuid="BOTTLEROCKET_DATA_${bank}_PARTGUID"
+    uuid="${!uuid}"
+    pp_uuid["DATA-${bank}"]="${uuid}"
+  done
+}

tools/rpm2img

@@ -109,7 +109,8 @@ ROOT_IMAGE="$(mktemp)"
 DATA_IMAGE="$(mktemp)"
 EFI_IMAGE="$(mktemp)"
 PRIVATE_IMAGE="$(mktemp)"
-BOTTLEROCKET_DATA="$(mktemp)"
+BOTTLEROCKET_DATA_A="$(mktemp)"
+BOTTLEROCKET_DATA_B="$(mktemp)"
 
 ROOT_MOUNT="$(mktemp -d)"
 BOOT_MOUNT="$(mktemp -d)"
@@ -136,22 +137,28 @@ case "${PARTITION_PLAN}" in
     ;;
 esac
 
-declare -A partlabel parttype partsize partoff
+declare -A partlabel parttype partguid partsize partoff
 set_partition_sizes \
   "${OS_IMAGE_SIZE_GIB}" "${DATA_IMAGE_SIZE_GIB}" "${PARTITION_PLAN}" \
   partsize partoff
 set_partition_labels partlabel
 set_partition_types parttype
+set_partition_uuids partguid
 
 declare -a partargs
 for part in \
   BIOS \
   EFI-A BOOT-A ROOT-A HASH-A RESERVED-A \
   EFI-B BOOT-B ROOT-B HASH-B RESERVED-B \
-  PRIVATE DATA ;
+  PRIVATE DATA-A DATA-B ;
 do
-  # We only append the data partition if we're using the unified layout.
-  if [ "${part}" == "DATA" ] && [ "${PARTITION_PLAN}" != "unified" ] ; then
+  # We only append the preferred DATA-A partition if we're using the unified layout.
+  if [ "${part}" == "DATA-A" ] && [ "${PARTITION_PLAN}" != "unified" ] ; then
+    continue
+  fi
+
+  # We only append the fallback DATA-B partition if we're using the split layout
+  if [ "${part}" == "DATA-B" ] && [ "${PARTITION_PLAN}" == "unified" ] ; then
     continue
   fi
 
@@ -166,6 +173,7 @@ do
   partargs+=(-n "0:${part_start}M:${part_end}")
   partargs+=(-c "0:${partlabel[${part}]}")
   partargs+=(-t "0:${parttype[${part}]}")
+  partargs+=(-u "0:${partguid[${part}]:-R}")
 
   # Boot partition attributes:
   #  48 = gptprio priority bit
@@ -180,13 +188,14 @@ sgdisk --clear "${partargs[@]}" --sort --print "${OS_IMAGE}"
 
 # Partition the separate data disk, if we're using the split layout.
 if [ "${PARTITION_PLAN}" == "split" ] ; then
-  data_start="${partoff[DATA]}"
-  data_end=$((data_start + partsize[DATA]))
+  data_start="${partoff[DATA-A]}"
+  data_end=$((data_start + partsize[DATA-A]))
   data_end=$((data_end * 2048 - 1))
   sgdisk --clear \
     -n "0:${data_start}M:${data_end}" \
-    -c "0:${partlabel[DATA]}" \
-    -t "0:${parttype[DATA]}" \
+    -c "0:${partlabel[DATA-A]}" \
+    -t "0:${parttype[DATA-A]}" \
+    -u "0:${partguid[DATA-A]}" \
     --sort --print "${DATA_IMAGE}"
 fi
 
@@ -383,22 +392,28 @@ cp /host/tools/bootconfig/empty-bootconfig.data "${PRIVATE_MOUNT}/bootconfig.dat
 mkfs.ext4 -b 4096 -i 4096 -I 256 -d "${PRIVATE_MOUNT}" "${PRIVATE_IMAGE}" "${partsize[PRIVATE]}M"
 dd if="${PRIVATE_IMAGE}" of="${OS_IMAGE}" conv=notrunc bs=1M seek="${partoff[PRIVATE]}"
 
-# BOTTLEROCKET-DATA
+# BOTTLEROCKET-DATA-A and BOTTLEROCKET-DATA-B
 
 # If we build on a host with SELinux enabled, we could end up with labels that
 # do not match our policy. Since we allow replacing the data volume at runtime,
 # we can't count on these labels being correct in any case, and it's better to
 # remove them all.
 UNLABELED=$(find "${DATA_MOUNT}" \
     | awk -v root="${DATA_MOUNT}" '{gsub(root"/","/"); gsub(root,"/"); print "ea_rm", $1, "security.selinux"}')
-mkfs.ext4 -d "${DATA_MOUNT}" "${BOTTLEROCKET_DATA}" "${partsize[DATA]}M"
-echo "${UNLABELED}" | debugfs -w -f - "${BOTTLEROCKET_DATA}"
+mkfs.ext4 -d "${DATA_MOUNT}" "${BOTTLEROCKET_DATA_A}" "${partsize[DATA-A]}M"
+echo "${UNLABELED}" | debugfs -w -f - "${BOTTLEROCKET_DATA_A}"
+
 case "${PARTITION_PLAN}" in
   split)
-    dd if="${BOTTLEROCKET_DATA}" of="${DATA_IMAGE}" conv=notrunc bs=1M seek="${partoff[DATA]}"
+    # Only append the DATA-B fallback data partition in "split" image layout configurations
+    mkfs.ext4 -d "${DATA_MOUNT}" "${BOTTLEROCKET_DATA_B}" "${partsize[DATA-B]}M"
+    echo "${UNLABELED}" | debugfs -w -f - "${BOTTLEROCKET_DATA_B}"
+    dd if="${BOTTLEROCKET_DATA_B}" of="${OS_IMAGE}" conv=notrunc bs=1M seek="${partoff[DATA-B]}"
+
+    dd if="${BOTTLEROCKET_DATA_A}" of="${DATA_IMAGE}" conv=notrunc bs=1M seek="${partoff[DATA-A]}"
     ;;
   unified)
-    dd if="${BOTTLEROCKET_DATA}" of="${OS_IMAGE}" conv=notrunc bs=1M seek="${partoff[DATA]}"
+    dd if="${BOTTLEROCKET_DATA_A}" of="${OS_IMAGE}" conv=notrunc bs=1M seek="${partoff[DATA-A]}"
     ;;
 esac