RFE: Allow `rpm-ostree rebase` (or `rpm-ostree install --apply-live`) on live iso

[JM1]

Host system details

$> rpm-ostree status
State: idle
Deployments:
● fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230609.3.0 (2023-06-26T21:56:57Z)
                   Commit: 248366c65732b30ae0dbd96be8b75db46f08f428f68254ee14ac52cb39f82240
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464

Expected vs actual behavior

Please allow to run rpm-ostree rebase on a system which has been launched with a Live ISO. For example, when FCOS is booted with a Live ISO both / and /system are mounted read-only. This causes rpm-ostree rebase ... to fail with:

error: Remounting /sysroot read-write: Permission denied

If rpm-ostree rebase cannot be implemented for live isos, another option might be to support rpm-ostree install --apply-live instead.

Rationale

OKD/FCOS uses FCOS as its bootimage, i.e. when booting cluster nodes the first time during installation. FCOS does not provide tools such as OpenShift Client (oc) or hyperkube which are used during single-node cluster installation at first boot (e.g. oc in bootkube.sh). RHCOS and SCOS include these tools, but FCOS has to pivot the root fs to okd-machine-os first in order to make those tools available. Pivoting uses rpm-ostree rebase but during SNO installation the node will be booted from a FCOS Live ISO where / and /sysroot are read-only. Thus rpm-ostree rebase fails and necessary tools for SNO installation will not be available, causing the setup to stall.

Allowing rpm-ostree install --apply-live on live isos would allow users to extend their live environment in the same way as traditional installations. Either to install necessary tools such as hyperkube in the OKD/FCOS SNO use case above or auxiliary tools such as fzf or interpreters.

[cgwalters]

I think I'd say we should add e.g. --apply-live-only or so, which would actually be useful for non-ISO cases too.

We could also default to having --apply-live mean --apply-live-only on physically readonly systems.

[JM1]

It looks like rpm-ostree already implements all necessary features to support rebasing a Live ISO ☺️ This PR for fedora-coreos-config changes the way FCOS accesses contents of a Live ISO. Together with systemd's soft-reboot this allows to rebase a running system.