prepare-root: Add composefs.enabled=verity

[ruihe774]

In the current implementation, a composefs deploy is either:

• signed by key and verified by fs-verity, or
• not signed and not verified.

This PR adds a new supported value verity to composefs.enabled, which:

• requires the composefs to be verified by fs-verity;
• does not require it to be signed.

The idea is that:

• If we do not specify LCFS_MOUNT_FLAGS_REQUIRE_VERITY (the underlying of which is verity=require when mounting overlayfs), overlayfs does not verify file data even if fs-verity digests are present in the image.
• It is difficult and troublesome to have the ostree commit signed in some circumstances. For example, when deploying with custom layering.
• In some cases, the integrity of the composefs image is already protected by other mechanisms, for example EVM. It is not necessary to verify the image by a key, but it is still necessary to verify file data referenced by the image using fs-verity to protect file integrity from disk corruption.

[openshift-ci]

Hi @ruihe774. Thanks for your PR.

I'm waiting for a ostreedev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[cgwalters]

ACK to the overall idea though the way I'd describe this is more "external signing". One thing I could imagine doing here is checking if a stamp file like /run/ostree/composefs-signed or so is present, then we skip our own signature checks and assume that an external systemd unit that ran before this code executed.

[cgwalters]

Thanks, looks sane. Can you also update man/ostree-prepare-root.xml?

[cgwalters]

Also ref composefs/composefs#251

[ruihe774]

One thing I could imagine doing here is checking if a stamp file like /run/ostree/composefs-signed or so is present, then we skip our own signature checks and assume that an external systemd unit that ran before this code executed.

I think it's a bit of complicated and over-designed. If we do that, the "external" verification tool needs to explicitly cooperate with ostree and write the stamp file, or the end user needs to write a custom systemd service and add it to initramfs, which is troublesome.

[ruihe774]

Can you also update man/ostree-prepare-root.xml?

Done. PTAL.

[cgwalters]

Looks good!

Our CI just fell over, I filed #3356 to get some things fixed and I think we need to have that resolved before merging more things. But I'll get this in after that, thanks again!

[cgwalters]

/ok-to-test

[cgwalters]

/override ci/prow/fcos-e2e
At some point will need to debug Prow but for now

[openshift-ci]

@cgwalters: Overrode contexts on behalf of cgwalters: ci/prow/fcos-e2e

Details

In response to this:

/override ci/prow/fcos-e2e
At some point will need to debug Prow but for now

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[champtar]

To help with upgrades, would a verity-maybe make sense ?

[ruihe774]

To help with upgrades, would a verity-maybe make sense ?

You mean setting option verity=on on the overlayfs mount? Unfortunately, libcompose does not support it.

It only supports either no option (equivalent to verity=off) or verity=require.

https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html#fs-verity-support
https://github.com/containers/composefs/blob/2e98bef7a783a850b431273fd6928c1d5ff280f8/libcomposefs/lcfs-mount.h#L25

BTW, LCFS_MOUNT_FLAGS_TRY_VERITY is to try whether the kernel support the verity mount option, not to try whether fs-verity is enabled.

To upgrade, you can simply enable fs-verity for all files under repo/objects (There are existing tools to do that) before you deploy with composefs.enabled=verity.

[allisonkarlitskaya]

I'm late to the party here but is it too late to change the option? Before this change, afaik, all options were boolean and the config file was effectively toml. This takes us into a somewhat more freeform direction.... probably ini-ish.... that might make things more difficult in the future if we want to add more advanced features.

Something more like

[composefs]
enable=true
verity=true

instead?

[cgwalters]

This takes us into a somewhat more freeform direction.... probably ini-ish

It's always been ini, we've parsed using GKeyFile; and especially booleans in ini/GKeyFile accept values like "yes/no" and "0/1" that aren't valid in TOML.

You're right of course regardless that it'd definitely have been cleaner to use a separate boolean for verity=true. (That said, verity=true should probably imply enable=true just to simplify things further) Happy to do that now too.

I'm assuming you're asking this because you want to reimplement it in composefs-rs, but note there is already Rust code in bootc that's parsing this too, and I did have to adjust it for this in e.g. bootc-dev/bootc@06933ed