Add bootc internals cfs #1477
Merged
Add bootc internals cfs #1477
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Split into a few separate crates:
- libraries:
- composefs
- composefs-oci
- composefs-boot
- binaries:
- cfsctl
- composefs-setup-root
- erofs-debug
Move our lint config (which only forbids missing debug impls) to the
workspace level and have all crates inherit from that.
Add a new workflow for testing that we can `cargo package` everything.
We need a nightly cargo in order to do this with workspaces containing
inter-dependent crates: rust-lang/cargo#13947
Make 'oci' an optional feature of cfsctl, but enable it by default.
Adjust our rawhide bls example (which included --no-default-features) to
*not* disable that.
This is not a huge improvement in terms of compile speed, and it has
some drawbacks (like 'cargo run' no longer defaulting to cfsctl) but it
seems like the right step at this point. I want to start to add some
more experimental code without making it part of the main crate.
Signed-off-by: Allison Karlitskaya <[email protected]>
Grub needs absolute paths to initrd and vmlinuz if we do not have `/boot` in a boot partition, which we do not in bootc. Add param `boot_subdir` which acts like a subdirectory in the boot directory in case the boot partition is mounted in another directory. Signed-off-by: Pragyan Poudyal <[email protected]>
It turns out that the information contained in splitstreams to assist
with garbage collection (ie: the list of things that we mustn't discard)
is exactly the required information for downloading (ie: the list of
things that we must acquire).
Use this fact to add support for fetching repository content from HTTP
servers. We only download the objects that are actually required, so
incremental pulls are very fast.
This works with just about any HTTP server, so you can do something like
python -m http.server -d ~/.var/lib/composefs
and download from that. With a fast enough web server on localhost,
pulling a complete image into an empty repository takes about as long as
pulling an `oci:` directory via skopeo with `cfsctl oci pull`.
In practice, this is intended to be used with a webserver which supports
static compression and pre-compressed objects stored on the server. In
particular, zstd support is enabled in the `reqwest` crate for this
reason, and it's working with something like:
find repo/objects/ -type f -name '*[0-9a-f]' -exec zstd -19 -v '{}' +
static-web-server -p 8888 --compression-static -d repo
There's also an included s3-uploader.py in the examples/ directory which
will upload a repository to an S3 bucket, with zstd compression.
Signed-off-by: Allison Karlitskaya <[email protected]>
Change the Repository::mount() API to return the mounted filesystem as an fd rather than taking the mountpoint as an argument. Create a new mount_at() API to replace the old one, replacing the canicalize() and mount_at() calls that used to be in mount_composefs_at(), which we remove. Update the various users. Making this change lets us simplify the logic in composefs-setup-root: it no longer has to manually open the image in order to perform the fsmount operation: it can use the new API on the repository. This allows us to make Repository::open_image() private, so do that too. Co-authored-by: Sanne Raymaekers <[email protected]> Signed-off-by: Allison Karlitskaya <[email protected]>
Allows cfsctl operations with fs-verity disabled. Signed-off-by: Sanne Raymaekers <[email protected]>
Supports writing `composefs=?<root-id>`. Signed-off-by: Sanne Raymaekers <[email protected]>
Signed-off-by: Pragyan Poudyal <[email protected]>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request effectively integrates cfsctl functionality as an internal bootc command. The changes are well-structured, and the integration of the forked code is clean. I've found a critical issue that could lead to a panic and a suggestion to improve code clarity and align it with modern Rust idioms. Great work on integrating this!
This exposes the current functionality of the cfsctl binary. It's not a crate right now, and it's not a lot of code, so we just fork it. I did take the effort to use `git subtree merge` to do the import. For the record, here's how I did it: - In composefs-rs: git subtree split --prefix=crates/cfsctl - In bootc: git subtree add --prefix=crates/lib/cfsctl ../../containers/composefs-rs/ <git sha1 from above> In cfsctl I also: - Adjusted it to accept the bootc-configured composefs repo (which note is right now hardcoded to sha512, not sha256) - Dropped the http stuff since I don't think it really makes sense vs OCI Signed-off-by: Colin Walters <[email protected]>
cgwalters
force-pushed
the
store-composefs-2
branch
from
09b046f to
9d3ccd0
Compare
jeckersb
approved these changes
Aug 1, 2025
Collaborator
jeckersb
left a comment
jeckersb
left a comment
There was a problem hiding this comment.
Went over this with @cgwalters on a call 👍
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This exposes the current functionality of the cfsctl binary.
It's not a crate right now, and it's not a lot of code, so we just
fork it.
I did take the effort to use
git subtree mergeto do the import.For the record, here's how I did it:
In cfsctl I also:
(which note is right now hardcoded to sha512, not sha256)
sense vs OCI
Signed-off-by: Colin Walters [email protected]