Skip to content

Conversation

@ckyrouac
Copy link
Collaborator

@ckyrouac ckyrouac commented Jul 8, 2025

This adds a new "specific_components" parameter to process_mapping to enable mapping individual files to specific layers. This will be used by the rechunker in rpm-ostree.

Related to: coreos/rpm-ostree#5431

@ckyrouac ckyrouac marked this pull request as draft July 8, 2025 17:48
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for exclusive chunks, which allows specific components to be placed in their own layers. The implementation is well-structured, and includes a comprehensive set of new unit tests. Suggestions have been provided to improve code clarity, reduce test code duplication, and strengthen a test assertion.

@ckyrouac ckyrouac force-pushed the chunking-xattrs branch 3 times, most recently from bd38bb6 to 7f35f22 Compare July 9, 2025 19:31
@ckyrouac ckyrouac marked this pull request as ready for review July 10, 2025 16:21
@ckyrouac ckyrouac changed the title WIP: chunking: Support exclusive chunks defined via xattrs chunking: Support exclusive chunks defined via xattrs Jul 10, 2025
Copy link
Collaborator

@cgwalters cgwalters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all very complex code (mostly my fault!) and I find it hard to get it all in my head, but overall this looks reasonable to me.

I think what would really help is seeing this more "end to end" in combination with coreos/rpm-ostree#5431 - especially via a Dockerfile build that sets xattrs and verifies the chunking.

Maybe we could try standing up a CI job that clones your rpm-ostree PR and runs a test job from there? But optional from my PoV right now.

@ckyrouac
Copy link
Collaborator Author

This is all very complex code (mostly my fault!) and I find it hard to get it all in my head

Yep, took me awhile to wrap my head around. Might take a stab at refactoring it a bit after this gets in.

I think what would really help is seeing this more "end to end" in combination with coreos/rpm-ostree#5431 - especially via a Dockerfile build that sets xattrs and verifies the chunking.

Maybe we could try standing up a CI job that clones your rpm-ostree PR and runs a test job from there? But optional from my PoV right now.

I've been testing locally using this Containerfile (thanks claude) and using my local checkout of bootc in the rpm-ostree Cargo.toml. I can take a stab at doing this in our CI.

FROM quay.io/fedora/fedora-bootc:42

# Install attr package to set extended attributes
RUN dnf install -y attr

# Create some test files with different content
RUN echo "Database component file 1" > /usr/local/db-config.conf
RUN echo "Database component file 2" > /usr/local/db-schema.sql
RUN echo "Web server component file 1" > /usr/local/nginx.conf
RUN echo "Web server component file 2" > /usr/local/web-static.html
RUN echo "Monitoring component file" > /usr/local/metrics.json
RUN echo "Cache component file 1" > /usr/local/redis.conf
RUN echo "Cache component file 2" > /usr/local/cache-policy.json

# Set user.component extended attributes for different components (using user namespace since system may not be supported)
RUN setfattr -n user.component -v "database" /usr/local/db-config.conf
RUN setfattr -n user.component -v "database" /usr/local/db-schema.sql
RUN setfattr -n user.component -v "webserver" /usr/local/nginx.conf
RUN setfattr -n user.component -v "webserver" /usr/local/web-static.html
RUN setfattr -n user.component -v "monitoring" /usr/local/metrics.json
RUN setfattr -n user.component -v "cache" /usr/local/redis.conf
RUN setfattr -n user.component -v "cache" /usr/local/cache-policy.json

# Verify the extended attributes are set correctly
RUN echo "=== Verifying extended attributes ===" && \
    getfattr -n user.component /usr/local/db-config.conf && \
    getfattr -n user.component /usr/local/db-schema.sql && \
    getfattr -n user.component /usr/local/nginx.conf && \
    getfattr -n user.component /usr/local/web-static.html && \
    getfattr -n user.component /usr/local/metrics.json && \
    getfattr -n user.component /usr/local/redis.conf && \
    getfattr -n user.component /usr/local/cache-policy.json

# Also create some files without component xattrs for comparison
RUN echo "No component file 1" > /usr/local/no-component-1.txt
RUN echo "No component file 2" > /usr/local/no-component-2.txt

# Display file listing with extended attributes
RUN echo "=== Final file listing with xattrs ===" && \
    ls -la /usr/local/ && \
    echo "=== Extended attributes summary ===" && \
    find /usr/local/ -name "*.conf" -o -name "*.sql" -o -name "*.html" -o -name "*.json" | \
    xargs -I {} sh -c 'echo "File: {}"; getfattr -n user.component {} 2>/dev/null || echo "  No user.component xattr"'

@cgwalters cgwalters merged commit 0ac3ef1 into bootc-dev:main Jul 11, 2025
34 checks passed
@cgwalters
Copy link
Collaborator

I can take a stab at doing this in our CI.

I think it could also live in the rpm-ostree CI and then we just run that job from here, but ultimately it's also fine if it just lives in rpm-ostree I think. Hmm, maybe what could actually make sense is to move the chunking code there instead since there's 0% chance anything else uses it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants