Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kernel: Create kernel-devel squashfs and bind into host containers #701

Merged
merged 2 commits into from
Feb 7, 2020

Conversation

sam-aws
Copy link
Contributor

@sam-aws sam-aws commented Jan 31, 2020

Issue #, if available:
#680

Description of changes:

As part of the kernel build install the kernel development headers into
a squashfs file installed onto the Thar host. On boot this will be
unpacked to make the headers available, saving a significant amount of
space in the root partition.

For superpowered host containers host-ctr bind mounts in

    /lib/modules -> /lib/modules, and
    /var/lib/thar/kernel -> /usr/src/kernels

allowing the kernel headers to be transparently accessed from within the
container.

Signed-off-by: Samuel Mendoza-Jonas [email protected]

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.


Tested by launching a Thar instance, ensuring the kernel headers are visible in both the host and container, and compiling a module against them successfully.

This allows them to be installed into the filesystem image without
creating a file conflict over "/usr".

Signed-off-by: Ben Cressey <[email protected]>
Comment on lines 9 to 11
What=/usr/share/thar/kernel-devel.squashfs
Where=/var/lib/thar/kernel
Type=squashfs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to mount this on /usr/src/kernels so that it shows up in a semi-standard place for bind mounting into orchestrated containers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC I went this way because /usr/src is read-only when mounting this; I'll go back and double check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actual issue here ends up being systemd complaining out symlinks in the path; I've worked around this by inserting the correct %{_cross_usrsrc} path into the .mount file at build time, and systemd-izing the filename.

@@ -28,7 +28,8 @@ BuildRequires: hostname
BuildRequires: kmod
BuildRequires: openssl-devel

%global kernel_sourcedir %{_usrsrc}/kernels/%{version}
%global kernel_sourcedir %{_cross_usrsrc}/kernels/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can omit trailing slash

done
mksquashfs src_squashfs kernel-devel.squashfs
install -D kernel-devel.squashfs %{buildroot}/%{_cross_datadir}/thar/kernel-devel.squashfs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: ... %{buildroot}%{_cross_datadir} ... to eliminate extra '/' in expanded form

Comment on lines 125 to 131
# For completeness set up a link in the host to the (eventual) headers location
install -d %{buildroot}%{kernel_sourcedir}
ln -sf %{_cross_sharedstatedir}/thar/kernel/%{version} %{buildroot}%{kernel_sourcedir}/%{version}

# Replace the incorrect links from modules_install. These will be bound
# into a host container (and unused in the host) so they must not point
# to %{_cross_usrsrc} (eg. /x86_64-thar-linux-gnu/sys-root/...)
rm -f %{buildroot}%{kernel_libdir}/build %{buildroot}%{kernel_libdir}/source
ln -sf %{_usrsrc}/kernels/%{version} %{buildroot}%{kernel_libdir}/build
ln -sf %{_usrsrc}/kernels/%{version} %{buildroot}%{kernel_libdir}/source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we mount the squashfs at /usr/src/kernels (per comment below), that would also make these links non-broken on the host.

We wouldn't need the "completeness" symlink but would still want the directory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixup here is separate from the mountpoint; these links by default point to the build tree:

[ec2-user@ip-192-168-11-152 ~]$ ls -l /lib/modules/4.19.75/build
lrwxrwxrwx 1 root root 42 Feb  3 17:56 /lib/modules/4.19.75/build -> /home/builder/rpmbuild/BUILD/linux-4.19.75

@sam-aws
Copy link
Contributor Author

sam-aws commented Feb 3, 2020

Pushed a new version to address @bcressey's comments.

Description=Kernel Development Sources
DefaultDependencies=no
Conflicts=umount.target
RequiresMountsFor=/var/lib/thar
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This RequiresMountsFor doesn't seem accurate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, forgot to update that in the latest change.

As part of the kernel build install the kernel development headers into
a squashfs file installed onto the Thar host. On boot this will be
unpacked to make the headers available, saving a significant amount of
space in the root partition.

For superpowered host containers host-ctr bind mounts in
	/lib/modules -> /lib/modules, and
	/usr/src/kernels -> /usr/src/kernels
allowing the kernel headers to be transparently accessed from within the
container.

Signed-off-by: Samuel Mendoza-Jonas <[email protected]>
@sam-aws
Copy link
Contributor Author

sam-aws commented Feb 4, 2020

Updated to remove the old "RequiresMountsFor" line as well as the "DefaultDependencies=no" as this mount has no special dependencies or requirements to be mounted early.
Cleaned up the .mount file also to use the proper path for both source and destination.

@bcressey bcressey requested a review from iliana February 7, 2020 01:48
@sam-aws sam-aws merged commit 9f52acf into develop Feb 7, 2020
@sam-aws sam-aws deleted the kernel branch February 7, 2020 16:54
@zmarouf zmarouf mentioned this pull request Mar 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants