odc fails on Fedora 38 and Centos Stream 9 images #20

runcom · 2023-11-30T13:04:23Z

Trying with the image in $title doesn't yield a working qcow2 - it just hangs in emergency shell where you can't even login as the root account is locked. No user added or nothing, it just doesn't get to a working prompt.

lmilbaum · 2023-12-06T16:52:11Z

Please switch to using quay.io/centos-bootc/centos-bootc:stream9. The one mentioned in the issue title should be deprecated soon enough.

runcom · 2023-12-06T17:05:10Z

thanks @lmilbaum ! I think this issue is still relevant with the new image too as bib hardcodes fedora and I have some more tests to make to understand what's going on

mvo5 · 2023-12-06T17:07:55Z

Fwiw, I added integration testing for odc in #34 and noticed that the quay.io/centos-bootc/centos-bootc:stream9 does not come up with working network - we are still investigating what is going on (we suspect we need to switch the buildroot in our code from fedora to centos).

mvo5 · 2023-12-12T15:33:36Z

We suspect that this happens because of /boot not being populated because of ostree stuff

mvo5 · 2023-12-13T10:04:07Z

A short update on this one. The resulting image fails to mount the /boot partition during boot at the fsck service in systemd. There is a warning: /dev/sda3 has unsupported feature(s): FEATURE_C12 [0] and it says: e2fsck: Get a newer version of e2fsck. Version 1.47.0 uses FEATURE_C12 by default [1] and we use that in our buildroot with mke2fs. However in the stream9 container we only have e2fsprogs 1.46.5.

We can easily workaround this by doing: mkfs.ext4 -O ^orphan_file when creating the filesystems. ~~Alternatively we can just specify to use fedora 38 (instead of 39) in the buildroot which gives us a slightly older e2fsprogs version that does not set this new flag.~~ (this will not work as we need a very current ostree version).

[0] https://github.com/tytso/e2fsprogs/blob/master/doc/RelNotes/v1.47.0.txt
[1] tytso/e2fsprogs#147

[edit: added another workaround]

ondrejbudai · 2023-12-13T13:26:53Z

Should we use Centos Stream 9 buildroot for building Centos Stream 9? Alternatively, we can use the container itself as a buildroot (this is how bootc install works).

mvo5 · 2023-12-13T14:42:49Z

We just had a chat about this and the agreement is to look into using the container itself as the buildroot and not constructing it from RPMs. This means that for the stream9 container we would use quay.io/centos-bootc/centos-bootc-dev as buildroot (we need this version because the version of ostree in the stream9 container is too old afaict) and for fedora quay.io/centos-bootc/fedora-bootc (or -dev if needed).

ondrejbudai · 2023-12-15T11:05:49Z

github got confused, this is not fixed yet.

This commit runs parameterized `bootc-image-builder` integration tests as part of the testing. The parameterization requires [0]. The test matrix right now is to generated a "qcow2" and "ami" image and for both: 1. test_image_is_generated(): just checks that the image can be build 2. test_image_boots(): the image boots in qemu and can be accessed via ssh 3. test_image_build_without_se_linux_denials(): ensure that the image is build without selinux denials Only test (3) strictly requires testing farm because it can only run inside a host that runs with a RHEL/fedora kernel and selinux enabled and it is not relevant for your testing (only required for bootc-image-builder). This is why this commit also adds a normal GH action - they support building the images and booting them with qemu so for this project testing farm is slightly less important. We can run arbitrary post-boot tests in step (2) if we want, at this point we have a ssh connection to the booted image so we can do any post-boot checks we consider important (code for this needs to be written though but shoudl be easy). Caveats: - bootc-image-builder cannot build quay.io/centos-bootc/centos-bootc:stream9 right now [1]. This is being worked on and we could start with testing the quay.io/centos-bootc/fedora-bootc:eln images until bib supports centos. Some ideas for further improvements: - tests could be extracted from bootc-image-builder into a shared bootc-testing repo - post-boot tests - upgrade tests, e.g. by building an image from the current "stable" bootc first and then upgrading to the bootc container that is tested I hope this is helpful! [0] osbuild/bootc-image-builder#87 [1] osbuild/bootc-image-builder#20

This commit runs parameterized `bootc-image-builder` integration tests as part of the testing. The parameterization requires [0]. The test matrix right now is to generated a "qcow2" and "ami" image and for both: 1. test_image_is_generated(): just checks that the image can be build 2. test_image_boots(): the image boots in qemu and can be accessed via ssh 3. test_image_build_without_se_linux_denials(): ensure that the image is build without selinux denials Only test (3) strictly requires testing farm because it can only run inside a host that runs with a RHEL/fedora kernel and selinux enabled and it is not relevant for your testing (only required for bootc-image-builder). This is why this commit also adds a normal GH action - they support building the images and booting them with qemu so for this project testing farm is slightly less important. We can run arbitrary post-boot tests in step (2) if we want, at this point we have a ssh connection to the booted image so we can do any post-boot checks we consider important (code for this needs to be written though but shoudl be easy). Caveats: - bootc-image-builder cannot build quay.io/centos-bootc/centos-bootc:stream9 right now [1]. This is being worked on and we could start with testing the quay.io/centos-bootc/fedora-bootc:eln images until bib supports centos. - `BIB_TEST_BOOTC_CONTAINER_TAG` is not set correctly right now, it needs to be parameterized with a reference to the bootc container that just got build and now needs testing. Some ideas for further improvements: - tests could be extracted from bootc-image-builder into a shared bootc-testing repo - post-boot tests - upgrade tests, e.g. by building an image from the current "stable" bootc first and then upgrading to the bootc container that is tested I hope this is helpful! [0] osbuild/bootc-image-builder#87 [1] osbuild/bootc-image-builder#20

This commit runs parameterized `bootc-image-builder` integration tests as part of the testing. The parameterization requires [0]. The test matrix right now is to generated a "qcow2" and "ami" image and for both: 1. test_image_is_generated(): just checks that the image can be build 2. test_image_boots(): the image boots in qemu and can be accessed via ssh [1] 3. test_image_build_without_se_linux_denials(): ensure that the image is build without selinux denials Only test (3) strictly requires testing farm because it can only run inside a host that runs with a RHEL/fedora kernel and selinux enabled and it is not relevant for your testing (only required for bootc-image-builder). This is why this commit also adds a normal GH action - they support building the images and booting them with qemu so for this project testing farm is slightly less important. We can run arbitrary post-boot tests in step (2) if we want, at this point we have a ssh connection to the booted image so we can do any post-boot checks we consider important (code for this needs to be written though but shoudl be easy). Caveats: - bootc-image-builder cannot build quay.io/centos-bootc/centos-bootc:stream9 right now [2]. This is being worked on and we could start with testing the quay.io/centos-bootc/fedora-bootc:eln images until bib supports centos. - `BIB_TEST_BOOTC_CONTAINER_TAG` is not set correctly right now, it needs to be parameterized with a reference to the bootc container that just got build and now needs testing. Some ideas for further improvements: - tests could be extracted from bootc-image-builder into a shared bootc-testing repo - post-boot tests - upgrade tests, e.g. by building an image from the current "stable" bootc first and then upgrading to the bootc container that is tested I hope this is helpful! [0] osbuild/bootc-image-builder#87 [1] https://github.com/osbuild/bootc-image-builder/blob/main/test/test_smoke.py#L128 [2] osbuild/bootc-image-builder#20

This commit runs parametrized `bootc-image-builder` integration tests as part of the testing. The parametrization requires [0]. The test matrix right now is to generated a "qcow2" and "ami" image and for both: 1. test_image_is_generated(): just checks that the image can be build 2. test_image_boots(): the image boots in qemu and can be accessed via ssh [1] 3. test_image_build_without_se_linux_denials(): ensure that the image is build without selinux denials Only test (3) strictly requires testing farm because it can only run inside a host that runs with a RHEL/fedora kernel and selinux enabled and it is not relevant for your testing (only required for bootc-image-builder). This is why this commit also adds a normal GH action - they support building the images and booting them with qemu so for this project testing farm is slightly less important. We can run arbitrary post-boot tests in step (2) if we want, at this point we have a ssh connection to the booted image so we can do any post-boot checks we consider important (code for this needs to be written though but should be easy). Caveats: - bootc-image-builder cannot build quay.io/centos-bootc/centos-bootc:stream9 right now [2]. This is being worked on and we could start with testing the quay.io/centos-bootc/fedora-bootc:eln images until bib supports centos. - `BIB_TEST_BOOTC_CONTAINER_TAG` is not set correctly right now, it needs to be parametrized with a reference to the bootc container that just got build and now needs testing. Some ideas for further improvements: - tests could be extracted from bootc-image-builder into a shared bootc-testing repo - post-boot tests - upgrade tests, e.g. by building an image from the current "stable" bootc first and then upgrading to the bootc container that is tested I hope this is helpful! [0] osbuild/bootc-image-builder#87 [1] https://github.com/osbuild/bootc-image-builder/blob/main/test/test_smoke.py#L128 [2] osbuild/bootc-image-builder#20

This commit runs parametrized `bootc-image-builder` integration tests as part of the testing. The parametrization requires [0]. The test matrix right now is to generated a "qcow2" and "ami" image and for both: 1. test_image_is_generated(): just checks that the image can be build 2. test_image_boots(): the image boots in qemu and can be accessed via ssh [1] 3. test_image_build_without_se_linux_denials(): ensure that the image is build without selinux denials We can run arbitrary post-boot tests in step (2) if we want, at this point we have a ssh connection to the booted image so we can do any post-boot checks we consider important (code for this needs to be written though but should be easy). Caveats: - bootc-image-builder cannot build quay.io/centos-bootc/centos-bootc:stream9 right now [2]. This is being worked on and we could start with testing the quay.io/centos-bootc/fedora-bootc:eln images until bib supports centos. Some ideas for further improvements: - tests could be extracted from bootc-image-builder into a shared bootc-testing repo - post-boot tests - upgrade tests, e.g. by building an image from the current "stable" bootc first and then upgrading to the bootc container that is tested I hope this is helpful! [0] osbuild/bootc-image-builder#87 [1] https://github.com/osbuild/bootc-image-builder/blob/main/test/test_smoke.py#L128 [2] osbuild/bootc-image-builder#20

cgwalters · 2024-01-25T20:29:22Z

This is fixed now by #138

This commit runs parametrized `bootc-image-builder` integration tests as part of the testing. The parametrization requires [0]. The test matrix right now is to generated a "qcow2" and "ami" image and for both: 1. test_image_is_generated(): just checks that the image can be build 2. test_image_boots(): the image boots in qemu and can be accessed via ssh [1] 3. test_image_build_without_se_linux_denials(): ensure that the image is build without selinux denials We can run arbitrary post-boot tests in step (2) if we want, at this point we have a ssh connection to the booted image so we can do any post-boot checks we consider important (code for this needs to be written though but should be easy). Caveats: - bootc-image-builder cannot build quay.io/centos-bootc/centos-bootc:stream9 right now [2]. This is being worked on and we could start with testing the quay.io/centos-bootc/fedora-bootc:eln images until bib supports centos. Some ideas for further improvements: - tests could be extracted from bootc-image-builder into a shared bootc-testing repo - post-boot tests - upgrade tests, e.g. by building an image from the current "stable" bootc first and then upgrading to the bootc container that is tested I hope this is helpful! [0] osbuild/bootc-image-builder#87 [1] https://github.com/osbuild/bootc-image-builder/blob/main/test/test_smoke.py#L128 [2] osbuild/bootc-image-builder#20

mvo5 changed the title ~~odc doesn't work with quay.io/centos-boot/centos-tier-1-dev:stream9~~ odc doesn't work with quay.io/centos-bootc/centos-bootc:stream9 Dec 13, 2023

mvo5 mentioned this issue Dec 13, 2023

stages(mkfs.ext4): add basic unit test osbuild/osbuild#1502

Merged

ondrejbudai closed this as completed in osbuild/osbuild#1502 Dec 15, 2023

ondrejbudai reopened this Dec 15, 2023

ondrejbudai mentioned this issue Dec 18, 2023

README.md: A few tweaks #17

Closed

mvo5 mentioned this issue Jan 5, 2024

github,tmt: run the bib integration tests CentOS/centos-bootc#119

Closed

cgwalters pinned this issue Jan 10, 2024

cgwalters mentioned this issue Jan 13, 2024

anaconda: bios boot for container-installer osbuild/images#360

Merged

cgwalters added the area/should-be-bootc Bugs that will be fixed when we switch to using bootc label Jan 16, 2024

ondrejbudai mentioned this issue Jan 18, 2024

Unable to build images that are not based on Fedora 39 #88

Closed

ondrejbudai changed the title ~~odc doesn't work with quay.io/centos-bootc/centos-bootc:stream9~~ odc fails on Fedora 38 and Centos Stream 9 images Jan 18, 2024

cgwalters closed this as completed Jan 25, 2024

ondrejbudai unpinned this issue Jan 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

odc fails on Fedora 38 and Centos Stream 9 images #20

odc fails on Fedora 38 and Centos Stream 9 images #20

runcom commented Nov 30, 2023

lmilbaum commented Dec 6, 2023

runcom commented Dec 6, 2023

mvo5 commented Dec 6, 2023

mvo5 commented Dec 12, 2023

mvo5 commented Dec 13, 2023 •

edited

Loading

ondrejbudai commented Dec 13, 2023

mvo5 commented Dec 13, 2023

ondrejbudai commented Dec 15, 2023

cgwalters commented Jan 25, 2024

odc fails on Fedora 38 and Centos Stream 9 images #20

odc fails on Fedora 38 and Centos Stream 9 images #20

Comments

runcom commented Nov 30, 2023

lmilbaum commented Dec 6, 2023

runcom commented Dec 6, 2023

mvo5 commented Dec 6, 2023

mvo5 commented Dec 12, 2023

mvo5 commented Dec 13, 2023 • edited Loading

ondrejbudai commented Dec 13, 2023

mvo5 commented Dec 13, 2023

ondrejbudai commented Dec 15, 2023

cgwalters commented Jan 25, 2024

mvo5 commented Dec 13, 2023 •

edited

Loading