feat(infra): Add base_os_version to support parallel builds #14128
Conversation
Introduces a hybrid approach for selecting the base-runner image to fix GLIBC version mismatches in the parallel trial_build CI pipeline. The image tag is now determined with the following priority: 1. --base-image-tag: A new command-line argument for CI. 2. base_os_version: A new field in project.yaml for local runs. 3. 'legacy': The default tag, preserving backward compatibility. A new GitHub Actions workflow is added to ensure consistency between the `base_os_version` in `project.yaml` and the `FROM` tag in the project's Dockerfile, preventing configuration errors. Fixes b/441792502
|
/gcbrun trial_build.py zlib bad_example --fuzzing-engines libfuzzer --sanitizers address --force |
|
/gcbrun trial_build.py zlib bad_example --fuzzing-engines libfuzzer --sanitizers address --force |
|
/gcbrun trial_build.py zlib bad_example --fuzzing-engines libfuzzer --sanitizers address --force |
| config = yaml.safe_load(file_handle) | ||
| version = 'legacy' | ||
| if config and 'base_os_version' in config: | ||
| version = config['base_os_version'] |
There was a problem hiding this comment.
I wonder why base_os_version is needed given that it can be extracted from dockerfiles? Currently ubuntu-24-04 should be specified in two different places to get it to work with Ubuntu 24.04 as far as I can see.
There was a problem hiding this comment.
Just to clarify it does work but I think ideally there should be one place where it should be specified instead of
diff --git a/projects/.../Dockerfile b/projects/.../Dockerfile
...
-FROM gcr.io/oss-fuzz-base/base-builder
+FROM gcr.io/oss-fuzz-base/base-builder:ubuntu-24-04
...
diff --git a/projects/.../project.yaml b/projects/.../project.yaml
...
+base_os_version: "ubuntu-24-04"It isn't the end of the world though.
There was a problem hiding this comment.
I understand your point. It is necessary in the project.yaml file because some projects, like ClusterFuzz, don't parse the Dockerfile; they only analyze the project.yaml file. That's why it's important there.
Think of it as a small redundancy to support the upgrade. In the future, this field can default to ubuntu-24-04 once we phase out ubuntu-20.04 entirely.
There was a problem hiding this comment.
Got it.
project.yaml is omitted in some projects using ClusterFuzzLite though so there it has to be created to bump the images.
…er image handling
|
/gcbrun trial_build.py zlib bad_example --fuzzing-engines libfuzzer --sanitizers address --force |
…aming for trial builds
|
/gcbrun trial_build.py zlib bad_example --fuzzing-engines libfuzzer --sanitizers address --force |
|
/gcbrun trial_build.py zlib bad_example --fuzzing-engines libfuzzer --sanitizers address --force |
…14144) Reverts #14128 This caused some breakages in https://github.com/google/oss-fuzz/actions/runs/18567323759/job/52931880439?pr=14064. `infra/helper.py` cannot have any external dependencies
With google/oss-fuzz#14112 and google/oss-fuzz#14128, we can now use Ubuntu 24.04. Let's bump the image version.
With google/oss-fuzz#14112 and google/oss-fuzz#14128, we can now use Ubuntu 24.04. Let's bump the image version. Note, the i386 build failure mentioned in the removed comment is related to https://bugs.launchpad.net/ubuntu/+source/linux-signed-azure/+bug/2071445 actions/runner-images#9977 and has been already fixed.
With google/oss-fuzz#14112 and google/oss-fuzz#14128, we can now use Ubuntu 24.04. Let's bump the image version. Note, the i386 build failure mentioned in the removed comment is related to https://bugs.launchpad.net/ubuntu/+source/linux-signed-azure/+bug/2071445 actions/runner-images#9977 and has been already fixed.
With google/oss-fuzz#14112 and google/oss-fuzz#14128, we can now use Ubuntu 24.04. Let's bump the image version. Note, the i386 build failure mentioned in the removed comment is related to https://bugs.launchpad.net/ubuntu/+source/linux-signed-azure/+bug/2071445 actions/runner-images#9977 and has been already fixed.
What this PR does
This PR resolves a critical GLIBC mismatch error in the parallel
trial_buildCI pipeline by enabling dynamic selection of thebase-runnerimage. It introduces a hybrid approach to ensure a project's runtime environment always matches its build environment.Why this PR is important
Currently, fuzzers built on a newer OS (e.g., Ubuntu 24.04) during parallel CI runs fail during the
check_buildstep because they are always tested against an older, hardcodedbase-runner:latestimage (Ubuntu 20.04). This change is crucial to unblock parallel builds and allow projects to use newer base images without breaking the CI.How the changes were implemented
trial_buildfunction now passes a--base-image-tagtohelper.py. This tag corresponds to the OS version of the build job (e.g.,ubuntu-24-04), ensuring thecheck_buildstep uses the correct runner image.base_os_versionfield is introduced inproject.yaml. This allows developers to specify a base OS for local runs ofrun_fuzzer,reproduce, etc., making local testing consistent with CI.helper.pythat use a runner image now call a refactored_get_base_runner_imagefunction. This function implements the priority system:--base-image-tag>base_os_version>legacydefault.check_base_os.yml) is added. It triggers on pull requests modifyingprojects/and verifies that thebase_os_versioninproject.yamlmatches theFROMtag in the project'sDockerfile, preventing configuration errors.reproduce --valgrind) was also corrected as part of the refactoring.How to test
trial_buildfor a project using a newerDockerfile(e.g., based on Ubuntu 24.04) should now pass thecheck_buildstep.base_os_version: ubuntu-24-04to aproject.yaml(assuming a matchingDockerfile) and confirm thatpython3 infra/helper.py run_fuzzer <project> <fuzzer>uses thegcr.io/oss-fuzz-base/base-runner:ubuntu-24-04image.Is it a breaking change?
No. The default behavior is unchanged. All existing projects without the new
base_os_versionfield will continue to use thelegacy(:latest) runner image, making the change fully backward-compatible.Related Task