Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop el-7, debian-buster, amazonlinux-2 #14990

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

drop el-7, debian-buster, amazonlinux-2 #14990

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3a03e2d
drop el-7, debian-buster, amazonlinux-2
Habbie Dec 20, 2024
c149fdd
remove %t hack, el8 does not need it
Habbie Feb 4, 2025
eaf9bca
simplify if grep call
Habbie Feb 4, 2025
7de1fe2
drop stray %endif
Habbie Feb 4, 2025
6131839
replace ; with &&
Habbie Feb 4, 2025
cc3fb63
oops
Habbie Feb 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 1 addition & 15 deletions .github/workflows/build-packages.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,8 @@ on:
# please remember to update the pkghashes below when you
# update this list, as well as the one in builder-dispatch.yml
default: >-
el-7
el-8
el-9
debian-buster
debian-bullseye
debian-bookworm
ubuntu-focal
Expand Down Expand Up @@ -73,20 +71,14 @@ jobs:
matrix:
os: ${{fromJson(needs.prepare.outputs.oslist)}}
runner-os: ${{ fromJson(needs.prepare.outputs.runnerlist )}}
exclude:
- os: el-7
runner-os: ubicloud-standard-2-arm
fail-fast: false
outputs:
product-name: ${{ steps.normalize-name.outputs.normalized-package-name }}
version: ${{ steps.getversion.outputs.version }}
pkghashes-el-7-x86_64: ${{ steps.pkghashes.outputs.pkghashes-el-7-x86_64 }}
pkghashes-el-8-x86_64: ${{ steps.pkghashes.outputs.pkghashes-el-8-x86_64 }}
pkghashes-el-8-aarch64: ${{ steps.pkghashes.outputs.pkghashes-el-8-aarch64 }}
pkghashes-el-9-x86_64: ${{ steps.pkghashes.outputs.pkghashes-el-9-x86_64 }}
pkghashes-el-9-aarch64: ${{ steps.pkghashes.outputs.pkghashes-el-9-aarch64 }}
pkghashes-debian-buster-x86_64: ${{ steps.pkghashes.outputs.pkghashes-debian-buster-x86_64 }}
pkghashes-debian-buster-aarch64: ${{ steps.pkghashes.outputs.pkghashes-debian-buster-aarch64 }}
pkghashes-debian-bullseye-x86_64: ${{ steps.pkghashes.outputs.pkghashes-debian-bullseye-x86_64 }}
pkghashes-debian-bullseye-aarch64: ${{ steps.pkghashes.outputs.pkghashes-debian-bullseye-aarch64 }}
pkghashes-debian-bookworm-x86_64: ${{ steps.pkghashes.outputs.pkghashes-debian-bookworm-x86_64 }}
Expand Down Expand Up @@ -173,7 +165,7 @@ jobs:
run: |
for os in ${{ inputs.os }}; do
for architecture in x86_64 ${{ vars.ARM64_USE_UBICLOUD == '1' && 'aarch64' || '' }}; do
[[ "$os" != "el-7" || "$architecture" != "aarch64" ]] && echo "\"pkghashes-$os-$architecture\"" | tee -a /tmp/os-inputs.txt
zeha marked this conversation as resolved.
Show resolved Hide resolved
echo "\"pkghashes-$os-$architecture\"" | tee -a /tmp/os-inputs.txt
done
done
sort -o /tmp/os-inputs.txt /tmp/os-inputs.txt
Expand All @@ -188,9 +180,6 @@ jobs:
matrix:
os: ${{fromJson(needs.prepare.outputs.oslist)}}
architecture: ${{ fromJson(needs.prepare.outputs.archlist )}}
exclude:
- os: el-7
architecture: aarch64
permissions:
actions: read # To read the workflow path.
id-token: write # To sign the provenance.
Expand Down Expand Up @@ -224,9 +213,6 @@ jobs:
matrix:
os: ${{fromJson(needs.prepare.outputs.oslist)}}
architecture: ${{ fromJson(needs.prepare.outputs.archlist )}}
exclude:
- os: el-7
architecture: aarch64
steps:
- name: Download source tarball provenance for ${{ inputs.product }} (${{ inputs.ref }})
id: download-src-provenance
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/builder-dispatch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,8 @@ on:
type: string
# please remember to update build-packages.yml as well
default: >-
el-7
el-8
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this list be in sync with the list in builder.yml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, in fact these days perhaps it should explicitly not overlap with it at all I think. builder-dispatch should contain a good default set for shipping packages (even if we don't builder-dispatch for releases any more). builder is older than all the more specific ones, and was for daily testing of distros we -don't- autobuild for in any other way.

builder-dispatch should be in sync with builder-packages (or ideally the list would live in one of them, but that's tricky). builder (which we should maybe give a useful suffix to explain what it does) should then cover distros we do not, or do not yet, ship for

el-9
debian-buster
debian-bullseye
debian-bookworm
ubuntu-focal
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/builder.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ jobs:
matrix:
product: ['authoritative', 'recursor', 'dnsdist']
os:
- el-7
- el-8
- centos-9-stream
- ubuntu-noble
Expand All @@ -44,9 +43,6 @@ jobs:
- debian-trixie
- amazon-2023
runner-os: ${{ fromJson(needs.prepare.outputs.runnerlist )}}
exclude:
- os: el-7
runner-os: ubicloud-standard-2-arm
fail-fast: false
steps:
- uses: actions/checkout@v4
Expand Down
12 changes: 3 additions & 9 deletions BUILDING-PACKAGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,20 +44,18 @@ Adding a new OS to the list

Adding a new OS is usually easy, provided that it does not differ too much from an existing one. For example, to add support for Debian Bookworm (already present in the current repository), one had to:

Copy the existing instructions for Debian Buster:
Copy the existing instructions for Debian bullseye:
```
cp builder-support/dockerfiles/Dockerfile.target.debian-buster builder-support/dockerfiles/Dockerfile.target.debian-bookworm
cp builder-support/dockerfiles/Dockerfile.target.debian-bullseye builder-support/dockerfiles/Dockerfile.target.debian-bookworm
```

In the new `builder-support/dockerfiles/Dockerfile.target.debian-bookworm` file, replace every occurrence of `debian-buster` by `debian-bookworm`, and of `debian:buster` by `debian:bookworm`
In the new `builder-support/dockerfiles/Dockerfile.target.debian-bookworm` file, replace every occurrence of `debian-bullseye` by `debian-bookworm`, and of `debian:bullseye` by `debian:bookworm`

Then add the new target to the list of OSes in the `.github/workflows/builder-dispatch.yml` workflow file:
```
default: >-
el-7
el-8
el-9
debian-buster
debian-bullseye
debian-bookworm
ubuntu-focal
Expand All @@ -68,10 +66,8 @@ If release packages should be automatically built for this new target, then `.gi
``
```
default: >-
el-7
el-8
el-9
debian-buster
debian-bullseye
debian-bookworm
ubuntu-focal
Expand All @@ -80,10 +76,8 @@ default: >-

Not forgetting to update the list of hashes later in the same file:
```
pkghashes-el-7: ${{ steps.pkghashes.outputs.pkghashes-el-7 }}
pkghashes-el-8: ${{ steps.pkghashes.outputs.pkghashes-el-8 }}
pkghashes-el-9: ${{ steps.pkghashes.outputs.pkghashes-el-9 }}
pkghashes-debian-buster: ${{ steps.pkghashes.outputs.pkghashes-debian-buster }}
pkghashes-debian-bullseye: ${{ steps.pkghashes.outputs.pkghashes-debian-bullseye }}
pkghashes-debian-bookworm: ${{ steps.pkghashes.outputs.pkghashes-debian-bookworm }}
pkghashes-ubuntu-focal: ${{ steps.pkghashes.outputs.pkghashes-ubuntu-focal }}
Expand Down
5 changes: 0 additions & 5 deletions build-scripts/docker/repo-test/generate-repo-files.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,11 +140,8 @@ def write_release_files (release):
'rec-48', 'rec-49', 'rec-50', 'rec-51', 'rec-master',
'dnsdist-17', 'dnsdist-18', 'dnsdist-19', 'dnsdist-master']:
write_pkg_pin_file(release)
write_dockerfile('el', '7', release)
write_dockerfile('el', '8', release)
write_dockerfile('el', '9', release)
write_dockerfile('debian', 'buster', release)
write_list_file('debian', 'buster', release)
write_dockerfile('debian', 'bullseye', release)
write_list_file('debian', 'bullseye', release)
write_dockerfile('ubuntu', 'focal', release)
Expand Down Expand Up @@ -241,8 +238,6 @@ def test_release (release, arch='x86_64'):
returned_versions = []
print('=== testing {} ({}) ==='.format(release, arch))
for df in dockerfiles:
if arch == 'aarch64' and str(df).endswith('el-7'):
continue
if arch == 'aarch64' and not release in ['rec-49', 'rec-50', 'rec-51', 'rec-master',
'dnsdist-19', 'dnsdist-master']:
continue
Expand Down
8 changes: 0 additions & 8 deletions builder-support/dockerfiles/CentOS-SCLo-scl-rh.repo
View file
Open in desktop

This file was deleted.

34 changes: 0 additions & 34 deletions builder-support/dockerfiles/Centos-Base-Vault.repo
View file
Open in desktop

This file was deleted.

42 changes: 10 additions & 32 deletions builder-support/dockerfiles/Dockerfile.rpmbuild
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
FROM dist-base as package-builder
RUN touch /var/lib/rpm/* && if $(grep -q 'release 7' /etc/redhat-release); then \
yum upgrade -y && \
yum install -y rpm-build rpmdevtools python2 python3 curl jq "@Development Tools"; \
else \
yum upgrade -y && \
yum install --allowerasing -y rpm-build rpmdevtools python3 curl jq "@Development Tools"; \
fi
RUN yum upgrade -y && \
yum install --allowerasing -y rpm-build rpmdevtools python3 curl jq "@Development Tools"

RUN mkdir /dist /pdns
WORKDIR /pdns
Expand All @@ -21,14 +16,10 @@ RUN cd /pdns/builder-support/helpers/ && ./install_rust.sh
@ENDIF

@IF [ -n "$M_dnsdist$M_all" ]
# We do not build Quiche (DNS over QUIC support) on el-7 because the clang
# version is too old to build the 'boring-sys' crate needed by Quiche
RUN if ! $(grep -q 'release 7' /etc/redhat-release); then \
cd /pdns/builder-support/helpers/ && ./install_rust.sh; \
yum install -y git cmake clang; \
cd /pdns/builder-support/helpers/ && \
./install_quiche.sh; \
fi
RUN cd /pdns/builder-support/helpers/ && ./install_rust.sh && \
yum install -y git cmake clang && \
cd /pdns/builder-support/helpers/ && \
./install_quiche.sh
@ENDIF

# Used for -p option to only build specific spec files
Expand All @@ -43,19 +34,15 @@ ADD builder-support/specs/ /pdns/builder-support/specs
RUN find /pdns/builder-support/specs/ -not -name '*.spec' -exec ln -s {} /root/rpmbuild/SOURCES/ \;

@IF [ -n "$M_authoritative$M_all" ]
RUN touch /var/lib/rpm/* && if $(grep -q 'release 7' /etc/redhat-release); then \
scl enable devtoolset-11 -- builder/helpers/build-specs.sh builder-support/specs/pdns.spec; \
elif $(grep -q 'release 8' /etc/redhat-release); then \
RUN touch /var/lib/rpm/* && if grep -q 'release 8' /etc/redhat-release; then \
scl enable gcc-toolset-13 -- builder/helpers/build-specs.sh builder-support/specs/pdns.spec; \
else \
builder/helpers/build-specs.sh builder-support/specs/pdns.spec; \
fi
@ENDIF

@IF [ -n "$M_recursor$M_all" ]
RUN touch /var/lib/rpm/* && if $(grep -q 'release 7' /etc/redhat-release); then \
scl enable devtoolset-11 -- builder/helpers/build-specs.sh builder-support/specs/pdns-recursor.spec; \
elif $(grep -q 'release 8' /etc/redhat-release); then \
RUN touch /var/lib/rpm/* && if grep -q 'release 8' /etc/redhat-release; then \
scl enable gcc-toolset-13 -- builder/helpers/build-specs.sh builder-support/specs/pdns-recursor.spec; \
else \
builder/helpers/build-specs.sh builder-support/specs/pdns-recursor.spec; \
Expand All @@ -64,29 +51,20 @@ RUN touch /var/lib/rpm/* && if $(grep -q 'release 7' /etc/redhat-release); then

@IF [ -n "$M_dnsdist$M_all" ]

# --allowerasing does not exist on el7, so we fall back to just installing
# this is fine because --allowerasing is only there to deal with libcurl conflicting with libcurl-minimal on some el9 images
RUN touch /var/lib/rpm/* && mkdir /libh2o && cd /libh2o && \
yum install -y --allowerasing curl libcurl openssl-devel cmake || yum install -y curl libcurl openssl-devel cmake && \
dnf install -y --allowerasing curl libcurl openssl-devel cmake && \
cd /pdns/builder-support/helpers/ && \
./install_h2o.sh

RUN touch /var/lib/rpm/* && if $(grep -q 'release 7' /etc/redhat-release); then \
scl enable devtoolset-11 -- builder/helpers/build-specs.sh builder-support/specs/dnsdist.spec; \
elif $(grep -q 'release 8' /etc/redhat-release); then \
RUN touch /var/lib/rpm/* && if grep -q 'release 8' /etc/redhat-release; then \
scl enable gcc-toolset-13 -- builder/helpers/build-specs.sh builder-support/specs/dnsdist.spec; \
else \
builder/helpers/build-specs.sh builder-support/specs/dnsdist.spec; \
fi
@ENDIF

# Generate provenance
@IF [ "${BUILDER_TARGET}" = "el-7" -o "${BUILDER_TARGET}" = "centos-7" ]
@EVAL RUN python builder/helpers/generate-yum-provenance.py /dist/packages-${BUILDER_TARGET}.json || python3 builder/helpers/generate-yum-provenance.py /dist/packages-${BUILDER_TARGET}.json
@ENDIF
@IF [ "${BUILDER_TARGET}" != "el-7" -a "${BUILDER_TARGET}" != "centos-7" ]
@EVAL RUN python builder/helpers/generate-dnf-provenance.py /dist/packages-${BUILDER_TARGET}.json || python3 builder/helpers/generate-dnf-provenance.py /dist/packages-${BUILDER_TARGET}.json
@ENDIF

# mv across layers with overlay2 is buggy in some kernel versions (results in empty dirs)
# See: https://github.com/moby/moby/issues/33733
Expand Down
19 changes: 5 additions & 14 deletions builder-support/dockerfiles/Dockerfile.rpmsbom
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,23 @@
ARG BUILDER_VERSION
@EVAL ARG BUILDER_TARGET=${BUILDER_TARGET}

# No SBOM on el-7, which only has Python 2 and will be EOL soon anyway
RUN touch /var/lib/rpm/* && \
if ! $(grep -q 'release 7' /etc/redhat-release); then \
yum install -y python3 python3-dnf; \
fi
yum install -y python3 python3-dnf
ADD builder-support/helpers/ /pdns/builder-support/helpers/

@IF [ -n "$M_recursor$M_all" ]
RUN cd /pdns/builder-support/helpers/ && \
if ! $(grep -q 'release 7' /etc/redhat-release); then \
python3 generate-sbom-dnf.py /dist/pdns-recursor-${BUILDER_VERSION}-${BUILDER_TARGET}.cyclonedx.json pdns-recursor rust.json; \
fi
python3 generate-sbom-dnf.py /dist/pdns-recursor-${BUILDER_VERSION}-${BUILDER_TARGET}.cyclonedx.json pdns-recursor rust.json
@ENDIF

@IF [ -n "$M_dnsdist$M_all" ]
RUN cd /pdns/builder-support/helpers/; \
if ! $(grep -q 'release 7' /etc/redhat-release); then \
python3 generate-sbom-dnf.py /dist/dnsdist-${BUILDER_VERSION}-${BUILDER_TARGET}.cyclonedx.json dnsdist rust.json quiche.json h2o.json; \
fi
python3 generate-sbom-dnf.py /dist/dnsdist-${BUILDER_VERSION}-${BUILDER_TARGET}.cyclonedx.json dnsdist rust.json quiche.json h2o.json
@ENDIF

@IF [ -n "$M_authoritative$M_all" ]
RUN cd /pdns/builder-support/helpers/ && \
if ! $(grep -q 'release 7' /etc/redhat-release); then \
for pkg in $(dnf list installed 'pdns*' | grep -E ^'pdns' | grep -vE '\-debuginfo|\-debugsource' | cut -d. -f1); do \
for pkg in $(dnf list installed 'pdns*' | grep -E ^'pdns' | grep -vE '\-debuginfo|\-debugsource' | cut -d. -f1); do \
python3 generate-sbom-dnf.py /dist/${pkg}-${BUILDER_VERSION}-${BUILDER_TARGET}.cyclonedx.json ${pkg}; \
done; \
fi
done
@ENDIF
15 changes: 0 additions & 15 deletions builder-support/dockerfiles/Dockerfile.target.amazon-2
View file
Open in desktop

This file was deleted.

17 changes: 0 additions & 17 deletions builder-support/dockerfiles/Dockerfile.target.centos-7
View file
Open in desktop

This file was deleted.

28 changes: 0 additions & 28 deletions builder-support/dockerfiles/Dockerfile.target.debian-buster
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion builder-support/dockerfiles/Dockerfile.target.el-7
View file
Open in desktop

This file was deleted.

Loading