Skip to content

Migrate to newer Power, zSeries, and MacOS distros #1791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
eramongodb merged 5 commits into from
Nov 19, 2024
Merged

Migrate to newer Power, zSeries, and MacOS distros #1791

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
279a1fa
Migrate from rhel83-zseries to rhel8-zeries
eramongodb Nov 15, 2024
0de0f3b
Migrate from rhel81-power8 to rhel8-power
eramongodb Nov 15, 2024
710351a
Add new macos, power and zseries distros
eramongodb Nov 15, 2024
62239b2
Migrate to newer MacOS distros and recommended architectures
eramongodb Nov 15, 2024
a20096a
Remove replicas from pre-7.0 cse.darwinssl matrix
eramongodb Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .evergreen/config_generator/components/cse/darwinssl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,21 @@
# pylint: disable=line-too-long
# fmt: off
COMPILE_MATRIX = [
('macos-1100', 'clang', None, ['cyrus']),
('macos-14', 'clang', None, ['cyrus']),

('macos-11-arm64', 'clang', None, ['cyrus']),
('macos-14-arm64', 'clang', None, ['cyrus']),
]

# TODO (CDRIVER-3789): test cse with the 'sharded' topology.
TEST_MATRIX = [
('macos-1100', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['4.2', '4.4', '5.0', '6.0' ]),
('macos-14', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['4.2', '4.4', '5.0']),

('macos-11-arm64', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['6.0']),
('macos-14-arm64', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['6.0']),
Copy link
Collaborator

@kevinAlbs kevinAlbs Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
('macos-14', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['4.2', '4.4', '5.0']),
('macos-11-arm64', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['6.0']),
('macos-14-arm64', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['6.0']),
('macos-14', 'clang', None, 'cyrus', ['auth'], ['server'], ['4.2', '4.4', '5.0']),
('macos-11-arm64', 'clang', None, 'cyrus', ['auth'], ['server'], ['6.0']),
('macos-14-arm64', 'clang', None, 'cyrus', ['auth'], ['server'], ['6.0']),

Suggest removing replica for <7.0 server tests to reduce macOS testing. The openssl and winssl CSE tasks similarly test only server for <7.0 server. I expect testing only server for <7.0 was originally intended (given the comment # Test 7.0+ with a replica set).


# Test 7.0+ with a replica set since Queryable Encryption does not support the 'server' topology. Queryable Encryption tests require 7.0+.
('macos-1100', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], [ '7.0', '8.0', 'latest']),
('macos-14-arm64', 'clang', None, 'cyrus', ['auth'], ['server', 'replica' ], ['7.0', '8.0', 'latest']),
]
# fmt: on
# pylint: enable=line-too-long
Expand Down
14 changes: 7 additions & 7 deletions .evergreen/config_generator/components/cse/openssl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,23 +20,23 @@
('debian92', 'clang', None, ['cyrus']),
('debian92', 'gcc', None, ['cyrus']),
('rhel80', 'gcc', None, ['cyrus']),
('rhel83-zseries', 'gcc', None, ['cyrus']),
('ubuntu2004', 'clang', None, ['cyrus']),
('rhel8-zseries', 'gcc', None, ['cyrus']),
('ubuntu2004', 'clang', None, ['cyrus']),
('ubuntu2004', 'gcc', None, ['cyrus']),
('ubuntu2004-arm64', 'gcc', None, ['cyrus']),
('windows-vsCurrent', 'vs2017x64', None, ['cyrus']),
]

# TODO (CDRIVER-3789): test cse with the 'sharded' topology.
TEST_MATRIX = [
# 4.2 and 4.4 not available on rhel83-zseries.
('rhel83-zseries', 'gcc', None, 'cyrus', ['auth'], ['server'], ['5.0']),
# 4.2 and 4.4 not available on rhel8-zseries.
('rhel8-zseries', 'gcc', None, 'cyrus', ['auth'], ['server'], ['5.0']),

('windows-vsCurrent', 'vs2017x64', None, 'cyrus', ['auth'], ['server'], ['4.2', '4.4', '5.0', '6.0' ]),

# Test 7.0+ with a replica set since Queryable Encryption does not support the 'server' topology. Queryable Encryption tests require 7.0+.
('ubuntu2004', 'gcc', None, 'cyrus', ['auth'], ['server', 'replica'], ['4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
('rhel83-zseries', 'gcc', None, 'cyrus', ['auth'], ['server', 'replica'], [ '7.0', '8.0', 'latest']),
('rhel8-zseries', 'gcc', None, 'cyrus', ['auth'], ['server', 'replica'], [ '7.0', '8.0', 'latest']),
('ubuntu2004-arm64', 'gcc', None, 'cyrus', ['auth'], ['server', 'replica'], ['4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
('windows-vsCurrent', 'vs2017x64', None, 'cyrus', ['auth'], ['server', 'replica'], [ '7.0', '8.0', 'latest']),

Expand Down Expand Up @@ -80,7 +80,7 @@ def tasks():

# PowerPC and zSeries are limited resources.
for task in res:
if any(pattern in task.run_on for pattern in ["power8", "zseries"]):
if any(pattern in task.run_on for pattern in ["power", "zseries"]):
task.patchable = False

return res
Expand All @@ -95,7 +95,7 @@ def variants():

# PowerPC and zSeries are limited resources.
for task in TASKS:
if any(pattern in task.run_on for pattern in ["power8", "zseries"]):
if any(pattern in task.run_on for pattern in ["power", "zseries"]):
tasks.append(
TaskRef(
name=task.name,
Expand Down
11 changes: 8 additions & 3 deletions .evergreen/config_generator/components/sasl/darwinssl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,17 @@
# pylint: disable=line-too-long
# fmt: off
COMPILE_MATRIX = [
('macos-1100', 'clang', None, ['cyrus']),
('macos-1100-arm64', 'clang', None, ['cyrus']),
('macos-14', 'clang', None, ['cyrus']),

('macos-11-arm64', 'clang', None, ['cyrus']),
('macos-14-arm64', 'clang', None, ['cyrus']),
]

TEST_MATRIX = [
('macos-1100', 'clang', None, 'cyrus', ['auth'], ['server'], ['4.0', '4.2', '4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
('macos-14', 'clang', None, 'cyrus', ['auth'], ['server'], ['4.0', '4.2', '4.4', '5.0']),

('macos-11-arm64', 'clang', None, 'cyrus', ['auth'], ['server'], ['6.0', '7.0', '8.0', 'latest']),
('macos-14-arm64', 'clang', None, 'cyrus', ['auth'], ['server'], ['6.0', '7.0', '8.0', 'latest']),
]
# fmt: on
# pylint: enable=line-too-long
Expand Down
14 changes: 7 additions & 7 deletions .evergreen/config_generator/components/sasl/openssl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,17 @@
('debian92', 'clang', None, ['cyrus']),
('debian92', 'gcc', None, ['cyrus']),
('rhel80', 'gcc', None, ['cyrus']),
('rhel81-power8', 'gcc', None, ['cyrus']),
('rhel83-zseries', 'gcc', None, ['cyrus']),
('ubuntu2004', 'clang', None, ['cyrus']),
('rhel8-power', 'gcc', None, ['cyrus']),
('rhel8-zseries', 'gcc', None, ['cyrus']),
('ubuntu2004', 'clang', None, ['cyrus']),
('ubuntu2004-arm64', 'gcc', None, ['cyrus']),
('ubuntu2004', 'gcc', None, ['cyrus']),
('windows-vsCurrent', 'vs2017x64', None, ['cyrus']),
]

TEST_MATRIX = [
('rhel81-power8', 'gcc', None, 'cyrus', ['auth'], ['server', ], [ '4.2', '4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
('rhel83-zseries', 'gcc', None, 'cyrus', ['auth'], ['server', ], [ '5.0', '6.0', '7.0', '8.0', 'latest']),
('rhel8-power', 'gcc', None, 'cyrus', ['auth'], ['server'], ['4.2', '4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
('rhel8-zseries', 'gcc', None, 'cyrus', ['auth'], ['server'], [ '5.0', '6.0', '7.0', '8.0', 'latest']),

('ubuntu2004-arm64', 'gcc', None, 'cyrus', ['auth'], ['server'], ['4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
('ubuntu2004', 'gcc', None, 'cyrus', ['auth'], ['server'], ['4.4', '5.0', '6.0', '7.0', '8.0', 'latest']),
Expand Down Expand Up @@ -75,7 +75,7 @@ def tasks():

# PowerPC and zSeries are limited resources.
for task in res:
if any(pattern in task.run_on for pattern in ["power8", "zseries"]):
if any(pattern in task.run_on for pattern in ["power", "zseries"]):
task.patchable = False

return res
Expand All @@ -88,7 +88,7 @@ def variants():

# PowerPC and zSeries are limited resources.
for task in TASKS:
if any(pattern in task.run_on for pattern in ["power8", "zseries"]):
if any(pattern in task.run_on for pattern in ["power", "zseries"]):
tasks.append(
TaskRef(
name=task.name,
Expand Down
2 changes: 1 addition & 1 deletion .evergreen/config_generator/components/scan_build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
# pylint: disable=line-too-long
# fmt: off
MATRIX = [
('macos-1100', 'clang', None ),
('macos-14', 'clang', None ),
('ubuntu2004-arm64', 'clang', None ),
('ubuntu2004', 'clang', 'i686'),
]
Expand Down
23 changes: 14 additions & 9 deletions .evergreen/config_generator/etc/distros.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ class Distro(BaseModel):
'vsMulti',
] | None = None
size: Literal['small', 'large'] | None = None
arch: Literal['arm64', 'power8', 'zseries'] | None = None
arch: Literal['arm64', 'power', 'zseries'] | None = None

@validator('os_ver')
@classmethod
Expand All @@ -52,11 +52,12 @@ def validate_os_ver(cls, value):
]

MACOS_DISTROS = [
Distro(name='macos-1100', os='macos', os_type='macos', os_ver='11.00'),
Distro(name='macos-14', os='macos', os_type='macos', os_ver='14'),
]

MACOS_ARM64_DISTROS = [
Distro(name='macos-1100-arm64', os='macos', os_type='macos', os_ver='11.00', arch='arm64'),
Distro(name='macos-11-arm64', os='macos', os_type='macos', os_ver='11', arch='arm64'),
Distro(name='macos-14-arm64', os='macos', os_type='macos', os_ver='14', arch='arm64'),
]

RHEL_DISTROS = [
Expand All @@ -77,14 +78,18 @@ def validate_os_ver(cls, value):
Distro(name='rhel92-arm64-small', os='rhel', os_type='linux', os_ver='9.0', size='small', arch='arm64'),
]

RHEL_POWER8_DISTROS = [
Distro(name='rhel81-power8-large', os='rhel', os_type='linux', os_ver='8.1', size='large', arch='power8'),
Distro(name='rhel81-power8-small', os='rhel', os_type='linux', os_ver='8.1', size='small', arch='power8'),
RHEL_POWER_DISTROS = [
Distro(name='rhel8-power-large', os='rhel', os_type='linux', os_ver='8', size='large', arch='power'),
Distro(name='rhel8-power-small', os='rhel', os_type='linux', os_ver='8', size='small', arch='power'),
Distro(name='rhel9-power-large', os='rhel', os_type='linux', os_ver='9', size='large', arch='power'),
Distro(name='rhel9-power-small', os='rhel', os_type='linux', os_ver='9', size='small', arch='power'),
]

RHEL_ZSERIES_DISTROS = [
Distro(name='rhel83-zseries-large', os='rhel', os_type='linux', os_ver='8.3', size='large', arch='zseries'),
Distro(name='rhel83-zseries-small', os='rhel', os_type='linux', os_ver='8.3', size='small', arch='zseries'),
Distro(name='rhel8-zseries-large', os='rhel', os_type='linux', os_ver='8', size='large', arch='zseries'),
Distro(name='rhel8-zseries-small', os='rhel', os_type='linux', os_ver='8', size='small', arch='zseries'),
Distro(name='rhel9-zseries-large', os='rhel', os_type='linux', os_ver='9', size='large', arch='zseries'),
Distro(name='rhel9-zseries-small', os='rhel', os_type='linux', os_ver='9', size='small', arch='zseries'),
]

UBUNTU_DISTROS = [
Expand Down Expand Up @@ -134,7 +139,7 @@ def validate_os_ver(cls, value):
MACOS_ARM64_DISTROS + \
RHEL_DISTROS + \
RHEL_ARM64_DISTROS + \
RHEL_POWER8_DISTROS + \
RHEL_POWER_DISTROS + \
RHEL_ZSERIES_DISTROS + \
UBUNTU_DISTROS + \
UBUNTU_ARM64_DISTROS + \
Expand Down
35 changes: 21 additions & 14 deletions .evergreen/generated_configs/legacy-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16355,10 +16355,10 @@ buildvariants:
- link-with-cmake-snappy-deprecated
- name: link-with-cmake-mac
distros:
- macos-1100
- macos-14-arm64
- name: link-with-cmake-mac-deprecated
distros:
- macos-1100
- macos-14-arm64
- name: link-with-cmake-windows
distros:
- windows-vsCurrent-large
Expand All @@ -16376,15 +16376,15 @@ buildvariants:
- ubuntu2004-test
- name: link-with-pkg-config-mac
distros:
- macos-1100
- macos-14-arm64
- link-with-pkg-config-ssl
- link-with-bson
- name: link-with-bson-windows
distros:
- windows-vsCurrent-large
- name: link-with-bson-mac
distros:
- macos-1100
- macos-14-arm64
- name: link-with-bson-mingw
distros:
- windows-vsCurrent-large
Expand Down Expand Up @@ -16522,12 +16522,11 @@ buildvariants:
display_name: '*Darwin, macOS (Apple LLVM)'
expansions:
CC: clang
run_on: macos-1100
run_on: macos-14-arm64
tasks:
- .compression !.snappy
- release-compile
- debug-compile-nosasl-nossl
- debug-compile-rdtscp
- debug-compile-no-align
- debug-compile-nosrv
- debug-compile-sasl-darwinssl
Expand All @@ -16539,8 +16538,16 @@ buildvariants:
- debug-compile-lto
- debug-compile-lto-thin
- debug-compile-aws
- test-aws-openssl-regular-4.4
- test-aws-openssl-regular-latest
- name: darwin-intel
display_name: '*Darwin, Intel macOS (Apple LLVM)'
expansions:
CC: clang
run_on: macos-14
tasks:
- debug-compile-aws
- debug-compile-rdtscp
- test-aws-openssl-regular-4.4
- name: windows-2017-32
display_name: Windows (i686) (VS 2017)
expansions:
Expand Down Expand Up @@ -16605,11 +16612,11 @@ buildvariants:
run_on: windows-vsCurrent-large
tasks:
- debug-compile-no-align
- name: power8-rhel81
display_name: Power8 (ppc64le) (RHEL 8.1)
- name: rhel8-power
display_name: Power (ppc64le) (RHEL 8)
expansions:
CC: gcc
run_on: rhel81-power8-test
run_on: rhel8-power-large
tasks:
- release-compile
- debug-compile-nosasl-nossl
Expand All @@ -16634,11 +16641,11 @@ buildvariants:
- .latest .nossl
- test-dns-openssl
batchtime: 1440
- name: zseries-rhel83
- name: zseries-rhel8
display_name: '*zSeries'
expansions:
CC: gcc
run_on: rhel83-zseries-small
run_on: rhel8-zseries-large
tasks:
- release-compile
- debug-compile-no-align
Expand Down Expand Up @@ -16691,14 +16698,14 @@ buildvariants:
- name: debug-compile-nosasl-openssl-static
- name: debug-compile-nosasl-darwinssl
distros:
- macos-1100
- macos-14-arm64
- name: debug-compile-nosasl-winssl
distros:
- windows-vsCurrent-large
- name: .ocsp-openssl
- name: .ocsp-darwinssl
distros:
- macos-1100
- macos-14-arm64
- name: .ocsp-winssl
distros:
- windows-vsCurrent-large
Expand Down
Loading