Skip to content

NO-JIRA: fix(e2e): use public multi-arch image for ARM64 karpenter test - #8532

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
maxcao13:fix-karpenter-arm-image
May 21, 2026
Merged

NO-JIRA: fix(e2e): use public multi-arch image for ARM64 karpenter test#8532
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
maxcao13:fix-karpenter-arm-image

Conversation

@maxcao13

@maxcao13 maxcao13 commented May 15, 2026

Copy link
Copy Markdown
Member

quay.io/hypershift/sleep:multiarch is a private image that requires auth not available in CI pull secrets, causing ImagePullBackOff. Use registry.k8s.io/pause:3.10 which is public and multi-arch.

See rehearsals failures here: openshift/release#79262

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes

Special notes for your reviewer:

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • Tests
    • Updated ARM64 provisioning tests to use a different container image to improve cross-architecture validation and test reliability.
    • Modified the test workload to run a persistent, long-lived process so pods remain running throughout end-to-end tests, reducing flakiness and improving stability.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot

Copy link
Copy Markdown

@maxcao13: This pull request explicitly references no jira issue.

Details

In response to this:

quay.io/hypershift/sleep:multiarch is a private image that requires auth not available in CI pull secrets, causing ImagePullBackOff. Use registry.k8s.io/pause:3.10 which is public and multi-arch.

See rehearsals failures here: openshift/release#79262

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes

Special notes for your reviewer:

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 15, 2026
@coderabbitai

coderabbitai Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 5bb17c72-167c-4b27-8c75-d1a1c3c7f3b5

📥 Commits

Reviewing files that changed from the base of the PR and between 4cedb62 and 83b7c2d.

📒 Files selected for processing (1)
  • test/e2e/karpenter_test.go

📝 Walkthrough

Walkthrough

This pull request updates the ARM64 provisioning subtest in the Karpenter e2e tests: it changes the workload image passed to testWorkloadWithImage from quay.io/hypershift/sleep:multiarch to registry.access.redhat.com/ubi10/ubi-minimal:10.1, and modifies testWorkloadWithImage to set the container command to "/bin/sh -c "sleep infinity"" so the Deployment pod stays running for the duration of the test.

Possibly related PRs

  • openshift/hypershift#8498: Refactors the ARM64 provisioning subtest in the same test file to use a local hostedCluster.DeepCopy() for assertions and waits.
  • openshift/hypershift#8510: Also updates the ARM64 provisioning test to use the registry.k8s.io/pause:3.10 image for the provisioned workload.

Suggested reviewers

  • enxebre
  • muraee
🚥 Pre-merge checks | ✅ 9 | ❌ 3

❌ Failed checks (3 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Microshift Test Compatibility ⚠️ Warning New test file test/e2e/karpenter_test.go uses OpenshiftEC2NodeClass and other Karpenter/OpenShift-specific APIs without MicroShift skip markers or guards. Add [apigroup:karpenter.sh] tag or [Skipped:MicroShift] label to TestKarpenter, or guard with exutil.IsMicroShiftCluster() check since Karpenter APIs unavailable on MicroShift.
Single Node Openshift (Sno) Test Compatibility ⚠️ Warning New test helper function testWorkloadWithImage added with PodAntiAffinity requiring multiple nodes, lacks SNO protection check or [Skipped:SingleReplicaTopology] label. Guard testWorkloadWithImage callers with exutil.IsSingleNode() check and skip, or refactor to conditionally disable PodAntiAffinity on SNO, or add [Skipped:SingleReplicaTopology] label.
✅ Passed checks (9 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: replacing a private multi-arch image with a public one for the ARM64 karpenter test, which directly addresses the ImagePullBackOff issue in CI.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed This test file uses standard Go t.Run() testing, not Ginkgo. All test names are static, descriptive strings with no dynamic values. Changes only affect helper function parameters.
Test Structure And Quality ✅ Passed Test code meets all five quality criteria: single responsibility, proper cleanup with t.Cleanup(), timeouts via helper functions, meaningful assertion messages, consistent with codebase patterns.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies E2E test code only (test/e2e/karpenter_test.go with //go:build e2e tag), not production deployment manifests or operator controllers. Custom check applies to production code only.
Ote Binary Stdout Contract ✅ Passed PR changes are pure configuration/data updates within test helper functions; no process-level code is modified and no stdout writes are introduced.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR does not add new Ginkgo e2e tests, only modifies existing test infrastructure. Changes are IPv6-compatible and contain no IPv4 assumptions or external connectivity requirements.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
openshift-ci Bot requested review from Nirshal and cblecker May 15, 2026 19:47
@openshift-ci openshift-ci Bot added area/testing Indicates the PR includes changes for e2e testing and removed do-not-merge/needs-area labels May 15, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/karpenter_test.go`:
- Line 319: The test hardcodes the image "registry.k8s.io/pause:3.10" when
creating armWorkLoads via testWorkloadWithImage, which prevents overriding in
disconnected CI; make the image configurable by reading a test parameter or
environment variable (e.g., TEST_PAUSE_IMAGE or a test flag) and pass that value
into testWorkloadWithImage for both armWorkLoads and any similar calls, falling
back to "registry.k8s.io/pause:3.10" if the variable is unset so local runs keep
the current behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4c05d8f1-85cd-4639-bdc5-018f09ae1cf3

📥 Commits

Reviewing files that changed from the base of the PR and between b419df5 and c4ca08f.

📒 Files selected for processing (1)
  • test/e2e/karpenter_test.go

Comment thread test/e2e/karpenter_test.go Outdated
@codecov

codecov Bot commented May 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 40.34%. Comparing base (d86f3d4) to head (83b7c2d).
⚠️ Report is 10 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8532   +/-   ##
=======================================
  Coverage   40.34%   40.34%           
=======================================
  Files         755      755           
  Lines       93167    93167           
=======================================
  Hits        37587    37587           
  Misses      52877    52877           
  Partials     2703     2703           
Flag Coverage Δ
cmd-support 34.30% <ø> (ø)
cpo-hostedcontrolplane 41.76% <ø> (ø)
cpo-other 40.14% <ø> (ø)
hypershift-operator 50.72% <ø> (ø)
other 31.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joshbranham

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 15, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@cwbotbot

cwbotbot commented May 15, 2026

Copy link
Copy Markdown

Test Results

e2e-aws

e2e-aks

@joshbranham

Copy link
Copy Markdown
Contributor

/retest-required

1 similar comment
@cblecker

Copy link
Copy Markdown
Member

/retest-required

@maxcao13

Copy link
Copy Markdown
Member Author

Wow, haven't seen that one before.

/retest-required

Comment thread test/e2e/karpenter_test.go Outdated
@maxcao13
maxcao13 force-pushed the fix-karpenter-arm-image branch from c4ca08f to d1c9e6f Compare May 18, 2026 18:47
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label May 18, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/karpenter_test.go`:
- Line 319: Replace the non-deterministic UBI image tag used when creating the
ARM workload: in the call to testWorkloadWithImage that constructs armWorkLoads,
change the image string "registry.access.redhat.com/ubi9/ubi-minimal:latest" to
a pinned, specific version (e.g.,
"registry.access.redhat.com/ubi9/ubi-minimal:9.7" or a timestamped build tag) so
tests are reproducible; update only the image argument in the armWorkLoads
assignment where testWorkloadWithImage is invoked.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 995fac25-1d90-4bbf-9969-eeca78d73ec5

📥 Commits

Reviewing files that changed from the base of the PR and between c4ca08f and d1c9e6f.

📒 Files selected for processing (1)
  • test/e2e/karpenter_test.go

Comment thread test/e2e/karpenter_test.go Outdated

@joshbranham joshbranham left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 18, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@maxcao13
maxcao13 force-pushed the fix-karpenter-arm-image branch from d1c9e6f to 4cedb62 Compare May 18, 2026 19:30
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label May 18, 2026
@csrwng

csrwng commented May 18, 2026

Copy link
Copy Markdown
Contributor

/approve

@openshift-ci

openshift-ci Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: csrwng, maxcao13

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 18, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@maxcao13

Copy link
Copy Markdown
Member Author

/retest-required

@maxcao13

Copy link
Copy Markdown
Member Author

/verified by @maxcao13

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label May 19, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@maxcao13: This PR has been marked as verified by @maxcao13.

Details

In response to this:

/verified by @maxcao13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@maxcao13

Copy link
Copy Markdown
Member Author

/retest

1 similar comment
@maxcao13

Copy link
Copy Markdown
Member Author

/retest

quay.io/hypershift/sleep:multiarch is a private image that requires
auth not available in CI pull secrets, causing ImagePullBackOff.
Use registry.access.redhat.com/ubi10/ubi-minimal:10.1 which is public and multi-arch.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Max Cao <macao@redhat.com>
@maxcao13
maxcao13 force-pushed the fix-karpenter-arm-image branch from 4cedb62 to 83b7c2d Compare May 20, 2026 19:40
@openshift-ci-robot openshift-ci-robot removed the verified Signifies that the PR passed pre-merge verification criteria label May 20, 2026
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label May 20, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@maxcao13: This pull request explicitly references no jira issue.

Details

In response to this:

quay.io/hypershift/sleep:multiarch is a private image that requires auth not available in CI pull secrets, causing ImagePullBackOff. Use registry.k8s.io/pause:3.10 which is public and multi-arch.

See rehearsals failures here: openshift/release#79262

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes

Special notes for your reviewer:

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • Tests
  • Updated ARM64 provisioning tests to use a different container image to improve cross-architecture validation and test reliability.
  • Modified the test workload to run a persistent, long-lived process so pods remain running throughout end-to-end tests, reducing flakiness and improving stability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@joshbranham

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 20, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@maxcao13

Copy link
Copy Markdown
Member Author

/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws

@maxcao13

Copy link
Copy Markdown
Member Author

/test e2e-aks
/test e2e-aws-4-22

@maxcao13

Copy link
Copy Markdown
Member Author

/test e2e-aws-4-22

@maxcao13

Copy link
Copy Markdown
Member Author

/verified by @maxcao13

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label May 21, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@maxcao13: This PR has been marked as verified by @maxcao13.

Details

In response to this:

/verified by @maxcao13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

@maxcao13: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/testing Indicates the PR includes changes for e2e testing jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants