Skip to content

CNTRLPLANE-3509: Resolve N-minor release images via release controller API - #79627

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
jparrill:CNTRLPLANE-3509
Jun 5, 2026
Merged

CNTRLPLANE-3509: Resolve N-minor release images via release controller API#79627
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
jparrill:CNTRLPLANE-3509

Conversation

@jparrill

@jparrill jparrill commented May 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace ci-operator release imports (n1minor through n4minor) with runtime resolution via the release controller REST API
  • Each nXminor entry in releases: spawned a concurrent import pod, causing race conditions that failed ~40-45% of e2e-aws runs before tests even start
  • The new approach adds a resolve_release_image() function to the 4 run-e2e scripts that queries https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/{version}.0-0.ci/latest
  • Versions are passed as NODEPOOL_NX_VERSION env vars from the job config
  • Same pattern used by telco5g and hypershift-mce multi-version-test teams
  • Scoped to main branch config only; release branches will follow in a separate PR

Changes

openshift-hypershift-main.yaml

  • Removed all nXminor release definitions (n1 through n4)
  • Removed OCP_IMAGE_NX dependency overrides from all jobs
  • Added NODEPOOL_N1_VERSION through NODEPOOL_N4_VERSION env vars to each job

4 run-e2e ref YAMLs

  • Added default (empty) env vars for NODEPOOL_NX_VERSION

4 run-e2e command scripts

  • Added resolve_release_image() function that queries the release controller API
  • Added resolution block before existing N-version comparison logic

Test plan

  • Run /pj-rehearse to validate affected jobs
  • Verify resolve_release_image() returns valid pullspecs in CI environment
  • Confirm jq is available in the hypershift-tests image
  • Validate N1-N4 test flags are correctly passed to the e2e binary

Summary by CodeRabbit

This PR updates OpenShift CI configuration for HyperShift end-to-end tests to resolve N-minor OpenShift release images at runtime via the release controller API instead of declaring them as static ci-operator release imports (n2/n3/n4).

What changed (practical impact)

  • Added a new step-registry step and script (hypershift-resolve-nodepool-releases) that:
  • Added the hypershift-resolve-nodepool-releases step to affected Hypershift e2e workflows (AWS external/nested/metrics/oidc, Azure AKS, Azure self-managed and v2 self-managed, nested workflows, etc.) so downstream steps source the resolved images before running e2e.
  • Reworked ci-operator job definitions in ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml:
    • Removed static release imports for n2minor, n3minor, n4minor (keep n1minor).
    • Removed OCP_IMAGE_N1..N4 dependency overrides and added NODEPOOL_N1_VERSION..NODEPOOL_N4_VERSION environment variables (defaults added empty in run-e2e ref YAMLs).
  • Updated run-e2e and related command scripts (AWS external/nested, Azure run-e2e/self-managed/create-selfmanaged-guests, etc.) to conditionally source ${SHARED_DIR}/nodepool_release_images when present and use resolved OCP_IMAGE_Nx variables for constructing e2e arguments.
  • Added owners and metadata for the new resolve-nodepool-releases step.

Why this was done

  • Pre-importing multiple N-minor releases created many concurrent import pods and race conditions in ci-tools import logic, causing high pre-test failure rates (~40–45% for e2e-aws).
  • N2–N4 were judged redundant given existing coverage (N4 covered by unit tests; N2/N3 exercise the same code paths as N1) and a companion hypershift PR removes corresponding e2e test flags.
  • Runtime resolution reduces concurrent imports (from six to three per job), reduces CI runtime (~15–30 minutes per job expected), and avoids import race conditions while preserving necessary image resolution for configured nodepool version tests.

Scope and notes

  • Changes apply to main branch CI configuration; release-branch backports will be submitted separately.
  • The new step requires curl and jq to be available in the test image and relies on NODEPOOL_NX_VERSION env vars supplied by job configs; the step writes ${SHARED_DIR}/nodepool_release_images for downstream consumption.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 21, 2026
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 21, 2026
@openshift-ci

openshift-ci Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci-robot

openshift-ci-robot commented May 21, 2026

Copy link
Copy Markdown
Contributor

@jparrill: This pull request references CNTRLPLANE-3509 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

  • Remove n2minor, n3minor, and n4minor release definitions from openshift-hypershift-main.yaml
  • Remove OCP_IMAGE_N2, OCP_IMAGE_N3, OCP_IMAGE_N4 dependencies from 8 jobs: e2e-aks, e2e-aks-4-22, e2e-aws, e2e-azure-self-managed, e2e-aws-minimal, e2e-aws-techpreview, e2e-aws-4-22, e2e-aws-autonode

Why

The e2e-aws job imports 6+ releases concurrently, triggering a race condition in ci-tools (pkg/steps/release/import_release.go) where CLI extraction pods disappear or hit UID mismatches. This causes ~40-45% of e2e-aws runs to fail before any test executes.

The n2/n3/n4 releases were only used by NodePool version skew tests that are redundant:

  • N4 (unsupported skew): already covered by unit tests in TestValidateVersionSkew
  • N3 (n-3 boundary): same labels/taints validation as N1
  • N2 (n-2): same code path as N1 with a different version number

Companion PR

openshift/hypershift#8570 — removes the e2e test cases and CLI flags

Impact

  • Reduces release imports from 6 to 3 per job, cutting CI time by ~15-30 min
  • Step registry still declares OCP_IMAGE_N2/N3/N4 as optional dependencies with defaults, so release branch configs are unaffected
  • No loss of test coverage

Jira: https://issues.redhat.com/browse/CNTRLPLANE-3509

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds a CI step that resolves NODEPOOL_N{1..4}_VERSION to OCP_IMAGE_N{n} pull specs, wires that step into HyperShift e2e workflows, updates job configs to accept NODEPOOL_N{n}_VERSION inputs, and makes e2e and guest scripts source ${SHARED_DIR}/nodepool_release_images.

Changes

Nodepool Release Image Resolution for HyperShift E2E

Layer / File(s) Summary
Core resolve-nodepool-releases step
ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh, ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.yaml, ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json, ci-operator/step-registry/hypershift/resolve-nodepool-releases/OWNERS
New Bash script resolves up to four NODEPOOL_N{1..4}_VERSION inputs to .pullSpec via the release-stream API, writes OCP_IMAGE_N{n}="..." into ${SHARED_DIR}/nodepool_release_images. Step registry YAML, metadata JSON, and OWNERS added.
Workflow wiring across AWS and Azure variants
ci-operator/step-registry/hypershift/aws/e2e/..., ci-operator/step-registry/hypershift/azure/...
Inserted - ref: hypershift-resolve-nodepool-releases into multiple e2e workflow steps.pre chains so resolution runs before install/test steps.
Job configuration refactor to use nodepool env variables
ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml
Updated several e2e job entries to remove steps.dependencies for OCP_IMAGE_N*, add NODEPOOL_N1_VERSION..NODEPOOL_N4_VERSION env inputs as appropriate, and removed some n2minor/n3minor/n4minor release candidates under 4.22.
E2E and guest scripts source resolved release images
ci-operator/step-registry/hypershift/aws/run-e2e/..., ci-operator/step-registry/hypershift/azure/run-e2e/..., ci-operator/step-registry/hypershift/azure/create-selfmanaged-guests/...
Scripts now conditionally source ${SHARED_DIR}/nodepool_release_images when present so OCP_IMAGE_N* variables from the resolve step can be used to build e2e arguments; create-selfmanaged-guests ref updated to release:latest.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

lgtm, area/pipelines, rehearsals-ack

Suggested reviewers

  • enxebre
🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title directly corresponds to the main objective: it references the Jira ticket (CNTRLPLANE-3509) and accurately describes the core change—resolving N-minor release images via the release controller API instead of static imports.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR modifies only CI/CD configuration files and scripts; no Ginkgo test definitions with dynamic names are present, making the check not applicable.
Test Structure And Quality ✅ Passed This PR contains no Ginkgo test code. It only modifies CI/CD configuration files, shell scripts, and metadata—not test code. The custom check is not applicable to this PR.
Microshift Test Compatibility ✅ Passed No Ginkgo e2e tests added. All 19 files modified are CI infrastructure: YAML workflows, shell scripts, metadata files. No Go test code present.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR adds no new Ginkgo e2e tests; changes are CI infrastructure only (YAML config, shell helpers, workflow definitions). Check not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed This PR modifies only CI test infrastructure files, not deployment manifests or operator code. The topology-aware scheduling check does not apply.
Ote Binary Stdout Contract ✅ Passed PR modifies only CI configuration and setup scripts, not test binaries. No Go source files modified; OTE Binary Stdout Contract applies only to test binary code, which is not changed here.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests added. PR modifies only CI infrastructure (YAML configs, shell scripts, metadata). Companion e2e test changes are in a separate hypershift PR.
No-Weak-Crypto ✅ Passed No weak crypto algorithms, custom implementations, or unsafe secret comparisons found. PR uses curl and jq to fetch public container image pullspecs.
Container-Privileges ✅ Passed No privileged container settings found in any PR files. Changes only add release resolution utilities and update job configs without introducing privileged containers.
No-Sensitive-Data-In-Logs ✅ Passed No sensitive data exposure found. PR logs only public API endpoints, public container image references (pullspecs), and file paths to mounted credentials (not the credentials themselves).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 21, 2026
@jparrill jparrill changed the title CNTRLPLANE-3509: Remove n2/n3/n4 minor release imports from hypershift main branch jobs CNTRLPLANE-3509: Resolve N-minor release images via release controller API Jun 2, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-ref.yaml (1)

72-77: ⚡ Quick win

NODEPOOL_N3_VERSION/NODEPOOL_N4_VERSION are unused in the self-managed flow.

The self-managed commands script only constructs --e2e.n1/n2-minor-release-image args (and the dependencies block here only declares OCP_IMAGE_N1/OCP_IMAGE_N2), so these two env entries have no effect. Drop them, or wire up N3/N4 args + dependencies if n-3/n-4 skew is intended for self-managed.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-ref.yaml`
around lines 72 - 77, The NODEPOOL_N3_VERSION and NODEPOOL_N4_VERSION
environment entries are unused in the self-managed flow; either remove them from
the job spec or wire them into the self-managed commands and dependencies. If
removing: delete NODEPOOL_N3_VERSION and NODEPOOL_N4_VERSION from the env list
and any related references. If enabling n-3/n-4 skew: add corresponding
dependency entries (e.g. OCP_IMAGE_N3/OCP_IMAGE_N4) to the dependencies block
and update the self-managed command construction to include
--e2e.n3-minor-release-image and --e2e.n4-minor-release-image flags sourced from
NODEPOOL_N3_VERSION/NODEPOOL_N4_VERSION so the variables are actually used.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh`:
- Around line 53-55: The curl call that populates pullspec in
resolve_release_image uses --retry/--retry-delay but no per-attempt timeouts, so
add --connect-timeout and --max-time flags to the curl invocation that sets
pullspec (the command assigned to the pullspec variable) to bound each attempt
(e.g. short connect and overall per-attempt limits); update the same helper
occurrences duplicated in hypershift AWS nested/external and Azure scripts
(where resolve_release_image or the pullspec assignment is present), and either
parameterize the releasestream host by arch or add a note/check after retrieving
pullSpec to ensure the returned pullSpec matches AWS_MULTI_ARCH handling.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh`:
- Around line 62-67: The two unused resolution blocks for
NODEPOOL_N3_VERSION/NODEPOOL_N4_VERSION (which call resolve_release_image and
set OCP_IMAGE_N3/OCP_IMAGE_N4) are dead because the script only emits
--e2e.n1/n2-minor-release-image; remove the entire
NODEPOOL_N3_VERSION/NODEPOOL_N4_VERSION blocks and any references or exported
env vars for OCP_IMAGE_N3 and OCP_IMAGE_N4 to eliminate the dead assignments, or
alternatively if you intended to include N3/N4 skew tests, add the corresponding
e2e args (e.g., --e2e.n3-minor-release-image/--e2e.n4-minor-release-image)
wherever arguments are built so the resolve_release_image calls for
NODEPOOL_N3_VERSION/NODEPOOL_N4_VERSION are actually consumed.

---

Nitpick comments:
In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-ref.yaml`:
- Around line 72-77: The NODEPOOL_N3_VERSION and NODEPOOL_N4_VERSION environment
entries are unused in the self-managed flow; either remove them from the job
spec or wire them into the self-managed commands and dependencies. If removing:
delete NODEPOOL_N3_VERSION and NODEPOOL_N4_VERSION from the env list and any
related references. If enabling n-3/n-4 skew: add corresponding dependency
entries (e.g. OCP_IMAGE_N3/OCP_IMAGE_N4) to the dependencies block and update
the self-managed command construction to include --e2e.n3-minor-release-image
and --e2e.n4-minor-release-image flags sourced from
NODEPOOL_N3_VERSION/NODEPOOL_N4_VERSION so the variables are actually used.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: b668e5b1-a47f-4fe4-a9cc-1cec1e0e1614

📥 Commits

Reviewing files that changed from the base of the PR and between fbf2481 and d326951.

📒 Files selected for processing (9)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml
  • ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh
  • ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-ref.yaml
  • ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh
  • ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-ref.yaml
  • ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh
  • ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-ref.yaml
  • ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh
  • ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-ref.yaml
✅ Files skipped from review due to trivial changes (1)
  • ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-ref.yaml

Comment on lines +53 to +55
pullspec=$(curl -q -L -s --retry 5 --retry-delay 10 \
"https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/${version}.0-0.${stream}/latest" \
| jq -r ".pullSpec // empty")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
curl -q -L -s --connect-timeout 10 --max-time 30 \
  "https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/4.19.0-0.ci/latest" | jq '{name, phase, pullSpec}'

Repository: openshift/release

Length of output: 230


🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n --fixed-string 'api/v1/releasestream/${version}.0-0.${stream}/latest' ci-operator/step-registry || true
rg -n --fixed-string 'pullSpec // empty' ci-operator/step-registry || true
rg -n --fixed-string 'releasestream/' ci-operator/step-registry || true

Repository: openshift/release

Length of output: 1405


🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n --fixed-strings 'api/v1/releasestream/${version}.0-0.${stream}/latest' ci-operator/step-registry || true
rg -n --fixed-strings 'pullSpec // empty' ci-operator/step-registry || true
rg -n --fixed-strings 'releasestream/' ci-operator/step-registry || true

Repository: openshift/release

Length of output: 3399


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate any multi-arch handling in hypershift AWS e2e scripts
rg -n 'AWS_MULTI_ARCH|MULTI_ARCH|multi[-_ ]arch|arch' ci-operator/step-registry/hypershift/aws/run-e2e ci-operator/step-registry/hypershift/aws || true

# Show the relevant curl blocks (and nearby variables) in the four hypershift scripts that match the releasestream URL.
for f in \
  ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh \
  ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh \
  ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh \
  ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh
do
  echo "==== $f (around curl) ===="
  sed -n '35,75p' "$f" | nl -ba
done

Repository: openshift/release

Length of output: 3345


🏁 Script executed:

#!/bin/bash
set -euo pipefail

for f in \
  ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh \
  ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh \
  ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh \
  ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh
do
  echo "==== $f (around curl) ===="
  sed -n '45,65p' "$f"
  echo
done

for f in \
  ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh \
  ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh
do
  echo "==== $f (AWS_MULTI_ARCH handling & e2e args) ===="
  sed -n '30,55p' "$f"
  echo
  sed -n '135,170p' "$f"
  echo
done

Repository: openshift/release

Length of output: 7110


🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n "arm64\.ocp\.releases\.ci\.openshift\.org|amd64\.ocp\.releases\.ci\.openshift\.org" ci-operator/step-registry/hypershift ci-operator/step-registry/telco5g ci-operator/step-registry/openshift || true
rg -n "latest-release-image|previous-release-image|OCP_IMAGE_LATEST|OCP_IMAGE_PREVIOUS|OCP_IMAGE_N1" ci-operator/step-registry/hypershift/aws/run-e2e ci-operator/step-registry/hypershift/azure/run-e2e || true

Repository: openshift/release

Length of output: 6976


Add per-attempt connect/transfer timeouts to release-resolver curl

resolve_release_image uses curl --retry/--retry-delay without --connect-timeout/--max-time, so a stalled connection to the releasestream API can block the step until the overall step timeout; bound each attempt with timeouts (same helper is duplicated in hypershift AWS nested/external and Azure self-managed/regular scripts). The releasestream/.../latest response includes a .pullSpec field (so the jq -r ".pullSpec // empty" extraction remains valid).

  • Potential arch skew: these scripts enable --e2e.aws-multi-arch=true when AWS_MULTI_ARCH=true, but still resolve the payload via the amd64.ocp... releasestream URL; confirm the returned pullSpec is multi-arch (or parameterize by arch).
♻️ Bound each attempt
-    pullspec=$(curl -q -L -s --retry 5 --retry-delay 10 \
+    pullspec=$(curl -q -L -s --connect-timeout 10 --max-time 60 --retry 5 --retry-delay 10 \
         "https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/${version}.0-0.${stream}/latest" \
         | jq -r ".pullSpec // empty")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
pullspec=$(curl -q -L -s --retry 5 --retry-delay 10 \
"https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/${version}.0-0.${stream}/latest" \
| jq -r ".pullSpec // empty")
pullspec=$(curl -q -L -s --connect-timeout 10 --max-time 60 --retry 5 --retry-delay 10 \
"https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/${version}.0-0.${stream}/latest" \
| jq -r ".pullSpec // empty")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh`
around lines 53 - 55, The curl call that populates pullspec in
resolve_release_image uses --retry/--retry-delay but no per-attempt timeouts, so
add --connect-timeout and --max-time flags to the curl invocation that sets
pullspec (the command assigned to the pullspec variable) to bound each attempt
(e.g. short connect and overall per-attempt limits); update the same helper
occurrences duplicated in hypershift AWS nested/external and Azure scripts
(where resolve_release_image or the pullspec assignment is present), and either
parameterize the releasestream host by arch or add a note/check after retrieving
pullSpec to ensure the returned pullSpec matches AWS_MULTI_ARCH handling.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jparrill, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't checkout base SHA d3e7de493e246c25b9e7d4317d8589a1231d7f6c: error checking out "d3e7de493e246c25b9e7d4317d8589a1231d7f6c": exit status 128 fatal: unable to read tree (d3e7de493e246c25b9e7d4317d8589a1231d7f6c)

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@jparrill
jparrill marked this pull request as ready for review June 2, 2026 16:23
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 2, 2026
@jparrill

jparrill commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jparrill: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci
openshift-ci Bot requested review from bryan-cox and csrwng June 2, 2026 16:24
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jparrill, pj-rehearse: unable to set up jobs ERROR:

failed resolve ReleaseBuildConfiguration: Failed resolve MultiStageTestConfiguration: [test/e2e-azure-v2-self-managed: workflow/hypershift-azure-e2e-v2-self-managed: parameter "NODEPOOL_N1_VERSION" is overridden in [test/e2e-azure-v2-self-managed] but not declared in any step, test/e2e-azure-v2-self-managed: workflow/hypershift-azure-e2e-v2-self-managed: parameter "NODEPOOL_N2_VERSION" is overridden in [test/e2e-azure-v2-self-managed] but not declared in any step]

If the problem persists, please contact Test Platform.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh (1)

70-92: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Same unbound OCP_IMAGE_N* risk under nounset.

Identical pattern to the other run-e2e scripts: the source on Line 71 is optional but Lines 75/80/85/90 dereference ${OCP_IMAGE_N1..N4} without defaults, so a missing file or unwritten variable aborts under set -o nounset (Line 28). Add the non-empty guard.

🛡️ Suggested guard
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi

 N3_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N3} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N3:-}" && "${OCP_IMAGE_N3}" != "${OCP_IMAGE_LATEST}" ]]; then
   N3_NP_VERSION_TEST_ARGS="--e2e.n3-minor-release-image=${OCP_IMAGE_N3}"
 fi

 N4_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N4} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N4:-}" && "${OCP_IMAGE_N4}" != "${OCP_IMAGE_LATEST}" ]]; then
   N4_NP_VERSION_TEST_ARGS="--e2e.n4-minor-release-image=${OCP_IMAGE_N4}"
 fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh`
around lines 70 - 92, The script dereferences OCP_IMAGE_N1..OCP_IMAGE_N4 without
guarding against unset values (breaking under set -o nounset); update the
conditional checks that set N1_NP_VERSION_TEST_ARGS, N2_NP_VERSION_TEST_ARGS,
N3_NP_VERSION_TEST_ARGS and N4_NP_VERSION_TEST_ARGS to use a non-empty guard or
safe parameter expansion (e.g. test [[ -n "${OCP_IMAGE_N1:-}" &&
"${OCP_IMAGE_N1:-}" != "${OCP_IMAGE_LATEST}" ]] before assigning) so the
optional source of nodepool_release_images can be absent without causing an
unbound variable error.
ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh (1)

39-51: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Same unbound OCP_IMAGE_N1/OCP_IMAGE_N2 risk under nounset.

As in the AWS nested script, the source on Line 40 is optional (-f guard) but Lines 44/49 reference ${OCP_IMAGE_N1}/${OCP_IMAGE_N2} without a default. With set -o nounset (Line 19), a missing shared file or an unwritten variable aborts the job. Apply the same non-empty guard.

🛡️ Suggested guard
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh`
around lines 39 - 51, The comparisons reference OCP_IMAGE_N1 and OCP_IMAGE_N2
while the shared file is optional and nounset is enabled; update the conditions
to avoid unbound variable errors by using parameter expansion or explicit
non-empty checks — e.g. change the ifs to check that the variable is set and
non-empty (if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" !=
"${OCP_IMAGE_LATEST}" ]]; then ...) and similarly for OCP_IMAGE_N2, or use
${OCP_IMAGE_N1:-} and ${OCP_IMAGE_N2:-} in the comparisons to provide safe
defaults.
ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh (1)

46-68: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fix nounset failure from potentially-unset OCP_IMAGE_N1..N4 when sourcing nodepool_release_images

hypershift-aws-run-e2e-nested-commands.sh runs with set -o nounset and then unconditionally references ${OCP_IMAGE_N1..N4} in the N1–N4 blocks. The resolve step always truncates/creates ${SHARED_DIR}/nodepool_release_images, but it only appends OCP_IMAGE_N${n}=... when NODEPOOL_N${n}_VERSION is non-empty—so the sourced file can be partial and ${OCP_IMAGE_N*} may be unset, aborting the job.

🛡️ Suggested guard for each N-version block
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi

 N3_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N3} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N3:-}" && "${OCP_IMAGE_N3}" != "${OCP_IMAGE_LATEST}" ]]; then
   N3_NP_VERSION_TEST_ARGS="--e2e.n3-minor-release-image=${OCP_IMAGE_N3}"
 fi

 N4_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N4} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N4:-}" && "${OCP_IMAGE_N4}" != "${OCP_IMAGE_LATEST}" ]]; then
   N4_NP_VERSION_TEST_ARGS="--e2e.n4-minor-release-image=${OCP_IMAGE_N4}"
 fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh`
around lines 46 - 68, The script references potentially-undefined variables
OCP_IMAGE_N1 OCP_IMAGE_N2 OCP_IMAGE_N3 and OCP_IMAGE_N4 under nounset; update
each N*-block (where N1_NP_VERSION_TEST_ARGS, N2_NP_VERSION_TEST_ARGS,
N3_NP_VERSION_TEST_ARGS, N4_NP_VERSION_TEST_ARGS are set) to guard the
comparison against OCP_IMAGE_LATEST by first ensuring the OCP_IMAGE_N* variable
is defined (for example check variable existence with [[ -v OCP_IMAGE_N1 ]] or
use a safe default via parameter expansion) and only perform the !=
comparison/set the corresponding N*_NP_VERSION_TEST_ARGS when the variable is
present; apply the same change to all four blocks that source the
SHARED_DIR/nodepool_release_images.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh`:
- Around line 46-68: The script references potentially-undefined variables
OCP_IMAGE_N1 OCP_IMAGE_N2 OCP_IMAGE_N3 and OCP_IMAGE_N4 under nounset; update
each N*-block (where N1_NP_VERSION_TEST_ARGS, N2_NP_VERSION_TEST_ARGS,
N3_NP_VERSION_TEST_ARGS, N4_NP_VERSION_TEST_ARGS are set) to guard the
comparison against OCP_IMAGE_LATEST by first ensuring the OCP_IMAGE_N* variable
is defined (for example check variable existence with [[ -v OCP_IMAGE_N1 ]] or
use a safe default via parameter expansion) and only perform the !=
comparison/set the corresponding N*_NP_VERSION_TEST_ARGS when the variable is
present; apply the same change to all four blocks that source the
SHARED_DIR/nodepool_release_images.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh`:
- Around line 39-51: The comparisons reference OCP_IMAGE_N1 and OCP_IMAGE_N2
while the shared file is optional and nounset is enabled; update the conditions
to avoid unbound variable errors by using parameter expansion or explicit
non-empty checks — e.g. change the ifs to check that the variable is set and
non-empty (if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" !=
"${OCP_IMAGE_LATEST}" ]]; then ...) and similarly for OCP_IMAGE_N2, or use
${OCP_IMAGE_N1:-} and ${OCP_IMAGE_N2:-} in the comparisons to provide safe
defaults.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh`:
- Around line 70-92: The script dereferences OCP_IMAGE_N1..OCP_IMAGE_N4 without
guarding against unset values (breaking under set -o nounset); update the
conditional checks that set N1_NP_VERSION_TEST_ARGS, N2_NP_VERSION_TEST_ARGS,
N3_NP_VERSION_TEST_ARGS and N4_NP_VERSION_TEST_ARGS to use a non-empty guard or
safe parameter expansion (e.g. test [[ -n "${OCP_IMAGE_N1:-}" &&
"${OCP_IMAGE_N1:-}" != "${OCP_IMAGE_LATEST}" ]] before assigning) so the
optional source of nodepool_release_images can be absent without causing an
unbound variable error.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 5c337fad-01c6-4db3-ba84-f587b19d606b

📥 Commits

Reviewing files that changed from the base of the PR and between d326951 and 050ee96.

📒 Files selected for processing (15)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/hypershift-aws-e2e-external-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/oidc/hypershift-aws-e2e-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/metrics/hypershift-aws-e2e-metrics-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/nested/hypershift-aws-e2e-nested-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh
  • ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh
  • ci-operator/step-registry/hypershift/azure/aks/e2e/hypershift-azure-aks-e2e-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/aks/external-oidc/hypershift-azure-aks-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/e2e/self-managed/hypershift-azure-e2e-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh
  • ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.yaml
✅ Files skipped from review due to trivial changes (3)
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json
  • ci-operator/step-registry/hypershift/azure/aks/e2e/hypershift-azure-aks-e2e-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/hypershift-aws-e2e-external-workflow.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml

@jparrill
jparrill force-pushed the CNTRLPLANE-3509 branch 2 times, most recently from 6467554 to 81f79a8 Compare June 2, 2026 16:37
@jparrill

jparrill commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-hypershift-main-e2e-aws

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jparrill: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (4)
ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh (1)

50-68: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Guard OCP_IMAGE_N1OCP_IMAGE_N4 against unset under set -o nounset.

Same issue as in the AWS external script: the producer only writes OCP_IMAGE_N${n} on successful resolution, so any of N1–N4 may be unset, and the bare ${OCP_IMAGE_Nx} comparison aborts the step under set -o nounset (line 3) now that the job configs no longer supply OCP_IMAGE_NX overrides.

🛡️ Proposed fix (apply to N1–N4)
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi

 N3_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N3} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N3:-}" && "${OCP_IMAGE_N3}" != "${OCP_IMAGE_LATEST}" ]]; then
   N3_NP_VERSION_TEST_ARGS="--e2e.n3-minor-release-image=${OCP_IMAGE_N3}"
 fi

 N4_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N4} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N4:-}" && "${OCP_IMAGE_N4}" != "${OCP_IMAGE_LATEST}" ]]; then
   N4_NP_VERSION_TEST_ARGS="--e2e.n4-minor-release-image=${OCP_IMAGE_N4}"
 fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh`
around lines 50 - 68, The comparison against possibly unset
OCP_IMAGE_N1..OCP_IMAGE_N4 will fail under set -o nounset; update each
conditional that sets N1_NP_VERSION_TEST_ARGS, N2_NP_VERSION_TEST_ARGS,
N3_NP_VERSION_TEST_ARGS and N4_NP_VERSION_TEST_ARGS to guard the referenced
variables (e.g. use ${OCP_IMAGE_N1:-} != "${OCP_IMAGE_LATEST}" or default them
earlier with : "${OCP_IMAGE_N1:=}" ) so the shell never expands an unset name;
keep the same assignment to the N?_NP_VERSION_TEST_ARGS variables and only set
them when the guarded comparison evaluates true.
ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh (1)

74-92: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Guard OCP_IMAGE_N1OCP_IMAGE_N4 against unset under set -o nounset.

Same root cause as the other run-e2e scripts: OCP_IMAGE_N1OCP_IMAGE_N4 are now only populated by the sourced file when each version resolves, so any can be unset. The bare comparisons abort the step under set -o nounset (line 28) now that the OCP_IMAGE_NX job dependencies were removed.

🛡️ Proposed fix (apply to N1–N4)
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi

 N3_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N3} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N3:-}" && "${OCP_IMAGE_N3}" != "${OCP_IMAGE_LATEST}" ]]; then
   N3_NP_VERSION_TEST_ARGS="--e2e.n3-minor-release-image=${OCP_IMAGE_N3}"
 fi

 N4_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N4} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N4:-}" && "${OCP_IMAGE_N4}" != "${OCP_IMAGE_LATEST}" ]]; then
   N4_NP_VERSION_TEST_ARGS="--e2e.n4-minor-release-image=${OCP_IMAGE_N4}"
 fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh`
around lines 74 - 92, The comparisons against OCP_IMAGE_N1..OCP_IMAGE_N4 will
fail under set -o nounset if any of those variables are unset; update the logic
that sets N1_NP_VERSION_TEST_ARGS (and N2_NP_VERSION_TEST_ARGS, N3..., N4...) to
safely handle unset variables by using parameter expansion defaults or explicit
existence checks before comparing to OCP_IMAGE_LATEST (e.g., test existence with
a guard like checking -n "${OCP_IMAGE_N1-}" or use "${OCP_IMAGE_N1:-}" in the
comparison), and only set the corresponding N*_NP_VERSION_TEST_ARGS when the
variable is defined and not equal to OCP_IMAGE_LATEST.
ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh (1)

50-68: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Guard OCP_IMAGE_N1OCP_IMAGE_N4 against unset under set -o nounset.

These vars now come solely from the sourced ${SHARED_DIR}/nodepool_release_images. The producer (hypershift-resolve-nodepool-releases-commands.sh) only writes OCP_IMAGE_N${n} when a version resolves (it || true's failures and logs a WARNING), so any of N1–N4 can be unset. With set -o nounset (line 3), the bare ${OCP_IMAGE_N1} in the comparison aborts the step with "unbound variable". Since the job configs dropped the OCP_IMAGE_NX dependency overrides, this is a reachable regression.

🛡️ Proposed fix (apply to N1–N4)
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi

 N3_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N3} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N3:-}" && "${OCP_IMAGE_N3}" != "${OCP_IMAGE_LATEST}" ]]; then
   N3_NP_VERSION_TEST_ARGS="--e2e.n3-minor-release-image=${OCP_IMAGE_N3}"
 fi

 N4_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N4} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N4:-}" && "${OCP_IMAGE_N4}" != "${OCP_IMAGE_LATEST}" ]]; then
   N4_NP_VERSION_TEST_ARGS="--e2e.n4-minor-release-image=${OCP_IMAGE_N4}"
 fi

Using -n "${OCP_IMAGE_Nx:-}" also prevents emitting an empty --e2e.nX-minor-release-image= when a version isn't resolved.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh`
around lines 50 - 68, Guard the OCP_IMAGE_N1..N4 checks against unset by testing
for non-empty with parameter expansion and only then comparing to
OCP_IMAGE_LATEST; e.g. for each block that sets
N1_NP_VERSION_TEST_ARGS/N2_NP_VERSION_TEST_ARGS/N3_NP_VERSION_TEST_ARGS/N4_NP_VERSION_TEST_ARGS,
change the condition to require -n "${OCP_IMAGE_Nx:-}" AND that
"${OCP_IMAGE_Nx}" != "${OCP_IMAGE_LATEST}", and when building the flag use the
safe expansion ${OCP_IMAGE_Nx:-} so you never emit an empty
--e2e.nX-minor-release-image= value.
ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh (1)

43-51: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Guard OCP_IMAGE_N1/OCP_IMAGE_N2 against unset under set -o nounset.

The resolve step only writes OCP_IMAGE_N1/OCP_IMAGE_N2 when their versions resolve, so they can be unset. With set -o nounset (line 19) the bare ${OCP_IMAGE_N1}/${OCP_IMAGE_N2} comparison will abort the step.

🛡️ Proposed fix
 N1_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N1} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1}" != "${OCP_IMAGE_LATEST}" ]]; then
   N1_NP_VERSION_TEST_ARGS="--e2e.n1-minor-release-image=${OCP_IMAGE_N1}"
 fi

 N2_NP_VERSION_TEST_ARGS=""
-if [[ ${OCP_IMAGE_N2} != "${OCP_IMAGE_LATEST}" ]]; then
+if [[ -n "${OCP_IMAGE_N2:-}" && "${OCP_IMAGE_N2}" != "${OCP_IMAGE_LATEST}" ]]; then
   N2_NP_VERSION_TEST_ARGS="--e2e.n2-minor-release-image=${OCP_IMAGE_N2}"
 fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh`
around lines 43 - 51, The comparisons against OCP_IMAGE_N1 and OCP_IMAGE_N2
under set -o nounset can fail if those variables are unset; update the
conditional checks in the blocks that set N1_NP_VERSION_TEST_ARGS and
N2_NP_VERSION_TEST_ARGS to safely handle unset variables (e.g., use parameter
expansion defaults or explicit -n checks) so the [[ ... ]] tests never reference
bare unset variables; locate the two if blocks that set N1_NP_VERSION_TEST_ARGS
and N2_NP_VERSION_TEST_ARGS and change the conditions to guard against unset
OCP_IMAGE_N1/OCP_IMAGE_N2 (for example using ${OCP_IMAGE_N1:-} /
${OCP_IMAGE_N2:-} or [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1:-}" !=
"${OCP_IMAGE_LATEST}" ]]).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh`:
- Around line 50-68: Guard the OCP_IMAGE_N1..N4 checks against unset by testing
for non-empty with parameter expansion and only then comparing to
OCP_IMAGE_LATEST; e.g. for each block that sets
N1_NP_VERSION_TEST_ARGS/N2_NP_VERSION_TEST_ARGS/N3_NP_VERSION_TEST_ARGS/N4_NP_VERSION_TEST_ARGS,
change the condition to require -n "${OCP_IMAGE_Nx:-}" AND that
"${OCP_IMAGE_Nx}" != "${OCP_IMAGE_LATEST}", and when building the flag use the
safe expansion ${OCP_IMAGE_Nx:-} so you never emit an empty
--e2e.nX-minor-release-image= value.

In
`@ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh`:
- Around line 50-68: The comparison against possibly unset
OCP_IMAGE_N1..OCP_IMAGE_N4 will fail under set -o nounset; update each
conditional that sets N1_NP_VERSION_TEST_ARGS, N2_NP_VERSION_TEST_ARGS,
N3_NP_VERSION_TEST_ARGS and N4_NP_VERSION_TEST_ARGS to guard the referenced
variables (e.g. use ${OCP_IMAGE_N1:-} != "${OCP_IMAGE_LATEST}" or default them
earlier with : "${OCP_IMAGE_N1:=}" ) so the shell never expands an unset name;
keep the same assignment to the N?_NP_VERSION_TEST_ARGS variables and only set
them when the guarded comparison evaluates true.

In
`@ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh`:
- Around line 43-51: The comparisons against OCP_IMAGE_N1 and OCP_IMAGE_N2 under
set -o nounset can fail if those variables are unset; update the conditional
checks in the blocks that set N1_NP_VERSION_TEST_ARGS and
N2_NP_VERSION_TEST_ARGS to safely handle unset variables (e.g., use parameter
expansion defaults or explicit -n checks) so the [[ ... ]] tests never reference
bare unset variables; locate the two if blocks that set N1_NP_VERSION_TEST_ARGS
and N2_NP_VERSION_TEST_ARGS and change the conditions to guard against unset
OCP_IMAGE_N1/OCP_IMAGE_N2 (for example using ${OCP_IMAGE_N1:-} /
${OCP_IMAGE_N2:-} or [[ -n "${OCP_IMAGE_N1:-}" && "${OCP_IMAGE_N1:-}" !=
"${OCP_IMAGE_LATEST}" ]]).

In
`@ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh`:
- Around line 74-92: The comparisons against OCP_IMAGE_N1..OCP_IMAGE_N4 will
fail under set -o nounset if any of those variables are unset; update the logic
that sets N1_NP_VERSION_TEST_ARGS (and N2_NP_VERSION_TEST_ARGS, N3..., N4...) to
safely handle unset variables by using parameter expansion defaults or explicit
existence checks before comparing to OCP_IMAGE_LATEST (e.g., test existence with
a guard like checking -n "${OCP_IMAGE_N1-}" or use "${OCP_IMAGE_N1:-}" in the
comparison), and only set the corresponding N*_NP_VERSION_TEST_ARGS when the
variable is defined and not equal to OCP_IMAGE_LATEST.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 15180902-ea3c-4c2c-be4d-1701c6ccb421

📥 Commits

Reviewing files that changed from the base of the PR and between 050ee96 and 81f79a8.

📒 Files selected for processing (17)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/hypershift-aws-e2e-external-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/oidc/hypershift-aws-e2e-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/metrics/hypershift-aws-e2e-metrics-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/nested/hypershift-aws-e2e-nested-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh
  • ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh
  • ci-operator/step-registry/hypershift/azure/aks/e2e/hypershift-azure-aks-e2e-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/aks/external-oidc/hypershift-azure-aks-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/e2e/self-managed/hypershift-azure-e2e-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/e2e/v2-self-managed/hypershift-azure-e2e-v2-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh
  • ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/OWNERS
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.yaml
✅ Files skipped from review due to trivial changes (3)
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/OWNERS
  • ci-operator/step-registry/hypershift/aws/e2e/nested/hypershift-aws-e2e-nested-workflow.yaml
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json
🚧 Files skipped from review as they are similar to previous changes (9)
  • ci-operator/step-registry/hypershift/aws/e2e/metrics/hypershift-aws-e2e-metrics-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/oidc/hypershift-aws-e2e-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/aks/e2e/hypershift-azure-aks-e2e-workflow.yaml
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/hypershift-aws-e2e-external-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/aks/external-oidc/hypershift-azure-aks-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/e2e/self-managed/hypershift-azure-e2e-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh (1)

15-17: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add per-attempt timeouts to curl in resolve_release_image.

The curl command uses --retry and --retry-delay but lacks --connect-timeout and --max-time, so a stalled connection to the release API can block the step until the overall step timeout. Bound each attempt with short timeouts.

⏱️ Add connection and per-attempt timeouts
-    pullspec=$(curl -q -L -s --retry 5 --retry-delay 10 \
+    pullspec=$(curl -q -L -s --connect-timeout 10 --max-time 60 --retry 5 --retry-delay 10 \
         "https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/${version}.0-0.${stream}/latest" \
         | jq -r ".pullSpec // empty")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh`
around lines 15 - 17, The curl call that sets pullspec in resolve_release_image
lacks per-attempt timeouts; update the curl invocation (the one assigning
pullspec) to include --connect-timeout (e.g., 5) and --max-time (e.g., 15) so
each retry attempt is bounded, keeping existing --retry and --retry-delay flags
intact; modify only the curl arguments in the pullspec assignment to add these
options.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In
`@ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh`:
- Around line 15-17: The curl call that sets pullspec in resolve_release_image
lacks per-attempt timeouts; update the curl invocation (the one assigning
pullspec) to include --connect-timeout (e.g., 5) and --max-time (e.g., 15) so
each retry attempt is bounded, keeping existing --retry and --retry-delay flags
intact; modify only the curl arguments in the pullspec assignment to add these
options.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: c97aeca1-137a-4411-b29f-0e71311b178c

📥 Commits

Reviewing files that changed from the base of the PR and between 81f79a8 and 47c7e23.

📒 Files selected for processing (19)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/hypershift-aws-e2e-external-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/oidc/hypershift-aws-e2e-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/metrics/hypershift-aws-e2e-metrics-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/nested/hypershift-aws-e2e-nested-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/run-e2e/external/hypershift-aws-run-e2e-external-commands.sh
  • ci-operator/step-registry/hypershift/aws/run-e2e/nested/hypershift-aws-run-e2e-nested-commands.sh
  • ci-operator/step-registry/hypershift/azure/aks/e2e/hypershift-azure-aks-e2e-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/aks/external-oidc/hypershift-azure-aks-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/create-selfmanaged-guests/hypershift-azure-create-selfmanaged-guests-commands.sh
  • ci-operator/step-registry/hypershift/azure/create-selfmanaged-guests/hypershift-azure-create-selfmanaged-guests-ref.yaml
  • ci-operator/step-registry/hypershift/azure/e2e/self-managed/hypershift-azure-e2e-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/e2e/v2-self-managed/hypershift-azure-e2e-v2-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/run-e2e-self-managed/hypershift-azure-run-e2e-self-managed-commands.sh
  • ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/OWNERS
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-commands.sh
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.yaml
✅ Files skipped from review due to trivial changes (3)
  • ci-operator/step-registry/hypershift/azure/create-selfmanaged-guests/hypershift-azure-create-selfmanaged-guests-ref.yaml
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.metadata.json
  • ci-operator/step-registry/hypershift/azure/e2e/self-managed/hypershift-azure-e2e-self-managed-workflow.yaml
🚧 Files skipped from review as they are similar to previous changes (8)
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/OWNERS
  • ci-operator/step-registry/hypershift/azure/e2e/v2-self-managed/hypershift-azure-e2e-v2-self-managed-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/hypershift-aws-e2e-external-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/metrics/hypershift-aws-e2e-metrics-workflow.yaml
  • ci-operator/step-registry/hypershift/azure/aks/external-oidc/hypershift-azure-aks-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/e2e/external/oidc/hypershift-aws-e2e-external-oidc-workflow.yaml
  • ci-operator/step-registry/hypershift/resolve-nodepool-releases/hypershift-resolve-nodepool-releases-ref.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml

@jparrill

jparrill commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-hypershift-main-e2e-aws

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jparrill: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jparrill

jparrill commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jparrill: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-merge-bot Bot pushed a commit that referenced this pull request Jun 10, 2026
…ntroller API (#79627)" (#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
jparrill added a commit to jparrill/release that referenced this pull request Jun 10, 2026
… API

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
jparrill added a commit to jparrill/release that referenced this pull request Jun 11, 2026
… API

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
jparrill added a commit to jparrill/release that referenced this pull request Jun 11, 2026
… API

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
jparrill added a commit to jparrill/release that referenced this pull request Jun 12, 2026
… API

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
jparrill added a commit to jparrill/release that referenced this pull request Jun 12, 2026
… API

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
openshift-merge-bot Bot pushed a commit that referenced this pull request Jun 12, 2026
… API (#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by #79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
kasturinarra pushed a commit to kasturinarra/release that referenced this pull request Jun 15, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
krisnababu pushed a commit to krisnababu/release that referenced this pull request Jun 29, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
krisnababu pushed a commit to krisnababu/release that referenced this pull request Jun 29, 2026
…ntroller API (openshift#79627)" (openshift#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
krisnababu pushed a commit to krisnababu/release that referenced this pull request Jun 29, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
krisnababu pushed a commit to oharan2/release that referenced this pull request Jul 3, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
krisnababu pushed a commit to oharan2/release that referenced this pull request Jul 3, 2026
…ntroller API (openshift#79627)" (openshift#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
krisnababu pushed a commit to oharan2/release that referenced this pull request Jul 3, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
andrej1991 pushed a commit to andrej1991/release that referenced this pull request Jul 6, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
andrej1991 pushed a commit to andrej1991/release that referenced this pull request Jul 6, 2026
…ntroller API (openshift#79627)" (openshift#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
andrej1991 pushed a commit to andrej1991/release that referenced this pull request Jul 6, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Jul 8, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Jul 8, 2026
…ntroller API (openshift#79627)" (openshift#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Jul 8, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SachinNinganure pushed a commit to SachinNinganure/release that referenced this pull request Jul 20, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SachinNinganure pushed a commit to SachinNinganure/release that referenced this pull request Jul 20, 2026
…ntroller API (openshift#79627)" (openshift#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
SachinNinganure pushed a commit to SachinNinganure/release that referenced this pull request Jul 20, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
amogh-redhat pushed a commit to amogh-redhat/release that referenced this pull request Aug 5, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
malingatembo pushed a commit to malingatembo/release that referenced this pull request Aug 5, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Sep 5, 2026
… API (openshift#79627)

Replace ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. Each nXminor
entry in the releases: section spawned a concurrent import pod, causing
race conditions that failed ~40-45% of e2e-aws runs before tests start.

The release resolution logic lives in a single dedicated step ref
(hypershift-resolve-nodepool-releases) that queries the release
controller API and writes resolved pullspecs to SHARED_DIR. The 4
run-e2e scripts source this file instead of duplicating the logic.

Versions are passed as NODEPOOL_NX_VERSION env vars from the job
config. This follows the same pattern used by telco5g and
hypershift-mce multi-version-test teams.

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Sep 5, 2026
…ntroller API (openshift#79627)" (openshift#80349)

This reverts commit b7407a3.

Co-authored-by: Chai Bot <chai-bot@redhat.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Sep 5, 2026
… API (openshift#80360)

Replace static ci-operator release imports (n1minor through n4minor) with
runtime resolution via the release controller REST API. This eliminates
concurrent import pods that caused race conditions in ci-tools, failing
~40-45% of e2e runs before tests executed.

Changes:
- Remove n1minor..n4minor and unused latest-418/419/420 release definitions
- Replace OCP_IMAGE_NX dependencies with NODEPOOL_NX_VERSION env vars
- Add hypershift-resolve-nodepool-releases step to all 8 e2e workflows
- Add nounset-safe variable expansion (${OCP_IMAGE_NX:-}) in run-e2e scripts
  to prevent crashes when variables are unset
- Improve resolve script with retry logic, HTTP status checks, and hard
  failure on resolution errors

Fixes the permafail introduced by openshift#79627 where unguarded variable expansion
under set -o nounset crashed run-e2e scripts before tests could execute.

Jira: CNTRLPLANE-3509

Signed-off-by: Juan Manuel Parrilla Madrid <jparrill@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants