Skip to content

osac-operator: copy Helm charts into installer overlay for presubmit CI - #80893

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
omer-vishlitzky:fix/osac-operator-copy-charts-to-installer
Jun 23, 2026
Merged

openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
omer-vishlitzky:fix/osac-operator-copy-charts-to-installer

Conversation

@omer-vishlitzky

@omer-vishlitzky omer-vishlitzky commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Copy charts/ from osac-operator PR branches into the osac-installer-with-pr image so that CRD and Helm chart changes are tested in presubmit CI

Problem

The osac-installer-with-pr image only copies config/ from the PR branch into the installer submodule path. The umbrella Helm chart reads CRDs from charts/operator-crds/ and deployment templates from charts/operator/ — not from config/crd/bases/. This means PRs that change CRD schemas (e.g., renaming status.jobsstatus.provisioningJobs) deploy the new operator binary against the old CRDs from the snapshot's submodule pin.

The result: the API server silently strips unknown status fields, the operator can't persist job tracking state, and it enters an infinite reconcile loop launching hundreds of duplicate AAP jobs. All VirtualNetwork/Subnet/PublicIP tests fail with timeout.

Evidence from osac-operator PR #299:

  • Operator log: unknown field "status.provisioningJobs" repeated 478 times
  • 122 duplicate provision job triggers in 29 minutes
  • 380+ total AAP jobs launched (vs ~30 expected)
  • 4 test failures + 14 test errors, all from VirtualNetwork stuck in Progressing

Fix

Add one line to the Dockerfile literal:

COPY charts/ /installer/base/osac-operator/charts/

This ensures helm upgrade during CI refresh picks up CRDs and deployment templates from the PR branch, not the stale submodule pin.

Test plan

  • Verified the Helm umbrella chart references file://../../base/osac-operator/charts/operator-crds and file://../../base/osac-operator/charts/operator
  • Verified charts/operator-crds/ is kept in sync with config/crd/bases/ by make helm-crds, enforced by CI (make check-helm-crds)
  • Confirmed no other OSAC repos need this fix (fulfillment-service and osac-aap have no CRDs; bare-metal-fulfillment-operator has no independent Prow presubmit)

🤖 Generated with Claude Code

Summary by CodeRabbit

This PR modifies the OpenShift CI build configuration for the osac-operator repository to fix test failures caused by schema version mismatches during PR validation.

Context: The CI infrastructure generates an osac-installer-with-pr image that overlays PR changes into the installer base image. When testing osac-operator PRs, the build was copying only the config/ directory from the PR branch, but not the charts/ directory containing Helm chart resources. This created a critical issue: while the PR's new operator binary would be deployed, it would be run against stale CRD definitions from the base snapshot, causing the Kubernetes API server to strip unknown fields from status updates. This led to infinite reconciliation loops and cascading test failures.

The fix: A single line was added to the Dockerfile configuration in ci-operator/config/osac-project/osac-operator/osac-project-osac-operator-main.yaml to copy the charts directory into the installer overlay:

COPY charts/ /installer/base/osac-operator/charts/

Impact: This ensures that when the umbrella Helm chart runs helm upgrade operations during CI refresh, it picks up both CRD schemas and deployment templates from the PR branch, allowing the operator to be tested against the correct schema version. The charts directory is kept in sync with config/crd/bases/ through an automated make helm-crds process enforced by CI checks.

Why it matters: Without this change, PRs that modify CRD schemas (such as renaming fields) cannot be properly validated in CI, leading to silent failures where the API server silently drops unknown fields, causing the operator to lose state and retry infinitely. This is particularly problematic for infrastructure-related resources like VirtualNetwork, Subnet, and PublicIP that are critical to cluster functionality.

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@openshift-ci
openshift-ci Bot requested review from jhernand and rgolangh June 23, 2026 08:50
@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 774024f2-6a97-44c7-9aa5-0e13cf98e433

📥 Commits

Reviewing files that changed from the base of the PR and between 81b7b36 and f83b1c9.

📒 Files selected for processing (1)
  • ci-operator/config/osac-project/osac-operator/osac-project-osac-operator-main.yaml

Walkthrough

A single COPY charts/ instruction was added to the embedded dockerfile_literal in the OSAC operator CI build configuration, copying the charts/ directory into /installer/base/osac-operator/charts/ within the installer image.

Changes

OSAC Operator Installer Image Update

Layer / File(s) Summary
Add charts/ COPY step to installer image build
ci-operator/config/osac-project/osac-operator/osac-project-osac-operator-main.yaml
Adds COPY charts/ to the embedded Dockerfile so charts/ is placed under /installer/base/osac-operator/charts/ in the built installer image.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Suggested labels

lgtm, approved

Suggested reviewers

  • adriengentil
  • eranco74
  • danmanor
🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: adding a COPY instruction for Helm charts into the installer overlay for presubmit CI testing.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR contains only CI/CD configuration changes (adding Helm chart copy to YAML), no Ginkgo test definitions or modifications to test files. Check not applicable.
Test Structure And Quality ✅ Passed PR contains only CI configuration changes (YAML file), not Ginkgo test code, so test structure review is not applicable.
Microshift Test Compatibility ✅ Passed This PR modifies CI build configuration only, not adding any new Ginkgo e2e tests. The check for MicroShift test compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests were added in this PR. The PR only modifies build configuration YAML to copy Helm charts into the installer overlay, making the SNO compatibility check not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed This PR only modifies a CI operator configuration file (ci-operator/config/) that defines how to build container images. It adds a single COPY instruction to a Dockerfile. No deployment manifests,...
Ote Binary Stdout Contract ✅ Passed PR modifies only a CI operator YAML config file adding a Dockerfile COPY instruction; no binary, test, or process-level code changes. OTE Binary Stdout Contract check is not applicable.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests added; PR only modifies CI configuration YAML to copy charts directory in installer image build.
No-Weak-Crypto ✅ Passed PR modifies only a single YAML configuration file to add a Docker COPY instruction for Helm charts. No weak cryptographic algorithms, custom crypto implementations, or non-constant-time comparisons...
Container-Privileges ✅ Passed The PR adds only a COPY instruction to a Dockerfile, copying charts directory files. No container privileges, hostPID/Network/IPC, SYS_ADMIN capabilities, or allowPrivilegeEscalation settings are i...
No-Sensitive-Data-In-Logs ✅ Passed No logging statements or sensitive data exposure in the PR. The change adds a simple COPY instruction to the Dockerfile with no logging, credentials, or debug output.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@omer-vishlitzky: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-osac-project-osac-operator-main-e2e-vmaas osac-project/osac-operator presubmit Ci-operator config changed
pull-ci-osac-project-osac-operator-main-images osac-project/osac-operator presubmit Ci-operator config changed

Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

@omer-vishlitzky: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@rgolangh

Copy link
Copy Markdown
Contributor

FYI @zszabo-rh

@rgolangh

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@omer-vishlitzky

Copy link
Copy Markdown
Contributor Author

/pj-rehearse ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 23, 2026
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 23, 2026
@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: omer-vishlitzky, rgolangh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 23, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 95f4479 into openshift:main Jun 23, 2026
14 checks passed
kasturinarra pushed a commit to kasturinarra/release that referenced this pull request Jun 26, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
krisnababu pushed a commit to krisnababu/release that referenced this pull request Jun 29, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
krisnababu pushed a commit to oharan2/release that referenced this pull request Jul 3, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
andrej1991 pushed a commit to andrej1991/release that referenced this pull request Jul 6, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Jul 8, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SachinNinganure pushed a commit to SachinNinganure/release that referenced this pull request Jul 20, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
amogh-redhat pushed a commit to amogh-redhat/release that referenced this pull request Aug 5, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TimurMP pushed a commit to TimurMP/release that referenced this pull request Sep 5, 2026
…CI (openshift#80893)

The osac-installer-with-pr image only copies config/ from the PR branch,
but the umbrella Helm chart reads CRDs from charts/operator-crds/ and
deployment templates from charts/operator/. Without this overlay, PRs
that change CRD schemas (e.g. field renames) deploy the new operator
binary against old CRDs, causing silent status update failures.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants