Skip to content

OSAC-1733: Build multiple components from one checkout via a components array - #283

Merged
eliorerz merged 1 commit into
osac-project:mainfrom
eliorerz:support-monorepo-multi-component-e2e
Jul 31, 2026
Merged

eliorerz merged 1 commit into
osac-project:mainfrom
eliorerz:support-monorepo-multi-component-e2e

Conversation

@eliorerz

@eliorerz eliorerz commented Jul 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Lets a single e2e run build any number of components from source, instead of just one. Replaces the previous single-component inputs (component-repo, component-ref, component-image-key, component-containerfile, component-build-type) with one components input: a JSON array string, one entry per component (repo, ref, containerfile, imageKey, buildType), e.g.:

[
  {"repo": "osac-project/osac", "ref": "my-branch", "containerfile": "fulfillment-service/Containerfile", "imageKey": "service.images.service"},
  {"repo": "osac-project/osac", "ref": "my-branch", "containerfile": "osac-operator/Containerfile", "imageKey": "operator.image.repository"}
]

Applied identically to e2e-bmaas-full-install.yml, e2e-vmaas-full-install.yml, and e2e-caas-full-install.yml:

  • A validation step checks the array shape and required fields per entry.
  • A loop builds/loads each component's image and records the result (image tag, key, repo, ref, containerfile, CRDs dir) after every component, so a failure partway through still leaves earlier successfully-built images tracked for teardown to clean up.
  • Loops (instead of single-component blocks) handle the installer-submodule replace and the image-overrides/CRD-apply logic at install time.

Backward compatible. Callers still using the old single-component inputs keep working unchanged — those inputs are still declared, and a synthesized one-entry components array is used when the new input is left empty. fulfillment-service, osac-operator, and osac-aap all still call these workflows with the old inputs today, so this avoids breaking their CI on merge.

Bugs fixed along the way

  • replace-installer-submodule.sh picked which osac-installer submodule to replace by the component's repo name, which breaks for a monorepo caller since every component shares the same repo name (e.g. osac-project/osac). It now uses the component's containerfile path to identify both which submodule to replace and which subdirectory of the checkout to copy — copying the whole checkout into one component's submodule was also wrong for monorepo callers. An unmapped or missing submodule is now a hard failure instead of a silent warning.
  • The "build the OSAC CLI from source" check was keyed off a component-repo substring match on "fulfillment-service", which also breaks for a monorepo caller. Now keyed off imageKey == "service.images.service", which identifies that component regardless of repo name.
  • teardown.sh now loops over every built component's image instead of a single COMPONENT_IMAGE var, validates COMPONENT_BUILD_RESULTS is a JSON array before looping (with a warning instead of a silent no-op on malformed input), and skips a literal "null" image value.
  • Pinned ansible-builder's version instead of an unpinned pip install, and added an explicit shell: bash to the image-build step.

Verification

  • pre-commit run --all-files passes (shellcheck skipped — needs Docker, unavailable in this dev environment; ran locally installed shellcheck directly on every touched script and on each workflow's extracted run: blocks instead, no issues beyond pre-existing/unrelated style notes).
  • YAML parses correctly for all three workflow files.
  • Functional scratch-directory tests for the submodule-copy scoping (confirms only the intended component subdirectory is copied, not the whole monorepo checkout), the fatal-on-unmapped-submodule path, and the legacy-input compatibility shim (old scalar inputs, new array input, and the no-op default case all resolve correctly).
  • Could not exercise this end-to-end myself (needs the self-hosted osac-ci/cluster-tool runner infra) — needs a real workflow_dispatch or the paired osac-project/osac caller PR to verify the multi-component build + merge + install path on real infra.

@openshift-ci-robot

openshift-ci-robot commented Jul 30, 2026 •

Copy link
Copy Markdown

@eliorerz: This pull request references OSAC-1733 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

  • Adds a second, optional component2-* input set (component2-repo, component2-ref, component2-image-key, component2-containerfile, component2-build-type) to the reusable e2e-bmaas-full-install.yml and e2e-vmaas-full-install.yml workflows, so a single e2e run can build and load two component images from source instead of one.
  • Purely additive: all new inputs default to ""/unused, so every existing caller (single-component) is unaffected.
  • Fixes a latent bug in the "build OSAC CLI from source" special case: it was keyed off component-repo containing the substring "fulfillment-service", which breaks once a caller's component-repo is a monorepo (e.g. osac-project/osac) that no longer has that name. Now keyed off component-image-key == "service.images.service", which already uniquely identifies the fulfillment-service component and is backward compatible with every existing caller.

Why

osac-project/osac now merges fulfillment-service and osac-operator into one repo. Their e2e workflows used to each call this reusable workflow once, testing "their own" component from source against the other component's stale pinned image from osac-installer — exactly the version-skew problem the mono-repo consolidation is meant to fix. The plan is for osac's new consolidated e2e workflow to build both components from current local code in one run via this second input set, rather than continuing to test one-at-a-time against stale defaults.

Scope note

Deliberately scoped to exactly 2 components (a literal second copy of the build/merge logic, not a loop or list input) rather than generalizing to N — workflow_call inputs are flat scalars with no native list type, and today's case is exactly 2. Full N-component generalization is deferred to OSAC-1734 (path-based CI matrix), once osac-aap and osac-installer also join the monorepo.

Test plan

  • YAML syntax validated locally (yaml.safe_load).
  • Could not exercise this end-to-end myself (requires the self-hosted osac-ci/cluster-tool runner infra) — needs a real workflow_dispatch or a real caller PR (the paired osac-project/osac PR) to fully verify the two-component build + merge + install path on real infra before relying on it.
  • Confirm no existing single-component caller (fulfillment-service, osac-operator, osac-aap's own e2e callers, if any still exist) regresses, since this only adds new optional inputs and does not change any existing code path.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 30, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: eliorerz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 30, 2026 •

Copy link
Copy Markdown

Review Change Stack

Walkthrough

BMaaS, CaaS, and VMaaS full-install workflows now accept JSON arrays of components, build and load each configured image, apply aggregated installer overrides and CRDs, and clean up all recorded component images.

Changes

Multi-component E2E installation

Layer / File(s) Summary
Component inputs and validation
.github/workflows/e2e-*-full-install.yml
Replaces fixed component inputs with validated JSON component definitions containing repository, ref, image key, and optional build settings.
Component builds and metadata
.github/workflows/e2e-*-full-install.yml
Builds, loads, and records multiple component images, optionally builds OSAC CLI, and captures component CRDs.
Installer replacement and deployment integration
.github/workflows/e2e-*-full-install.yml, .github/scripts/replace-installer-submodule.sh
Processes every built component for submodule replacement, image and AAP overrides, and server-side CRD application.
Aggregated image cleanup
.github/scripts/teardown.sh, .github/workflows/e2e-vmaas-full-install.yml
Removes all images listed in COMPONENT_BUILD_RESULTS during teardown and updates related workflow documentation.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Workflow
  participant ComponentRepositories
  participant ClusterNode
  participant OSACInstaller
  Workflow->>ComponentRepositories: Clone and build configured components
  ComponentRepositories-->>Workflow: Images, metadata, and optional CRDs
  Workflow->>ClusterNode: Load component images
  Workflow->>OSACInstaller: Pass aggregated build results
  OSACInstaller->>ClusterNode: Apply image overrides and CRDs
Loading

Possibly related PRs

Suggested labels: lgtm

Suggested reviewers: larsks, ygalblum, carbonin

🚥 Pre-merge checks | ✅ 11
✅ Passed checks (11 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
No-Hardcoded-Secrets ✅ Passed No hardcoded secrets were added: scans found no real credential formats, private keys, embedded-auth URLs, or credential-named literals; workflow secrets come from Vault/action outputs.
No-Weak-Crypto ✅ Passed Changed workflows/scripts add no MD5, SHA1, DES/3DES, RC4, Blowfish, ECB, custom crypto, or secret/token comparisons; Ed25519 key generation is not weak crypto.
No-Injection-Vectors ✅ Passed No flagged injection vector found: inputs are quoted or passed via jq --arg; jq filters use local loop indices, and exec is only FD locking. No eval, unsafe YAML/pickle, os.system, or shell=True.
Container-Privileges ✅ Passed No new privileged:true, hostPID/hostNetwork/hostIPC, SYS_ADMIN, allowPrivilegeEscalation, or securityContext settings; existing --network=host and chroot /host commands are unchanged.
No-Sensitive-Data-In-Logs ✅ Passed Added logs expose component metadata and image refs only; no new echo/printenv/secret-object dump logs passwords, tokens, API keys, or PII. Secret files/values are not printed.
Ai-Attribution ✅ Passed The PR has one unique commit with no AI-tool mention, Assisted-by/Generated-by requirement trigger, or Co-Authored-By trailer.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the primary change: enabling multiple component builds through a components array.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/e2e-vmaas-full-install.yml (1)

746-753: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Clean up all locally loaded component images in teardown.

.github/scripts/teardown.sh only prunes ${COMPONENT_IMAGE:-}, but this workflow can also load COMPONENT2_IMAGE from localhost/component2-pr:${GITHUB_RUN_ID} directly onto the runner. Add the same podman rmi branch for ${COMPONENT2_IMAGE:-} to prevent accumulated images on reused self-hosted runners.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e-vmaas-full-install.yml around lines 746 - 753, Update
teardown.sh to also remove the locally loaded COMPONENT2_IMAGE, using the same
conditional podman rmi branch already applied to COMPONENT_IMAGE; preserve the
existing cleanup behavior and reference the component2-pr image value supplied
by this workflow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/e2e-vmaas-full-install.yml:
- Around line 746-753: Update teardown.sh to also remove the locally loaded
COMPONENT2_IMAGE, using the same conditional podman rmi branch already applied
to COMPONENT_IMAGE; preserve the existing cleanup behavior and reference the
component2-pr image value supplied by this workflow.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: osac-project/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b67813d1-c4ea-4392-91d2-4a125c6c7ba1

📥 Commits

Reviewing files that changed from the base of the PR and between 5575466 and f4903cd.

📒 Files selected for processing (2)
  • .github/workflows/e2e-bmaas-full-install.yml
  • .github/workflows/e2e-vmaas-full-install.yml

@eliorerz
eliorerz force-pushed the support-monorepo-multi-component-e2e branch from b6ed717 to df6db4b Compare July 31, 2026 01:07
@eliorerz eliorerz changed the title OSAC-1733: Support two-component builds in e2e reusable workflows (for osac monorepo) OSAC-1733: Support building N components from one checkout via a components array Jul 31, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/scripts/replace-installer-submodule.sh (1)

69-69: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Monorepo component subdirectory is dropped everywhere it matters. The new monorepo contract puts a component under <dir>/ (fulfillment-service/Containerfile), but both the submodule copy and the CRD capture still assume the component is at the checkout root.

  • .github/scripts/replace-installer-submodule.sh#L69-L69: copy ${COMPONENT_SRC}/<containerfile dir>/. into the staging dir when the containerfile is nested, instead of the whole checkout.
  • .github/workflows/e2e-bmaas-full-install.yml#L328-L333: resolve config/crd/bases under ${src}/$(dirname "${file}") before deciding crds_dir.
  • .github/workflows/e2e-vmaas-full-install.yml#L346-L351: apply the same dirname-based resolution for the CRD capture.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/scripts/replace-installer-submodule.sh at line 69, Update
.github/scripts/replace-installer-submodule.sh lines 69-69 to copy the nested
containerfile directory from COMPONENT_SRC into STAGING instead of always
copying the checkout root. Update .github/workflows/e2e-bmaas-full-install.yml
lines 328-333 and .github/workflows/e2e-vmaas-full-install.yml lines 346-351 to
resolve config/crd/bases beneath the directory returned by dirname of the
containerfile path before selecting crds_dir.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/scripts/replace-installer-submodule.sh:
- Around line 33-37: Update the repository-name resolution around
COMPONENT_CONTAINERFILE to use the containerfile’s basename rather than dirname,
while preserving COMPONENT_REPO_NAME as the fallback for standalone or unmapped
paths. In the SUBMODULE_MAP lookup and warning path, make an unmapped component
fatal by exiting nonzero so stale submodule pins cannot proceed silently.

In @.github/scripts/teardown.sh:
- Around line 61-64: Update the component-image cleanup flow around
COMPONENT_BUILD_RESULTS to parse and validate the JSON before entering the loop,
failing teardown if jq is unavailable or parsing fails. Filter out missing,
null, and empty .image values so the loop passes only non-empty image strings to
podman rmi.

In @.github/workflows/e2e-bmaas-full-install.yml:
- Around line 335-336: Add shell: bash to the image-loading steps in
.github/workflows/e2e-bmaas-full-install.yml lines 335-336,
.github/workflows/e2e-vmaas-full-install.yml lines 353-354, and
.github/workflows/e2e-caas-full-install.yml lines 329-330, and begin each script
with set -euo pipefail so podman save failures propagate through the pipe.
- Around line 141-164: Replace the duplicated validation loops with a shared
composite action at .github/actions/validate-components. The action must
tolerate empty-string input consistently with the build step, validate each
component’s repo, ref, imageKey, containerfile, and buildType, and preserve the
existing JSON-array and org/repo checks. Update
.github/workflows/e2e-bmaas-full-install.yml lines 141-164,
.github/workflows/e2e-caas-full-install.yml lines 141-164, and
.github/workflows/e2e-vmaas-full-install.yml lines 141-164 to invoke the shared
action.
- Around line 141-164: Update the Validate components input step to treat an
empty COMPONENTS_JSON value consistently with the build-step guard by skipping
component validation/build processing when it is empty. For non-empty input,
validate each component’s buildType against the supported values and require
containerfile when that build type uses it, rejecting containerfile values
containing path traversal such as “../”.
- Around line 309-315: Replace the unpinned, system-level ansible-builder
installation in the build_type=="ansible-builder" branches with an exact,
identical ansible-builder version installed in an isolated venv or pipx
environment, and invoke that environment for ansible-builder create. Apply the
same change in .github/workflows/e2e-bmaas-full-install.yml lines 309-315,
.github/workflows/e2e-vmaas-full-install.yml lines 327-333, and
.github/workflows/e2e-caas-full-install.yml lines 313-319.

In @.github/workflows/e2e-caas-full-install.yml:
- Around line 332-335: Update the CaaS result construction around the jq
expression in the workflow so crdsDir is populated using the same CRD capture
flow as e2e-vmaas-full-install.yml, rather than hardcoded to an empty string.
Preserve the existing CRD-apply branch behavior so operator PR runs use the
captured CRDs.

---

Outside diff comments:
In @.github/scripts/replace-installer-submodule.sh:
- Line 69: Update .github/scripts/replace-installer-submodule.sh lines 69-69 to
copy the nested containerfile directory from COMPONENT_SRC into STAGING instead
of always copying the checkout root. Update
.github/workflows/e2e-bmaas-full-install.yml lines 328-333 and
.github/workflows/e2e-vmaas-full-install.yml lines 346-351 to resolve
config/crd/bases beneath the directory returned by dirname of the containerfile
path before selecting crds_dir.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: osac-project/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2d260e0a-8a8e-428b-9bbf-5a1211b6b4db

📥 Commits

Reviewing files that changed from the base of the PR and between f4903cd and df6db4b.

📒 Files selected for processing (5)
  • .github/scripts/replace-installer-submodule.sh
  • .github/scripts/teardown.sh
  • .github/workflows/e2e-bmaas-full-install.yml
  • .github/workflows/e2e-caas-full-install.yml
  • .github/workflows/e2e-vmaas-full-install.yml

Comment thread .github/scripts/replace-installer-submodule.sh
Comment thread .github/scripts/teardown.sh Outdated
Comment thread .github/workflows/e2e-bmaas-full-install.yml Outdated
Comment thread .github/workflows/e2e-bmaas-full-install.yml
Comment on lines +335 to +336
echo "Loading image onto node ${NODE_NAME}..."
podman save "${tag}" | oc debug "node/${NODE_NAME}" -t=false -- chroot /host podman load

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Image-load pipe hides podman save failures in all three workflows. These steps use the default bash -e {0} shell (no pipefail), so only oc debug's exit status is checked and a missing image surfaces much later as a rollout timeout.

  • .github/workflows/e2e-bmaas-full-install.yml#L335-L336: add shell: bash plus set -euo pipefail at the top of this step's script.
  • .github/workflows/e2e-vmaas-full-install.yml#L353-L354: add the same shell: bash / set -euo pipefail.
  • .github/workflows/e2e-caas-full-install.yml#L329-L330: add the same shell: bash / set -euo pipefail.
📍 Affects 3 files
  • .github/workflows/e2e-bmaas-full-install.yml#L335-L336 (this comment)
  • .github/workflows/e2e-vmaas-full-install.yml#L353-L354
  • .github/workflows/e2e-caas-full-install.yml#L329-L330
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e-bmaas-full-install.yml around lines 335 - 336, Add
shell: bash to the image-loading steps in
.github/workflows/e2e-bmaas-full-install.yml lines 335-336,
.github/workflows/e2e-vmaas-full-install.yml lines 353-354, and
.github/workflows/e2e-caas-full-install.yml lines 329-330, and begin each script
with set -euo pipefail so podman save failures propagate through the pipe.

Comment thread .github/workflows/e2e-caas-full-install.yml
…ts array

Lets a single e2e run build any number of components from source, instead
of just one. Replaces the previous single-component inputs with one
`components` input: a JSON array, one entry per component (repo, ref,
containerfile, imageKey, buildType). Applied identically to
e2e-bmaas-full-install.yml, e2e-vmaas-full-install.yml, and
e2e-caas-full-install.yml:

- A validation step checks the array shape and required fields per entry.
- A loop builds/loads each component's image and records the result
  (image tag, key, repo, ref, containerfile, CRDs dir) after every
  component, so a failure partway through still leaves earlier
  successfully-built images tracked for teardown to clean up.
- Loops (instead of single-component blocks) handle the installer-submodule
  replace and the image-overrides/CRD-apply logic at install time.

Backward compatible: callers still using the old single-component inputs
(component-repo, component-ref, component-image-key, component-containerfile,
component-build-type) keep working unchanged -- those inputs are still
declared, and a synthesized one-entry components array is used when the new
input is left empty. fulfillment-service, osac-operator, and osac-aap all
still call these workflows with the old inputs today, so this avoids
breaking their CI on merge.

Bugs fixed along the way:
- replace-installer-submodule.sh picked which osac-installer submodule to
  replace by the component's repo name, which breaks for a monorepo caller
  since every component shares the same repo name (e.g. osac-project/osac).
  It now uses the component's containerfile path to identify both which
  submodule to replace and which subdirectory of the checkout to copy --
  copying the whole checkout into one component's submodule was also wrong
  for monorepo callers. An unmapped or missing submodule is now a hard
  failure instead of a silent warning.
- The "build the OSAC CLI from source" check was keyed off a component-repo
  substring match on "fulfillment-service", which also breaks for a
  monorepo caller. Now keyed off imageKey == "service.images.service",
  which identifies that component regardless of repo name.
- teardown.sh now loops over every built component's image instead of a
  single COMPONENT_IMAGE var, validates COMPONENT_BUILD_RESULTS is a JSON
  array before looping (with a warning instead of a silent no-op on
  malformed input), and skips a literal "null" image value.
- Pinned ansible-builder's version instead of an unpinned pip install, and
  added an explicit shell: bash to the image-build step.
@eliorerz
eliorerz force-pushed the support-monorepo-multi-component-e2e branch from ad7a0f7 to ed4f252 Compare July 31, 2026 01:46
@eliorerz eliorerz changed the title OSAC-1733: Support building N components from one checkout via a components array OSAC-1733: Build multiple components from one checkout via a components array Jul 31, 2026
@eliorerz
eliorerz merged commit 807de68 into osac-project:main Jul 31, 2026
6 of 7 checks passed
@eliorerz
eliorerz deleted the support-monorepo-multi-component-e2e branch July 31, 2026 23:38

This branch was previously deployed

1 inactive deployment
e2e-test — ed4f252f Deployed Jul 31, 2026 by eliorerz via e2e-caas-full-install / e2e #124
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants