OCPBUGS-11939, OCPBUGS-18128, OCPBUGS-18460 , OCPBUGS-18602, OCPBUGS-18879: Support Disconnected HCP

[bryan-cox]

What this PR does / why we need it:
This PR contains the work needed to support HCP in a disconnected environment. The majority of the work in this PR is to determine image metadata in a disconnected environment, include user-ca bundles in appropriate HCP pods, and fixing the ignition server to support extracting image files in disconnected environments.

OCPBUGS-11939 - Get the image metadata from any ICSP/IDMS in a management cluster allowing a hosted cluster to be created in a disconnected environment with image mirror registry data from the ICSP/IDMS.

OCPBUGS-18128 - Automatically include the user created CA ConfigMap in the CPO deployment. This is accomplished in the HCC by combining the image registry CAs, listed in an Image (image.config.openshift.io) resource named 'clusters' in the 'openshift-config' namespace, to the CAs in the openshift-config-managed-trusted-ca-bundle ConfigMap from the
'hypershift' namespace and saving this to a new ConfigMap in the hostedcluster namespace. This new ConfigMap is then mounted as a volume in the CPO deployment.

OCPBUGS-18460 - Includes the user created CA ConfigMap in the ignition server deployment. This is accomplished by checking for the existence of a ConfigMap containing the needed CA data and mounting it to the ignition server deployment if the ConfigMap exists.

OCPBUGS-18602 - Includes the OpenShiftTrustedCABundle ConfigMap in the HCCO deployment.

OCPBUGS-18602 - Updates the HCCO release provider to include registry overrides and OpenShift image registry overrides.

OCPBUGS-18879 - Adds functionality to support extracting image files into release-manifests/image-reference config directory when working in a disconnected environment in the ignition server. Also, now catches the error when extracting image references to the config directory.

Which issue(s) this PR fixes:

• Fixes OCPBUGS-11939
• Fixes OCPBUGS-18128
• Fixes OCPBUGS-18460
• Fixes OCPBUGS-18602
• Fixes OCPBUGS-18879

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[bryan-cox]

Would we want to add imageContentSources here -

hypershift/hypershift-operator/main.go

Line 350 in 21bb0e1

ImageMetadataProvider: &hyperutil.RegistryClientImageMetadataProvider{},

?

[openshift-ci-robot]

@bryan-cox: This pull request references Jira Issue OCPBUGS-11939, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.14.0) matches configured target version for branch (4.14.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (lshilin@redhat.com), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@bryan-cox: This pull request references Jira Issue OCPBUGS-11939, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.14.0) matches configured target version for branch (4.14.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (lshilin@redhat.com), skipping review request.

Details

In response to this:

What this PR does / why we need it:

Which issue(s) this PR fixes:
Fixes OCPBUGS-11939

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@bryan-cox: This pull request references Jira Issue OCPBUGS-11939, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.14.0) matches configured target version for branch (4.14.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (lshilin@redhat.com), skipping review request.

Details

In response to this:

What this PR does / why we need it:
Get the image metadata from any ICSP/IDMS in a management cluster allowing a hosted cluster to be created in a disconnected environment with image mirror registry data from the ICSP/IDMS.

Which issue(s) this PR fixes:
Fixes OCPBUGS-11939

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@bryan-cox: This pull request references Jira Issue OCPBUGS-18128, which is invalid:

• expected the bug to target the "4.14.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What this PR does / why we need it:
Get the image metadata from any ICSP/IDMS in a management cluster allowing a hosted cluster to be created in a disconnected environment with image mirror registry data from the ICSP/IDMS.

Which issue(s) this PR fixes:
Fixes OCPBUGS-11939

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[enxebre]

lgtm I'll defer to @jparrill to tag

[bryan-cox]

/test verify

[jparrill]

/lgtm

[openshift-ci-robot]

@bryan-cox: This pull request references Jira Issue OCPBUGS-18879, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.15.0) matches configured target version for branch (4.15.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @elsapassaro

Details

In response to this:

What this PR does / why we need it:
This PR contains the work needed to support HCP in a disconnected environment. The majority of the work in this PR is to determine image metadata in a disconnected environment, include user-ca bundles in appropriate HCP pods, and fixing the ignition server to support extracting image files in disconnected environments.

OCPBUGS-11939 - Get the image metadata from any ICSP/IDMS in a management cluster allowing a hosted cluster to be created in a disconnected environment with image mirror registry data from the ICSP/IDMS.

OCPBUGS-18128 - Automatically include the user created CA ConfigMap in the CPO deployment. This is accomplished in the HCC by combining the image registry CAs, listed in an Image (image.config.openshift.io) resource named 'clusters' in the 'openshift-config' namespace, to the CAs in the openshift-config-managed-trusted-ca-bundle ConfigMap from the
'hypershift' namespace and saving this to a new ConfigMap in the hostedcluster namespace. This new ConfigMap is then mounted as a volume in the CPO deployment.

OCPBUGS-18460 - Includes the user created CA ConfigMap in the ignition server deployment. This is accomplished by checking for the existence of a ConfigMap containing the needed CA data and mounting it to the ignition server deployment if the ConfigMap exists.

OCPBUGS-18602 - Includes the OpenShiftTrustedCABundle ConfigMap in the HCCO deployment.

OCPBUGS-18602 - Updates the HCCO release provider to include registry overrides and OpenShift image registry overrides.

OCPBUGS-18879 - Adds functionality to support extracting image files into release-manifests/image-reference config directory when working in a disconnected environment in the ignition server. Also, now catches the error when extracting image references to the config directory.

Which issue(s) this PR fixes:

• Fixes OCPBUGS-11939
• Fixes OCPBUGS-18128
• Fixes OCPBUGS-18460
• Fixes OCPBUGS-18602
• Fixes OCPBUGS-18879

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bryan-cox]

/cherry-pick release-4.13

[openshift-cherrypick-robot]

@bryan-cox: once the present PR merges, I will cherry-pick it on top of release-4.13 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick release-4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bryan-cox]

/cherry-pick release-4.14

cherry-pick on 4.13 was a mistake. Will close that PR when it opens.

[openshift-cherrypick-robot]

@bryan-cox: once the present PR merges, I will cherry-pick it on top of release-4.14 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick release-4.14

cherry-pick on 4.13 was a mistake. Will close that PR when it opens.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[elsapassaro]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox, elsapassaro, enxebre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [enxebre]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@bryan-cox: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-ci-robot]

@bryan-cox: Jira Issue OCPBUGS-18879: All pull requests linked via external trackers have merged:

• openshift/hypershift#2950

Jira Issue OCPBUGS-18879 has been moved to the MODIFIED state.

Details

In response to this:

What this PR does / why we need it:
This PR contains the work needed to support HCP in a disconnected environment. The majority of the work in this PR is to determine image metadata in a disconnected environment, include user-ca bundles in appropriate HCP pods, and fixing the ignition server to support extracting image files in disconnected environments.

OCPBUGS-11939 - Get the image metadata from any ICSP/IDMS in a management cluster allowing a hosted cluster to be created in a disconnected environment with image mirror registry data from the ICSP/IDMS.

OCPBUGS-18128 - Automatically include the user created CA ConfigMap in the CPO deployment. This is accomplished in the HCC by combining the image registry CAs, listed in an Image (image.config.openshift.io) resource named 'clusters' in the 'openshift-config' namespace, to the CAs in the openshift-config-managed-trusted-ca-bundle ConfigMap from the
'hypershift' namespace and saving this to a new ConfigMap in the hostedcluster namespace. This new ConfigMap is then mounted as a volume in the CPO deployment.

OCPBUGS-18460 - Includes the user created CA ConfigMap in the ignition server deployment. This is accomplished by checking for the existence of a ConfigMap containing the needed CA data and mounting it to the ignition server deployment if the ConfigMap exists.

OCPBUGS-18602 - Includes the OpenShiftTrustedCABundle ConfigMap in the HCCO deployment.

OCPBUGS-18602 - Updates the HCCO release provider to include registry overrides and OpenShift image registry overrides.

OCPBUGS-18879 - Adds functionality to support extracting image files into release-manifests/image-reference config directory when working in a disconnected environment in the ignition server. Also, now catches the error when extracting image references to the config directory.

Which issue(s) this PR fixes:

• Fixes OCPBUGS-11939
• Fixes OCPBUGS-18128
• Fixes OCPBUGS-18460
• Fixes OCPBUGS-18602
• Fixes OCPBUGS-18879

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-cherrypick-robot]

@bryan-cox: #2950 failed to apply on top of branch "release-4.13":

Applying: OCPBUGS-11939: Use img metadata in from ICSP/IDMS
Using index info to reconstruct a base tree...
M	hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go
M	hypershift-operator/controllers/nodepool/haproxy.go
M	hypershift-operator/controllers/nodepool/nodepool_controller.go
M	hypershift-operator/main.go
M	support/util/fakeimagemetadataprovider/fakeimagemetadataprovider.go
M	support/util/imagemetadata.go
Falling back to patching base and 3-way merge...
Auto-merging support/util/imagemetadata.go
CONFLICT (content): Merge conflict in support/util/imagemetadata.go
Auto-merging hypershift-operator/main.go
CONFLICT (content): Merge conflict in hypershift-operator/main.go
Auto-merging hypershift-operator/controllers/nodepool/nodepool_controller.go
Auto-merging hypershift-operator/controllers/nodepool/haproxy.go
Auto-merging hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 OCPBUGS-11939: Use img metadata in from ICSP/IDMS
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Details

In response to this:

/cherry-pick release-4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-cherrypick-robot]

@bryan-cox: new pull request created: #3064

Details

In response to this:

/cherry-pick release-4.14

cherry-pick on 4.13 was a mistake. Will close that PR when it opens.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-merge-robot]

Fix included in accepted release 4.15.0-0.nightly-2023-09-30-003019