Skip to content

OCPBUGS-85351: Revert "fix(azure): prevent externalDNSDomain from shadowing cluster apps domain" - #8585

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
Nirshal:revert-pr-8480
May 26, 2026
Merged

OCPBUGS-85351: Revert "fix(azure): prevent externalDNSDomain from shadowing cluster apps domain"#8585
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
Nirshal:revert-pr-8480

Conversation

@Nirshal

@Nirshal Nirshal commented May 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The fix was originally introduced in OCPBUGS-85351: fix(azure): prevent externalDNSDomain from shadowing cluster apps domain #8480
  • It is being reverted in this PR because the CEL validation rejects legitimate ARO-HCP HostedCluster configurations. ARO-HCP uses service hostnames (e.g., api-int.{name}.domain) that share a parent domain with the cluster baseDomain (aro.{name}.domain), which the validation incorrectly flags as DNS zone shadowing.
  • This revert unblocks ARO-HCP E2E tests that are failing after bumping to the latest HyperShift operator. The shadowing prevention will need to be re-implemented with ARO-HCP's DNS configuration pattern taken into account.

References

/cc @bryan-cox @enxebre

…al-dns-validation"

This reverts commit 1d34880, reversing
changes made to c23dbb8.
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@coderabbitai

coderabbitai Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR removes DNS domain shadowing validation that previously prevented Azure private hosted clusters from using external DNS domains that overlapped with the cluster's base domain. The change removes validation rules across four layers: the API schema definition removes a kubebuilder constraint, the CLI validator removes external DNS domain conflict checking, the control-plane-operator removes shadowing detection logic and simplifies base-domain DNS reconciliation, and documentation removes corresponding warnings and remediation guidance.

Possibly Related PRs

  • openshift/hypershift#8480: Directly related as it removes external-dns-domain/base-domain shadowing validation and controller logic that was previously added.

Suggested Reviewers

  • enxebre
🚥 Pre-merge checks | ✅ 8
✅ Passed checks (8 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR modifies two test files using standard Go testing (testing.T), not Ginkgo. Zero Ginkgo It/Describe blocks found; check not applicable.
Test Structure And Quality ✅ Passed The test files use standard Go tests with Gomega, not Ginkgo syntax (no Describe/It blocks), making the Ginkgo-specific check inapplicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR reverts DNS validation logic and removes related controller code; it does not introduce any pod scheduling constraints, deployment manifests, or topology assumptions.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No Ginkgo e2e tests are added in this revert PR. Only 1694 standard Go unit tests (func Test*) are added, not Ginkgo tests (It(), Describe()). Check is not applicable.
Title check ✅ Passed The title directly references the OCPBUGS ticket and clearly indicates this is a revert of a previous DNS validation fix that was causing legitimate Azure cluster configurations to be rejected.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@Nirshal Nirshal changed the title Revert "Merge pull request #8480 from bryan-cox/azure-external-dns-validation" NO-JIRA: Revert "Merge pull request #8480 from bryan-cox/azure-external-dns-validation" May 26, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 26, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@Nirshal: This pull request explicitly references no jira issue.

Details

In response to this:

This reverts merge commit 1d34880, reverting PR #8480.

Reason

The CEL validation introduced in #8480 rejects legitimate ARO-HCP HostedCluster configurations. ARO-HCP uses service hostnames (e.g., api-int.{name}.domain) that share a parent domain with the cluster baseDomain (aro.{name}.domain). The CEL rule incorrectly flags this as DNS zone shadowing.

Reported by the ARO-HCP team — their E2E tests are failing after bumping to the latest HyperShift operator.

References

/cc @bryan-cox

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added area/api Indicates the PR includes changes for the API area/cli Indicates the PR includes changes for CLI area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/documentation Indicates the PR includes changes for documentation area/platform/azure PR/issue for Azure (AzurePlatform) platform and removed do-not-merge/needs-area labels May 26, 2026
@bryan-cox

Copy link
Copy Markdown
Member

/approve
/lgtm

This regressed ARO HCP.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 26, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@bryan-cox

Copy link
Copy Markdown
Member

/verified bypass

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label May 26, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: The verified label has been added.

Details

In response to this:

/verified bypass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
cmd/cluster/azure/create_test.go (1)

28-83: ⚡ Quick win

Add a regression test for overlapping external/base domain acceptance.

This revert addresses a real behavior break; please add one focused test that uses an overlapping parent-domain ExternalDNSDomain and asserts Validate does not fail, so the regression does not return later.

As per coding guidelines, "Always include unit tests when creating new functions or modifying existing ones".

Also applies to: 338-387

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cmd/cluster/azure/create_test.go` around lines 28 - 83, Add a regression unit
test to cmd/cluster/azure/create_test.go that constructs DefaultOptions(), sets
CredentialsFile and an overlapping ExternalDNSDomain (a parent domain of
BaseDomain) and calls opts.Validate(context.Background(),
&core.CreateOptions{}), asserting that Validate returns no error; follow the
existing TestValidateEndpointAccess pattern (use t.Run, set
opts.ExternalDNSDomain and opts.BaseDomain accordingly) so the test ensures
overlapping parent-domain ExternalDNSDomain is accepted and the regression
cannot reappear.
control-plane-operator/controllers/azureprivatelinkservice/controller_test.go (1)

185-207: ⚡ Quick win

Preserve DNS zone visibility in mockRecordSets assertions.

Line 199 currently ignores privateDnsZoneName, which makes it harder for tests to catch records being created in the wrong zone. Keep zone capture in the mock and assert zone+record combinations in base-domain/hypershift-local tests.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/controllers/azureprivatelinkservice/controller_test.go`
around lines 185 - 207, The mockRecordSets.CreateOrUpdate currently ignores the
DNS zone argument (the second string parameter) so tests can't assert
zone+record combos; update the mockRecordSets struct to capture the zone (e.g.,
add a createdRecordZones or store zone-prefixed names) and modify CreateOrUpdate
to record the incoming privateDnsZoneName along with relativeRecordSetName
(populate a new field like createdRecordZones or append
zone+"|"+relativeRecordSetName to createdRecordNames), and update any
base-domain/hypershift-local tests to assert the zone+record combination using
the new field(s) instead of only relativeRecordSetName; reference:
mockRecordSets struct and its CreateOrUpdate method, plus
createdRecordNames/createdRecordZones fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@cmd/cluster/azure/create_test.go`:
- Around line 28-83: Add a regression unit test to
cmd/cluster/azure/create_test.go that constructs DefaultOptions(), sets
CredentialsFile and an overlapping ExternalDNSDomain (a parent domain of
BaseDomain) and calls opts.Validate(context.Background(),
&core.CreateOptions{}), asserting that Validate returns no error; follow the
existing TestValidateEndpointAccess pattern (use t.Run, set
opts.ExternalDNSDomain and opts.BaseDomain accordingly) so the test ensures
overlapping parent-domain ExternalDNSDomain is accepted and the regression
cannot reappear.

In
`@control-plane-operator/controllers/azureprivatelinkservice/controller_test.go`:
- Around line 185-207: The mockRecordSets.CreateOrUpdate currently ignores the
DNS zone argument (the second string parameter) so tests can't assert
zone+record combos; update the mockRecordSets struct to capture the zone (e.g.,
add a createdRecordZones or store zone-prefixed names) and modify CreateOrUpdate
to record the incoming privateDnsZoneName along with relativeRecordSetName
(populate a new field like createdRecordZones or append
zone+"|"+relativeRecordSetName to createdRecordNames), and update any
base-domain/hypershift-local tests to assert the zone+record combination using
the new field(s) instead of only relativeRecordSetName; reference:
mockRecordSets struct and its CreateOrUpdate method, plus
createdRecordNames/createdRecordZones fields.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 1f19ac3e-628d-4a17-a15d-0096c2725d5a

📥 Commits

Reviewing files that changed from the base of the PR and between fa30196 and eb45c67.

⛔ Files ignored due to path filters (19)
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/AAA_ungated.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/ClusterUpdateAcceptRisks.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/ClusterVersionOperatorConfiguration.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/ExternalOIDC.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/ExternalOIDCWithUIDAndExtraClaimMappings.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/ExternalOIDCWithUpstreamParity.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/GCPPlatform.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/HCPEtcdBackup.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/HyperShiftOnlyDynamicResourceAllocation.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/ImageStreamImportMode.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/KMSEncryptionProvider.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/OpenStack.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • api/hypershift/v1beta1/zz_generated.featuregated-crd-manifests/hostedclusters.hypershift.openshift.io/TLSAdherence.yaml is excluded by !**/zz_generated.featuregated-crd-manifests/**
  • cmd/install/assets/crds/hypershift-operator/tests/hostedclusters.hypershift.openshift.io/stable.hostedclusters.azure.testsuite.yaml is excluded by !cmd/install/assets/**/*.yaml
  • cmd/install/assets/crds/hypershift-operator/zz_generated.crd-manifests/hostedclusters-Hypershift-CustomNoUpgrade.crd.yaml is excluded by !**/zz_generated.crd-manifests/**, !cmd/install/assets/**/*.yaml
  • cmd/install/assets/crds/hypershift-operator/zz_generated.crd-manifests/hostedclusters-Hypershift-Default.crd.yaml is excluded by !**/zz_generated.crd-manifests/**, !cmd/install/assets/**/*.yaml
  • cmd/install/assets/crds/hypershift-operator/zz_generated.crd-manifests/hostedclusters-Hypershift-TechPreviewNoUpgrade.crd.yaml is excluded by !**/zz_generated.crd-manifests/**, !cmd/install/assets/**/*.yaml
  • docs/content/reference/aggregated-docs.md is excluded by !docs/content/reference/aggregated-docs.md
  • vendor/github.com/openshift/hypershift/api/hypershift/v1beta1/hostedcluster_types.go is excluded by !vendor/**, !**/vendor/**
📒 Files selected for processing (7)
  • api/hypershift/v1beta1/hostedcluster_types.go
  • cmd/cluster/azure/create.go
  • cmd/cluster/azure/create_test.go
  • control-plane-operator/controllers/azureprivatelinkservice/controller.go
  • control-plane-operator/controllers/azureprivatelinkservice/controller_test.go
  • docs/content/how-to/azure/create-self-managed-azure-cluster.md
  • docs/content/how-to/azure/deploy-azure-private-clusters.md
💤 Files with no reviewable changes (2)
  • docs/content/how-to/azure/create-self-managed-azure-cluster.md
  • api/hypershift/v1beta1/hostedcluster_types.go

@Nirshal Nirshal changed the title NO-JIRA: Revert "Merge pull request #8480 from bryan-cox/azure-external-dns-validation" OCPBUGS-85351: Revert "fix(azure): prevent externalDNSDomain from shadowing cluster apps domain" May 26, 2026
@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label May 26, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@Nirshal: This pull request references Jira Issue OCPBUGS-85351, which is invalid:

  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

  • The fix was originally introduced in OCPBUGS-85351: fix(azure): prevent externalDNSDomain from shadowing cluster apps domain #8480
  • It is being reverted in this PR because the CEL validation rejects legitimate ARO-HCP HostedCluster configurations. ARO-HCP uses service hostnames (e.g., api-int.{name}.domain) that share a parent domain with the cluster baseDomain (aro.{name}.domain), which the validation incorrectly flags as DNS zone shadowing.
  • This revert unblocks ARO-HCP E2E tests that are failing after bumping to the latest HyperShift operator. The shadowing prevention will need to be re-implemented with ARO-HCP's DNS configuration pattern taken into account.

References

/cc @bryan-cox @alberto-lamela

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@Nirshal

Nirshal commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

/cc @enxebre

@openshift-ci
openshift-ci Bot requested a review from enxebre May 26, 2026 11:20
@cwbotbot

cwbotbot commented May 26, 2026

Copy link
Copy Markdown

Test Results

e2e-aws

e2e-aks

@enxebre

enxebre commented May 26, 2026

Copy link
Copy Markdown
Member

can we please follow up to:

  • Replicate the same dns pattern in the CI azure e2e
  • Include that case in the envtest suite

/approve

@openshift-ci

openshift-ci Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox, enxebre, Nirshal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 26, 2026
@bryan-cox

Copy link
Copy Markdown
Member

/test e2e-aks

@bryan-cox

Copy link
Copy Markdown
Member

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels May 26, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: This pull request references Jira Issue OCPBUGS-85351, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD fa30196 and 2 for PR HEAD eb45c67 in total

@bryan-cox

Copy link
Copy Markdown
Member

/test e2e-aws

@bryan-cox

Copy link
Copy Markdown
Member

/test e2e-aws

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

Now I have all the evidence I need. Let me produce the final report.

Test Failure Analysis Complete

Job Information

Test Failure Analysis

Error

1. TestCreateCluster/Main/EnsureGlobalPullSecret/When_management-cluster_hostedCluster.Spec.PullSecret_is_updated_in-place_it_should_propagate_to_guest_without_rollout:
   failed to wait for DaemonSet global-pull-secret-syncer to be ready: context deadline exceeded (2/3 pods ready for ~20 minutes)

2. TestCreateCluster/Main/EnsureGlobalPullSecret/Check_if_the_config.json_is_correct_in_all_of_the_nodes:
   daemonsets.apps "kubelet-config-verifier" already exists (cascade from failure 1)

3. TestKarpenter/Teardown:
   Failed to wait for infra resources in guest cluster to be deleted: context deadline exceeded (10 AWS resources not cleaned up)

Summary

All 7 test failures stem from two independent infrastructure-level issues completely unrelated to the PR changes. PR #8585 only modifies Azure-specific CRD validation, Azure CLI code, Azure private link service controller, and documentation — none of which affect AWS e2e test execution. The first root failure is TestCreateCluster/Main/EnsureGlobalPullSecret where the global-pull-secret-syncer DaemonSet remained stuck at 2/3 pods ready for the entire 20-minute timeout (while ovnkube-node was healthy at 3/3), suggesting one guest cluster node had a scheduling or node-readiness issue preventing the DaemonSet pod from running. The second root failure is TestKarpenter/Teardown where 10 AWS resources (9 EBS volumes across various Karpenter nodepools + 1 NLB) could not be cleaned up within the teardown deadline. All Karpenter functional tests (Main, ValidateHostedCluster, EnsureHostedCluster) passed — only the post-test resource cleanup timed out. Additionally, at teardown the TestCreateCluster HostedCluster showed ClusterVersionSucceeding=False due to the monitoring cluster operator being unavailable, which is consistent with the DaemonSet scheduling issue on the same cluster.

Root Cause

Failure 1 — TestCreateCluster (EnsureGlobalPullSecret):

The global-pull-secret-syncer DaemonSet in the guest cluster was stuck at 2/3 pods ready for the entire ~20-minute polling period (364 retry log lines). The test creates this DaemonSet to verify that pull secret changes propagate to all guest cluster nodes' kubelet configurations. The ovnkube-node DaemonSet on the same cluster was healthy (3/3), confirming all 3 nodes existed, but one node likely had a condition preventing the global-pull-secret-syncer pod from scheduling or becoming ready. This caused a cascade:

  • The kubelet-config-verifier DaemonSet (from the next subtest) failed with "already exists" because the previous subtest's DaemonSet was not cleaned up.
  • The parent EnsureGlobalPullSecret, Main, and TestCreateCluster tests all failed as a result.
  • At teardown, the HostedCluster condition showed ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator monitoring is not available), indicating the monitoring operator on that cluster was also experiencing issues — consistent with a node-level problem.

Failure 2 — TestKarpenter (Teardown):

All Karpenter functional subtests passed successfully. The failure occurred exclusively during teardown when the test framework attempted to clean up AWS infrastructure resources. 10 resources timed out during deletion: 9 EBS volumes (associated with various Karpenter nodepools: capacity-reservation-test, arbitrary-subnet-test, kubelet-config-test, on-demand, instance-profile-test, version-test, plus 2 CAPI node volumes) and 1 NLB (for openshift-ingress/router-default). This is a known pattern where Karpenter-provisioned resources can become orphaned when the cluster is destroyed before Kubernetes finalizers complete resource cleanup.

Relationship to PR #8585: None. The PR changes are entirely scoped to Azure platform: Azure CEL validation in the HostedCluster CRD, Azure CLI create command, Azure private link service controller, and Azure documentation. None of the failing tests (TestCreateCluster/EnsureGlobalPullSecret on AWS, TestKarpenter/Teardown on AWS) exercise any Azure code paths.

Recommendations
  1. These failures are safe to ignore for PR OCPBUGS-85351: Revert "fix(azure): prevent externalDNSDomain from shadowing cluster apps domain" #8585 — they are infrastructure flakes on AWS with no code path overlap to the Azure-only changes in this PR. Consider re-triggering the job or merging with override if other required jobs pass.

  2. For the GlobalPullSecret flake: The global-pull-secret-syncer DaemonSet readiness check should investigate why 1 of 3 nodes consistently failed to run the pod. Adding diagnostic logging (node conditions, pod events, describe output for the unready pod) when the DaemonSet readiness times out would help debug future occurrences.

  3. For the Karpenter teardown flake: The Karpenter test teardown could benefit from a more aggressive resource cleanup strategy — e.g., force-deleting EBS volumes and NLBs via AWS API when the Kubernetes-level finalizer-based cleanup times out, or extending the teardown deadline for Karpenter tests that provision many nodepools.

  4. Monitoring operator unavailability: The ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator monitoring is not available) condition on the TestCreateCluster HostedCluster suggests a transient issue with the monitoring stack, possibly related to the same node-level problem that prevented the DaemonSet from becoming fully ready.

Evidence
Evidence Detail
PR #8585 changed files All 26 files are Azure-specific: CRD manifests, cmd/cluster/azure/, azureprivatelinkservice/controller.go, Azure docs
Failure 1: DaemonSet stuck global-pull-secret-syncer logged "not ready: 2/3 pods ready" 364 times over ~20 min before timeout
Failure 1: ovnkube-node healthy DaemonSet ovnkube-node ready: 3/3 pods — confirms all 3 nodes existed
Failure 1: cascade error kubelet-config-verifier subtest failed with HTTP 409 "already exists" due to prior subtest not cleaning up
Failure 1: monitoring CO ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator monitoring is not available) at RV 183109
Failure 2: Karpenter Main passed All TestKarpenter/Main/* and TestKarpenter/EnsureHostedCluster/* subtests passed
Failure 2: 10 orphaned resources 9 EBS volumes (from nodepools: capacity-reservation-test, arbitrary-subnet-test, kubelet-config-test, on-demand, instance-profile-test, version-test, 2 CAPI nodes) + 1 NLB
Failure 2: teardown duration TestKarpenter/Teardown ran for 1449.95s before timing out
Overall test pass rate 592/599 tests passed (98.8%), 25 skipped, only 2 independent root failures
Test step timing e2e-aws-hypershift-aws-run-e2e-nested ran for 1h13m36s total

@bryan-cox

Copy link
Copy Markdown
Member

/retest

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 962cdbb and 1 for PR HEAD eb45c67 in total

@openshift-ci

openshift-ci Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

@Nirshal: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit d433acd into openshift:main May 26, 2026
22 checks passed
@openshift-ci-robot

Copy link
Copy Markdown

@Nirshal: Jira Issue OCPBUGS-85351: Some pull requests linked via external trackers have merged:

The following pull request, linked via external tracker, has not merged:

All associated pull requests must be merged or unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with /jira refresh.

Jira Issue OCPBUGS-85351 has not been moved to the MODIFIED state.

This PR is marked as verified. If the remaining PRs listed above are marked as verified before merging, the issue will automatically be moved to VERIFIED after all of the changes from the PRs are available in an accepted nightly payload.

Details

In response to this:

Summary

  • The fix was originally introduced in OCPBUGS-85351: fix(azure): prevent externalDNSDomain from shadowing cluster apps domain #8480
  • It is being reverted in this PR because the CEL validation rejects legitimate ARO-HCP HostedCluster configurations. ARO-HCP uses service hostnames (e.g., api-int.{name}.domain) that share a parent domain with the cluster baseDomain (aro.{name}.domain), which the validation incorrectly flags as DNS zone shadowing.
  • This revert unblocks ARO-HCP E2E tests that are failing after bumping to the latest HyperShift operator. The shadowing prevention will need to be re-implemented with ARO-HCP's DNS configuration pattern taken into account.

References

/cc @bryan-cox @enxebre

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

brucebarrera pushed a commit to brucebarrera/ARO-HCP that referenced this pull request May 27, 2026
Unpin hypershift operator now that the domain shadowing revert has
landed upstream (openshift/hypershift#8585). The new image (89e19f83)
includes the fix. Also bumps ACM operator v2.16.2-440 and MCE v2.11.2-480.
wanghaoran1988 pushed a commit to wanghaoran1988/ARO-HCP that referenced this pull request Jun 23, 2026
Unpin hypershift operator now that the domain shadowing revert has
landed upstream (openshift/hypershift#8585). The new image (89e19f83)
includes the fix. Also bumps ACM operator v2.16.2-440 and MCE v2.11.2-480.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/api Indicates the PR includes changes for the API area/cli Indicates the PR includes changes for CLI area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/documentation Indicates the PR includes changes for documentation area/platform/azure PR/issue for Azure (AzurePlatform) platform jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants