Skip to content

OCPSTRAT-886: Support dual-stack LB in service disruption monitor#30936

Merged
openshift-merge-bot[bot] merged 1 commit intoopenshift:mainfrom
alebedev87:dual-stack-disruption-monitor-fix
Mar 31, 2026
Merged

OCPSTRAT-886: Support dual-stack LB in service disruption monitor#30936
openshift-merge-bot[bot] merged 1 commit intoopenshift:mainfrom
alebedev87:dual-stack-disruption-monitor-fix

Conversation

@alebedev87
Copy link
Copy Markdown
Contributor

@alebedev87 alebedev87 commented Mar 26, 2026

Configure the service load balancer disruption test to use NLB with RequireDualStack IP family policy when the Infrastructure CR indicates dual-stack IP families on AWS, matching the primary IP family order.

Failure on IPv6Primary cluster:

: [Monitor:service-type-load-balancer-availability][Jira:"Networking / router"] monitor test service-type-load-balancer-availability preparation expand_less	15m0s
{  failed during preparation
error waiting for load balancer: timed out waiting for service "service-test" to have a load balancer: context deadline exceeded}

Example of a failed job: link.

@openshift-ci-robot
Copy link
Copy Markdown

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 26, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 008f6c27-d275-4eab-a424-69ab48b25db7

📥 Commits

Reviewing files that changed from the base of the PR and between ffa7b5a and ed71c18.

📒 Files selected for processing (1)
  • pkg/monitortests/network/disruptionserviceloadbalancer/monitortest.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/monitortests/network/disruptionserviceloadbalancer/monitortest.go

Walkthrough

Inspect Infrastructure CR AWS IPFamily and, when dual-stack, annotate TCP LoadBalancer services for NLB and set Service.Spec.IPFamilyPolicy to RequireDualStack with IPFamilies ordered per AWS primary IP family (IPv4-primary or IPv6-primary).

Changes

Cohort / File(s) Summary
AWS Dual-Stack LoadBalancer Configuration
pkg/monitortests/network/disruptionserviceloadbalancer/monitortest.go
On TCP Service creation, if infra.Status.PlatformStatus.AWS is non-nil and IPFamily is DualStackIPv4Primary or DualStackIPv6Primary, set service.beta.kubernetes.io/aws-load-balancer-type: "nlb", Spec.IPFamilyPolicy = RequireDualStack, and Spec.IPFamilies to [IPv4, IPv6] or [IPv6, IPv4] respectively. No dual-stack fields are set for other/undefined values.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.11.4)

Error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions
The command is terminated due to an error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions


Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci bot requested review from p0lyn0mial and sjenning March 26, 2026 09:42
@openshift-ci-robot
Copy link
Copy Markdown

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@alebedev87
Copy link
Copy Markdown
Contributor Author

/testwith openshift/installer/main/e2e-aws-ovn-dualstack-ipv6-primary-techpreview openshift/cloud-provider-aws#135

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 26, 2026

@alebedev87, testwith: could not generate prow job. ERROR:

no ref for requested test included in command. The org, repo, and branch containing the requested test need to be targeted by at least one of the included PRs

@alebedev87
Copy link
Copy Markdown
Contributor Author

/testwith openshift/installer/main/e2e-aws-ovn-dualstack-ipv6-primary-techpreview openshift/cloud-provider-aws#135

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 26, 2026

@alebedev87, testwith: could not generate prow job. ERROR:

no ref for requested test included in command. The org, repo, and branch containing the requested test need to be targeted by at least one of the included PRs

@alebedev87
Copy link
Copy Markdown
Contributor Author

/testwith openshift/installer/main/e2e-aws-ovn-dualstack-ipv6-primary-techpreview openshift/cloud-provider-aws#135

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 26, 2026

@alebedev87, testwith: could not generate prow job. ERROR:

no ref for requested test included in command. The org, repo, and branch containing the requested test need to be targeted by at least one of the included PRs

@alebedev87
Copy link
Copy Markdown
Contributor Author

/testwith openshift/installer/main/e2e-aws-ovn-dualstack-ipv6-primary-techpreview openshift/cloud-provider-aws#135 openshift/installer#10380

Comment on lines +188 to +190
s.Annotations["service.beta.kubernetes.io/aws-load-balancer-type"] = "nlb"
dualStack := corev1.IPFamilyPolicyRequireDualStack
s.Spec.IPFamilyPolicy = &dualStack
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor and shouldn't block merging if it's passing tests, but could we move these 3 lines out of the case statements so it's more clear that the only variable is the order of IP families?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the boilerplate bothered me too. I tried a couple of approaches but they brought even more code. If you have an idea, I'm open for suggestions.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a little late to the party, but we probably can try something like:

// Configure dual-stack if the Infrastructure CR indicates dual-stack IP families.
// NLB is required on AWS for dual-stack load balancers.
if infra.Status.PlatformStatus.AWS != nil {                                                                                                                                
      var ipFamilies []corev1.IPFamily                                                                                                                                       
                                                                                                                                                                             
      switch infra.Status.PlatformStatus.AWS.IPFamily {                                                                                                                      
      case configv1.DualStackIPv4Primary:                                                                                                                                    
          ipFamilies = []corev1.IPFamily{corev1.IPv4Protocol, corev1.IPv6Protocol}                                                                                           
      case configv1.DualStackIPv6Primary:                                                                                                                                    
          ipFamilies = []corev1.IPFamily{corev1.IPv6Protocol, corev1.IPv4Protocol}                                                                                           
      }                                                                                                                                                                      
                                                                                                                                                                             
      if len(ipFamilies) > 1 {                                                                                                                                               
          s.Annotations["service.beta.kubernetes.io/aws-load-balancer-type"] = "nlb"
          dualStack := corev1.IPFamilyPolicyRequireDualStack                                                                                                                 
          s.Spec.IPFamilyPolicy = &dualStack                                                                                                                                 
          s.Spec.IPFamilies = ipFamilies                                                                                                                                     
      }                                                                                                                                                                      
  }                   

Not blocking though as tests passed 😁

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! Thanks @tthvo! Updated the PR.

@alebedev87
Copy link
Copy Markdown
Contributor Author

: [Monitor:service-type-load-balancer-availability][Jira:"Networking / router"] monitor test service-type-load-balancer-availability preparation

The modified test passed for the dualstack-ipv6primary job but interesting that it failed on the presubmit, all is pointing to some hiccups from AWS, retrying to be sure..

/test e2e-aws-ovn-fips

Configure the service load balancer disruption test to use NLB with
`RequireDualStack` IP family policy when the Infrastructure CR indicates
dual-stack IP families on AWS, matching the primary IP family order.
@alebedev87 alebedev87 force-pushed the dual-stack-disruption-monitor-fix branch from ffa7b5a to ed71c18 Compare March 27, 2026 08:05
@openshift-ci-robot
Copy link
Copy Markdown

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@alebedev87
Copy link
Copy Markdown
Contributor Author

FIPS job is broken, latest router changes are suspected, Claude Code points to a TLS curve which is not available in FIPS:

Here are the router logs. The root cause is crystal clear at lines 17-26:                                                          
                                      
  error reloading router: exit status 1                                                                                              
  [NOTICE]   (13) : haproxy version is 2.8.18-ae90be6                                                                                
  [ALERT]    (13) : config : Proxy 'fe_sni': unable to set SSL curves list to                                                        
    'X25519MLKEM768:X25519:P-256' for bind 'unix@/var/lib/haproxy/run/haproxy-sni.sock'                                              
  [ALERT]    (13) : config : Proxy 'fe_no_sni': unable to set SSL curves list to                                                     
    'X25519MLKEM768:X25519:P-256' for bind 'unix@/var/lib/haproxy/run/haproxy-no-sni.sock'                                           
  [ALERT]    (13) : config : Fatal errors found in configuration.                                                                    
                                                                                                                                     
  HAProxy cannot start because it's unable to set the SSL curves list to X25519MLKEM768:X25519:P-256. This is a FIPS-specific issue —
   the X25519MLKEM768 curve (a post-quantum key exchange algorithm) is not available in FIPS mode because it's not FIPS-approved.    
  HAProxy exits with a fatal configuration error, so:                                                                                
              
  1. HAProxy never starts -> no socket at /var/lib/haproxy/run/haproxy.sock                                                          
  2. The backend-proxy-http healthz check fails (connection refused on port 80)
  3. Startup probe kills the container after repeated failures                                                                       
  4. Both router pods enter CrashLoopBackOff                                                                                         
  5. Ingress operator reports unavailable -> authentication and console also fail                                                    
  6. Installation times out

@alebedev87
Copy link
Copy Markdown
Contributor Author

/test e2e-vsphere-ovn-upi
/test e2e-metal-ipi-ovn-ipv6

Copy link
Copy Markdown
Member

@tthvo tthvo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 27, 2026
@nrb
Copy link
Copy Markdown
Contributor

nrb commented Mar 27, 2026

/lgtm

@alebedev87
Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ovn-fips

@ShudiLi
Copy link
Copy Markdown
Member

ShudiLi commented Mar 30, 2026

/verified by e2e

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@ShudiLi: This PR has been marked as verified by e2e.

Details

In response to this:

/verified by e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@sadasu
Copy link
Copy Markdown
Contributor

sadasu commented Mar 30, 2026

/approve

@sadasu
Copy link
Copy Markdown
Contributor

sadasu commented Mar 30, 2026

/retitle NO-JIRA: Support dual-stack LB in service disruption monitor

@openshift-ci openshift-ci bot changed the title Support dual-stack LB in service disruption monitor test NO-JIRA: Support dual-stack LB in service disruption monitor Mar 30, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@alebedev87: This pull request explicitly references no jira issue.

Details

In response to this:

Configure the service load balancer disruption test to use NLB with RequireDualStack IP family policy when the Infrastructure CR indicates dual-stack IP families on AWS, matching the primary IP family order.

Failure on IPv6Primary cluster:

: [Monitor:service-type-load-balancer-availability][Jira:"Networking / router"] monitor test service-type-load-balancer-availability preparation expand_less	15m0s
{  failed during preparation
error waiting for load balancer: timed out waiting for service "service-test" to have a load balancer: context deadline exceeded}

Example of a failed job: link.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@alebedev87 alebedev87 changed the title NO-JIRA: Support dual-stack LB in service disruption monitor OCPSTRAT-886: Support dual-stack LB in service disruption monitor Mar 31, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Mar 31, 2026

@alebedev87: This pull request references OCPSTRAT-886 which is a valid jira issue.

Details

In response to this:

Configure the service load balancer disruption test to use NLB with RequireDualStack IP family policy when the Infrastructure CR indicates dual-stack IP families on AWS, matching the primary IP family order.

Failure on IPv6Primary cluster:

: [Monitor:service-type-load-balancer-availability][Jira:"Networking / router"] monitor test service-type-load-balancer-availability preparation expand_less	15m0s
{  failed during preparation
error waiting for load balancer: timed out waiting for service "service-test" to have a load balancer: context deadline exceeded}

Example of a failed job: link.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@neisw
Copy link
Copy Markdown
Contributor

neisw commented Mar 31, 2026

/approve

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 31, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alebedev87, neisw, nrb, sadasu, tthvo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 31, 2026
@openshift-ci-robot
Copy link
Copy Markdown

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@alebedev87
Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ovn-serial-1of2

@alebedev87
Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ovn-fips

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 31, 2026

@alebedev87: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 5f58686 into openshift:main Mar 31, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants