SPLAT-2220: tests/ote: introduce openshift-only e2e to validate feature by feature gate by mtulio · Pull Request #126 · openshift/cloud-provider-aws

mtulio · 2026-02-10T06:53:29Z

Summary

This PR introduces OpenShift-specific e2e tests for the AWSServiceLBNetworkSecurityGroup feature gate, which enables managed security groups for Network Load Balancers (NLB).

The tests introduced in this proposal exercises the feature, which is currently (4.21) under TechPreviewNoUpgrade feature set, validating the following changes exposed by the feature:

cloud config option NLBSecurityGroupMode is added and set to value Managed, so that CCM controller will react to new configuration by default creating security groups for all NLBs
ensure load balancer bound to the service is created with managed security group
ensure the default router's service NLB, openshift-component mainly impacted and targeted by this change, is created with managed security group

Test Coverage

The test suite includes three comprehensive test cases:

Cloud Config Validation - Verifies cloud-config ConfigMap contains NLBSecurityGroupMode = Managed
NLB Service Creation - Creates a test NLB service and validates security groups are attached
Default Ingress Controller - Validates the default router NLB has security groups attached

Test Behavior

TechPreviewNoUpgrade feature set: Tests execute and validate functionality
Default/GA feature set: Tests skip when feature gate is disabled
Tests accessible via OpenShift Test Extension (OTE) binary interface

Validation

✅ CI tests passing on both e2e-aws-ovn and e2e-aws-ovn-techpreview
✅ Tests correctly skip when feature gate disabled
✅ Tests validate managed security group attachment to NLBs

Note to Reviewer

There are two commits on this PR:

the new e2e test package using the ginkgo DSL, imported by OTE: UPSTREAM: : test/ote added tests for NLB SG feature
go.mod dependencies (vendoring) updates required in the OTE (only downstream) binary: UPSTREAM: : test/ote/vendor update dependencies

Jira: SPLAT-2220

Reviewed-by: Claude Sonnet 4.5 noreply@anthropic.com (also generated summary and function comments)

openshift-ci · 2026-02-10T06:53:33Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

mtulio · 2026-02-10T06:53:36Z

/test ?

mtulio · 2026-02-10T06:54:00Z

/test e2e-aws-ovn-techpreview

mtulio · 2026-02-10T06:58:37Z

/test e2e-aws-ovn-techpreview

mtulio · 2026-02-10T14:20:08Z

2/3 tests with prefix cloud-provider-aws-e2e-openshift succeeded. The cloud-config check failed due test not considering tabulation between k/v of the config. I just fixed it, let me get more signals for both TP and regular/GA:

/test e2e-aws-ovn-techpreview
/test e2e-aws-ovn

mtulio · 2026-02-10T18:43:14Z

Perfect, both e2e-aws-ovn(1) and e2e-aws-ovn-techpreview(2) are now green, and tests with prefix cloud-provider-aws-e2e-openshift are passing in the TPNU(2) and skipped in the default/GA(1) feature set.

I just sent cleaned up the code to capture readiness.

/test e2e-aws-ovn-techpreview
/test e2e-aws-ovn

openshift-ci-robot · 2026-02-10T18:44:22Z

@mtulio: This pull request references SPLAT-2220 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

SPLAT-2220

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-02-10T18:46:07Z

@mtulio: This pull request references SPLAT-2220 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

This PR introduces an openshift-only e2e tests to validate OpenShift features, merging those on tests exposed by OTE to the test framework.

SPLAT-2220

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

mtulio · 2026-02-10T20:21:00Z

/test okd-scos-images

tests/aws-cloud-controller-manager-tests-ext/e2e/loadbalancer.go

mtulio · 2026-02-10T21:58:27Z

Second round of jobs are passing for test prefix cloud-provider-aws-e2e-openshift:

e2e-aws-ovn: have 3/3 skipped tests
e2e-aws-ovn-techpreview: have 3/3 passed tests

I just added minor changes improving how the feature gate is validated from the cluster, instead of unstructured API validation, now it is using the openshift API. Re-run for green signals:

/test e2e-aws-ovn-techpreview

Tested locally successfully (except last one which was skipped due my cluster limitation: non NLB for ingress):


  Ran 1 of 1 Specs in 1.978 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped

]	Command being timed: "./aws-cloud-controller-manager-tests-ext run-test [cloud-provider-aws-e2e-openshift] loadbalancer NLB feature AWSServiceLBNetworkSecurityGroup should have NLBSecurityGroupMode with 'Managed value in cloud-config [Suite:openshift/conformance/parallel]"


$./aws-cloud-controller-manager-tests-ext run-test "[cloud-provider-aws-e2e-openshift] loadbalancer NLB feature AWSServiceLBNetworkSecurityGroup should create NLB service with security group attached [Suite:openshift/conformance/parallel]"
...

  Ran 1 of 1 Specs in 7.440 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
...

$(which time) -v ./aws-cloud-controller-manager-tests-ext run-test "[cloud-provider-aws-e2e-openshift] loadbalancer NLB feature AWSServiceLBNetworkSecurityGroup should have security groups attached to default ingress controller NLB [Suite:openshift/conformance/parallel]"

  Ran 0 of 1 Specs in 1.752 seconds
  SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 1 Skipped

rvanderp3 · 2026-02-10T22:07:08Z

/lgtm

mtulio · 2026-02-11T02:20:50Z

/retest-required

openshift-ci-robot · 2026-02-11T02:30:13Z

@mtulio: This pull request references SPLAT-2220 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

This PR introduces an openshift-only e2e tests to validate OpenShift features, merging those on tests exposed by OTE to the test framework.

Summary

This PR introduces OpenShift-specific e2e tests for the AWSServiceLBNetworkSecurityGroup feature gate, which enables managed security groups for Network Load Balancers (NLB).

The tests introduced in this proposal exercises the feature, which is currently (4.21) under TechPreviewNoUpgrade feature set, validating the following changes exposed by the feature:

cloud config option NLBSecurityGroupMode is added and set to value Managed, so that CCM controller will react to new configuration by default creating security groups for all NLBs

ensure load balancer bound to the service is created with managed security group

ensure the default router's service NLB, openshift-component mainly impacted and targeted by this change, is created with managed security group

Test Coverage

The test suite includes three comprehensive test cases:

Cloud Config Validation - Verifies cloud-config ConfigMap contains NLBSecurityGroupMode = Managed

NLB Service Creation - Creates a test NLB service and validates security groups are attached

Default Ingress Controller - Validates the default router NLB has security groups attached

Test Behavior

TechPreviewNoUpgrade feature set: Tests execute and validate functionality

Default/GA feature set: Tests skip when feature gate is disabled

Tests accessible via OpenShift Test Extension (OTE) binary interface

Validation

✅ CI tests passing on both e2e-aws-ovn and e2e-aws-ovn-techpreview

✅ Tests correctly skip when feature gate disabled

✅ Tests validate managed security group attachment to NLBs

Jira: SPLAT-2220

SPLAT-2220

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-02-11T02:33:49Z

@mtulio: This pull request references SPLAT-2220 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

This PR introduces an openshift-only e2e tests to validate OpenShift features, merging those on tests exposed by OTE to the test framework.

Summary

This PR introduces OpenShift-specific e2e tests for the AWSServiceLBNetworkSecurityGroup feature gate, which enables managed security groups for Network Load Balancers (NLB).

The tests introduced in this proposal exercises the feature, which is currently (4.21) under TechPreviewNoUpgrade feature set, validating the following changes exposed by the feature:

cloud config option NLBSecurityGroupMode is added and set to value Managed, so that CCM controller will react to new configuration by default creating security groups for all NLBs

ensure load balancer bound to the service is created with managed security group

ensure the default router's service NLB, openshift-component mainly impacted and targeted by this change, is created with managed security group

Test Coverage

The test suite includes three comprehensive test cases:

Cloud Config Validation - Verifies cloud-config ConfigMap contains NLBSecurityGroupMode = Managed

NLB Service Creation - Creates a test NLB service and validates security groups are attached

Default Ingress Controller - Validates the default router NLB has security groups attached

Test Behavior

TechPreviewNoUpgrade feature set: Tests execute and validate functionality

Default/GA feature set: Tests skip when feature gate is disabled

Tests accessible via OpenShift Test Extension (OTE) binary interface

Validation

✅ CI tests passing on both e2e-aws-ovn and e2e-aws-ovn-techpreview

✅ Tests correctly skip when feature gate disabled

✅ Tests validate managed security group attachment to NLBs

Note to Reviewer

There are two commits on this PR:

the new e2e test package using the ginkgo DSL, imported by OTE: UPSTREAM: : test/ote added tests for NLB SG feature

go.mod dependencies (vendoring) updates required in the OTE (only downstream) binary: UPSTREAM: : test/ote/vendor update dependencies

Jira: SPLAT-2220

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

mtulio · 2026-02-11T02:34:20Z

/verified by @mtulio #126 (comment)

openshift-ci-robot · 2026-02-11T02:34:33Z

@mtulio: This PR has been marked as verified by @mtulio https://github.com/openshift/cloud-provider-aws/pull/126#issuecomment-3880959472.

Details

In response to this:

/verified by @mtulio #126 (comment)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

mtulio · 2026-02-11T15:24:01Z

/assign @nrb @elmiko

openshift-ci-robot · 2026-02-12T14:57:37Z

@mtulio: This pull request references SPLAT-2220 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Summary

This PR introduces OpenShift-specific e2e tests for the AWSServiceLBNetworkSecurityGroup feature gate, which enables managed security groups for Network Load Balancers (NLB).

The tests introduced in this proposal exercises the feature, which is currently (4.21) under TechPreviewNoUpgrade feature set, validating the following changes exposed by the feature:

cloud config option NLBSecurityGroupMode is added and set to value Managed, so that CCM controller will react to new configuration by default creating security groups for all NLBs

ensure load balancer bound to the service is created with managed security group

ensure the default router's service NLB, openshift-component mainly impacted and targeted by this change, is created with managed security group

Test Coverage

The test suite includes three comprehensive test cases:

Cloud Config Validation - Verifies cloud-config ConfigMap contains NLBSecurityGroupMode = Managed

NLB Service Creation - Creates a test NLB service and validates security groups are attached

Default Ingress Controller - Validates the default router NLB has security groups attached

Test Behavior

TechPreviewNoUpgrade feature set: Tests execute and validate functionality

Default/GA feature set: Tests skip when feature gate is disabled

Tests accessible via OpenShift Test Extension (OTE) binary interface

Validation

✅ CI tests passing on both e2e-aws-ovn and e2e-aws-ovn-techpreview

✅ Tests correctly skip when feature gate disabled

✅ Tests validate managed security group attachment to NLBs

Note to Reviewer

There are two commits on this PR:

the new e2e test package using the ginkgo DSL, imported by OTE: UPSTREAM: : test/ote added tests for NLB SG feature

go.mod dependencies (vendoring) updates required in the OTE (only downstream) binary: UPSTREAM: : test/ote/vendor update dependencies

Jira: SPLAT-2220

Reviewed-by: Claude Sonnet 4.5 noreply@anthropic.com (also generated summary and function comments)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

tests/aws-cloud-controller-manager-tests-ext/e2e/loadbalancer.go

Update Go dependencies for OTE binary to use openshift api for feature gate. https://issues.redhat.com/browse/SPLAT-2220 https://issues.redhat.com/browse/SPLAT-2553

Introduce e2e tests (openshift/downstream only) to test features on OpenShift, exposing those through OTE binary interface. The tests are exercising the feature 'managed security group to NLB' under the gate "AWSServiceLBNetworkSecurityGroup". Currently it is under TechPreviewNoUpgrade feature set, but those tests must check if the gate is enabled under the current feature set, skip otherwise. https://issues.redhat.com/browse/SPLAT-2220 https://issues.redhat.com/browse/SPLAT-2553 Reviewed-by: Claude Sonnet 4.5 <noreply@anthropic.com>

mtulio · 2026-02-16T17:36:01Z

Last commit only removed header comments.

/verified by #126 (comment)

openshift-ci-robot · 2026-02-16T17:36:13Z

@mtulio: This PR has been marked as verified by https://github.com/openshift/cloud-provider-aws/pull/126#issuecomment-3881773775.

Details

In response to this:

Last commit only removed header comments.

/verified by #126 (comment)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

mtulio · 2026-02-17T17:08:56Z

/test e2e-aws-ovn-techpreview

openshift-ci · 2026-02-17T20:29:11Z

@mtulio: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

mtulio · 2026-02-18T12:55:07Z

This is a critical feature for product:
/label acknowledge-critical-fixes-only

jcpowermac · 2026-02-18T12:56:13Z

/lgtm

nrb

/lgtm
/approve

nrb · 2026-02-18T20:10:18Z

tests/aws-cloud-controller-manager-tests-ext/e2e/loadbalancer.go

+	})
+
+	// Checker function to verify if the feature gate is enabled for the group of tests for feature AWSServiceLBNetworkSecurityGroup.
+	isNLBFeatureEnabled := func(ctx context.Context) {


Might be nice to mark this as a GinkgoHelper in case we hit errors, but not a blocker.

Good suggestion, Nolan. Since this PR has been merged, I created the card SPLAT-2638 to track this.

thanks for reviewing

openshift-ci · 2026-02-18T20:23:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nrb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [nrb]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 10, 2026

mtulio force-pushed the e2e-ocp-svc-nlb-sg branch from ad54805 to e2e75ad Compare February 10, 2026 06:58

mtulio force-pushed the e2e-ocp-svc-nlb-sg branch from abba814 to 1c7edf2 Compare February 10, 2026 18:42

mtulio marked this pull request as ready for review February 10, 2026 18:43

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 10, 2026

openshift-ci bot requested review from chrischdi and mdbooth February 10, 2026 18:43

mtulio changed the title ~~DNM/e2e/ote: introduce downstream-only tests for feature readiness~~ SPLAT-2220: tests/ote: introduce openshift-only e2e to validate feature by feature gate Feb 10, 2026

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 10, 2026

mtulio commented Feb 10, 2026

View reviewed changes

tests/aws-cloud-controller-manager-tests-ext/e2e/loadbalancer.go Outdated Show resolved Hide resolved

mtulio force-pushed the e2e-ocp-svc-nlb-sg branch from 1c7edf2 to 19a61ca Compare February 10, 2026 21:58

openshift-ci bot assigned rvanderp3 Feb 10, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 10, 2026

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Feb 11, 2026

openshift-ci bot assigned elmiko and nrb Feb 11, 2026

mtulio mentioned this pull request Feb 12, 2026

SPLAT-2587: aws/ccm: introduce configuration to CCM managed Security Groups for NLB openshift/hypershift#7460

Open

4 tasks

nrb reviewed Feb 16, 2026

View reviewed changes

tests/aws-cloud-controller-manager-tests-ext/e2e/loadbalancer.go Outdated Show resolved Hide resolved

UPSTREAM: <carry>: test/ote/vendor update dependencies

39c96d3

Update Go dependencies for OTE binary to use openshift api for feature gate. https://issues.redhat.com/browse/SPLAT-2220 https://issues.redhat.com/browse/SPLAT-2553

mtulio force-pushed the e2e-ocp-svc-nlb-sg branch from 19a61ca to cc1d49e Compare February 16, 2026 17:27

openshift-ci-robot removed the verified Signifies that the PR passed pre-merge verification criteria label Feb 16, 2026

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Feb 16, 2026

mtulio force-pushed the e2e-ocp-svc-nlb-sg branch from cc1d49e to 716f71d Compare February 16, 2026 17:28

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Feb 16, 2026

openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Feb 18, 2026

openshift-ci bot assigned jcpowermac Feb 18, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 18, 2026

nrb approved these changes Feb 18, 2026

View reviewed changes

mtulio mentioned this pull request Feb 18, 2026

SPLAT-2637: Promoting gate AWSServiceLBNetworkSecurityGroup to GA openshift/api#2717

Merged

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 18, 2026

openshift-merge-bot bot merged commit 214f667 into openshift:main Feb 18, 2026
9 checks passed

mtulio deleted the e2e-ocp-svc-nlb-sg branch February 18, 2026 21:28

mtulio mentioned this pull request Feb 25, 2026

SPLAT-2220: tests/ote: extend tests for gate AWSServiceLBNetworkSecurityGroup #129

Merged

mtulio mentioned this pull request Mar 26, 2026

SPLAT-2672: e2e/ote/ccm-aws: moving OTE binary from CCM-AWS openshift/cluster-cloud-controller-manager-operator#437

Merged

Conversation

mtulio commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Coverage

Test Behavior

Validation

Note to Reviewer

Uh oh!

openshift-ci bot commented Feb 10, 2026

Uh oh!

mtulio commented Feb 10, 2026

Uh oh!

mtulio commented Feb 10, 2026

Uh oh!

mtulio commented Feb 10, 2026

Uh oh!

mtulio commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mtulio commented Feb 10, 2026

Uh oh!

openshift-ci-robot commented Feb 10, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Feb 10, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mtulio commented Feb 10, 2026

Uh oh!

Uh oh!

mtulio commented Feb 10, 2026

Uh oh!

rvanderp3 commented Feb 10, 2026

Uh oh!

mtulio commented Feb 11, 2026

Uh oh!

openshift-ci-robot commented Feb 11, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Coverage

Test Behavior

Validation

Uh oh!

openshift-ci-robot commented Feb 11, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Coverage

Test Behavior

Validation

Note to Reviewer

Uh oh!

mtulio commented Feb 11, 2026

Uh oh!

openshift-ci-robot commented Feb 11, 2026

Uh oh!

mtulio commented Feb 11, 2026

Uh oh!

openshift-ci-robot commented Feb 12, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Coverage

Test Behavior

Validation

Note to Reviewer

Uh oh!

Uh oh!

mtulio commented Feb 16, 2026

Uh oh!

openshift-ci-robot commented Feb 16, 2026

Uh oh!

mtulio commented Feb 17, 2026

Uh oh!

openshift-ci bot commented Feb 17, 2026

Uh oh!

mtulio commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcpowermac commented Feb 18, 2026

Uh oh!

mtulio commented Feb 10, 2026 •

edited

Loading

mtulio commented Feb 10, 2026 •

edited

Loading

openshift-ci-robot commented Feb 10, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Feb 10, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Feb 11, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Feb 11, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Feb 12, 2026 •

edited by openshift-ci bot

Loading

mtulio commented Feb 18, 2026 •

edited

Loading