Skip to content

Bug 1960284: Set the "local-with-fallback" service annotation#622

Merged
openshift-merge-robot merged 2 commits intoopenshift:masterfrom
Miciah:BZ1960284-set-the-local-with-fallback-service-annotation
Jun 7, 2021
Merged

Bug 1960284: Set the "local-with-fallback" service annotation#622
openshift-merge-robot merged 2 commits intoopenshift:masterfrom
Miciah:BZ1960284-set-the-local-with-fallback-service-annotation

Conversation

@Miciah
Copy link
Contributor

@Miciah Miciah commented Jun 1, 2021

Set the "local-with-fallback" service annotation

Set the traffic-policy.network.alpha.openshift.io/local-with-fallback annotation on LoadBalancer- and NodePort-type services that use the Local external traffic policy.

  • pkg/operator/controller/ingress/load_balancer_service.go (localWithFallbackAnnotation): New const. Define the annotation key for the local-with-fallback service annotation.
    (desiredLoadBalancerService): Set the local-with-fallback annotation when using the Local external traffic policy.
    (managedLoadBalancerServiceAnnotations): New variable. Define the keys for annotations that the operator manages on LoadBalancer-type services, which now include the local-with-fallback annotation.
    (loadBalancerServiceChanged): Fix the update logic to properly handle annotations with empty values using go-cmp and managedLoadBalancerServiceAnnotations.
  • pkg/operator/controller/ingress/load_balancer_service_test.go (TestDesiredLoadBalancerService): Verify that the local-with-fallback annotation is set when appropriate.
    (TestLoadBalancerServiceChanged): Verify that loadBalancerServiceChanged properly handles updates to the local-with-fallback annotation.
  • pkg/operator/controller/ingress/nodeport_service.go (desiredNodePortService): Set the local-with-fallback annotation.
    (managedNodePortServiceAnnotations): New variable. Define the annotation keys that the operator manages on NodePort-type services.
    (nodePortServiceChanged): Check if any of the annotations in managedNodePortServiceAnnotations have changed. Update managed annotations if they have changed.
  • pkg/operator/controller/ingress/nodeport_service_test.go (TestDesiredNodePortService): Verify that the expected annotations are set.
    (TestNodePortServiceChanged): Add a test case to verify that nodePortServiceChanged properly handles updates to the local-with-fallback annotation.

Add local-with-fallback unsupported config override

  • pkg/operator/controller/ingress/load_balancer_service.go (desiredLoadBalancerService): Use the new shouldUseLocalWithFallbackfunction to determine whether to set the local-with-fallback annotation.
    (shouldUseLocalWithFallback): New function. Check the service's external traffic policy and the ingresscontroller's unsupported config overrides and return a Boolean value indicating whether local-with-fallback should be enabled, or an error if the override could not be parsed.
  • pkg/operator/controller/ingress/load_balancer_service_test.go (TestShouldUseLocalWithFallback): New test. Verify that shouldUseLocalWithFallback behaves as expected.
  • pkg/operator/controller/ingress/nodeport_service.go (ensureNodePortService): Check the error value from desiredNodePortService.
    (desiredNodePortService): Add error return value. Use the new shouldUseLocalWithFallback function (which can return an error) to determine whether to set the local-with-fallback annotation.
  • pkg/operator/controller/ingress/nodeport_service_test.go (TestDesiredNodePortService): Check the error value from desiredNodePortService.
  • test/e2e/operator_test.go (TestLocalWithFallbackOverrideForLoadBalancerService, TestLocalWithFallbackOverrideForNodePortService): New tests. Verify that the operator does not set the local-with-fallback annotation on a LoadBalancer or NodePort service if the localWithFallback unsupported config override is set to "false".

@openshift-ci openshift-ci bot added the bugzilla/severity-urgent Referenced Bugzilla bug's severity is urgent for the branch this PR is targeting. label Jun 1, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 1, 2021

@Miciah: This pull request references Bugzilla bug 1960284, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.8.0) matches configured target release for branch (4.8.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @lihongan

Details

In response to this:

Bug 1960284: Set the "local-with-fallback" service annotation

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jun 1, 2021
@openshift-ci openshift-ci bot requested review from danehans, frobware and lihongan June 1, 2021 22:23
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 1, 2021
@Miciah Miciah force-pushed the BZ1960284-set-the-local-with-fallback-service-annotation branch from d4554f6 to 912b190 Compare June 2, 2021 01:04
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 2, 2021

@Miciah: This pull request references Bugzilla bug 1960284, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.8.0) matches configured target release for branch (4.8.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @lihongan

Details

In response to this:

Bug 1960284: Set the "local-with-fallback" service annotation

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Miciah
Copy link
Contributor Author

Miciah commented Jun 2, 2021

/retest

@sgreene570
Copy link
Contributor

/assign

@Miciah Miciah force-pushed the BZ1960284-set-the-local-with-fallback-service-annotation branch from 912b190 to dc339fe Compare June 2, 2021 18:20

// By default, use local-with-fallback when using the "Local" external
// traffic policy.
if service.Spec.ExternalTrafficPolicy == corev1.ServiceExternalTrafficPolicyTypeLocal {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return false if service.Spec.ExternalTrafficPolicy is not local?
As is, this logic would let a user use an unsupported config override to set the local-with-fallback annotation when service.Spec.ExternalTrafficPolicy is cluster, which is a state we shouldn't allow, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kind of ambivalent about allowing the user to do weird things with an unsupported config override. I'll go ahead and make the suggested change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Seems like some low-hanging fruit that we might as well patch while we can IMO.

// loadBalancerServiceChanged checks if the current load balancer service
// matches the expected and if not returns an updated one.
func loadBalancerServiceChanged(current, expected *corev1.Service) (bool, *corev1.Service) {
annotationCmpOpts := []cmp.Option{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome clean up!

if want && (!have || currentVal != expectedVal) {
updated.Annotations[annotation] = expected.Annotations[annotation]
} else if have && !want {
delete(updated.Annotations, annotation)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a unit test to cover this scenario?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Thanks!

if want && (!have || currentVal != expectedVal) {
updated.Annotations[annotation] = expected.Annotations[annotation]
} else if have && !want {
delete(updated.Annotations, annotation)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a unit test for this scenario?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. However, I had to replace the test case for adding the annotation with test cases for updating and deleting. To cover all three would make the test setup more complicated. If you want coverage for all three, I'd just as soon add a separate test. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Current coverage looks good enough to me! Thanks.

return !ok, nil
})
if _, ok := service.Annotations[annotation]; ok {
t.Fatalf("failed to observe removal of the %q annotation on ingresscontroller %q", annotation, defaultName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
t.Fatalf("failed to observe removal of the %q annotation on ingresscontroller %q", annotation, defaultName)
t.Fatalf("failed to observe removal of the %q annotation on service %q", annotation, serviceName)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Thanks!

return !ok, nil
})
if _, ok := service.Annotations[annotation]; ok {
t.Fatalf("failed to observe removal of the %q annotation on the %q ingresscontroller", annotation, icName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
t.Fatalf("failed to observe removal of the %q annotation on the %q ingresscontroller", annotation, icName)
t.Fatalf("failed to observe removal of the %q annotation on the %q service", annotation, serviceName)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Thanks!

@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 2, 2021
@Miciah Miciah force-pushed the BZ1960284-set-the-local-with-fallback-service-annotation branch from dc339fe to a83ff0c Compare June 3, 2021 18:14
Set the "traffic-policy.network.alpha.openshift.io/local-with-fallback"
annotation on LoadBalancer- and NodePort-type services that use the "Local"
external traffic policy.

This commit is related to bug 1960284.

https://bugzilla.redhat.com/show_bug.cgi?id=1960284

* pkg/operator/controller/ingress/load_balancer_service.go
(localWithFallbackAnnotation): New const.  Define the annotation key for
the "local-with-fallback" service annotation.
(desiredLoadBalancerService): Set the "local-with-fallback" annotation when
using the "Local" external traffic policy.
(managedLoadBalancerServiceAnnotations): New variable.  Define the keys for
annotations that the operator manages on LoadBalancer-type services, which
now include the "local-with-fallback" annotation.
(loadBalancerServiceChanged): Fix the update logic to properly handle
annotations with empty values using go-cmp and
managedLoadBalancerServiceAnnotations.
* pkg/operator/controller/ingress/load_balancer_service_test.go
(TestDesiredLoadBalancerService): Verify that the "local-with-fallback"
annotation is set when appropriate.
(TestLoadBalancerServiceChanged): Verify that loadBalancerServiceChanged
properly handles updates to the "local-with-fallback" annotation.
* pkg/operator/controller/ingress/nodeport_service.go
(desiredNodePortService): Set the "local-with-fallback" annotation.
(managedNodePortServiceAnnotations): New variable.  Define the annotation
keys that the operator manages on NodePort-type services.
(nodePortServiceChanged): Check if any of the annotations in
managedNodePortServiceAnnotations have changed.  Update managed annotations
if they have changed.
* pkg/operator/controller/ingress/nodeport_service_test.go
(TestDesiredNodePortService): Verify that the expected annotations are set.
(TestNodePortServiceChanged): Add a test case to verify that
nodePortServiceChanged properly handles updates to the
"local-with-fallback" annotation.
@Miciah Miciah force-pushed the BZ1960284-set-the-local-with-fallback-service-annotation branch from a83ff0c to 3326d75 Compare June 3, 2021 18:26
@Miciah
Copy link
Contributor Author

Miciah commented Jun 3, 2021

Rebased.

@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 3, 2021
@Miciah Miciah force-pushed the BZ1960284-set-the-local-with-fallback-service-annotation branch from 3326d75 to 56685eb Compare June 3, 2021 19:55
* pkg/operator/controller/ingress/load_balancer_service.go
(desiredLoadBalancerService): Use the new shouldUseLocalWithFallback
function to determine whether to set the "local-with-fallback" annotation.
(shouldUseLocalWithFallback): New function.  Check the service's external
traffic policy and the ingresscontroller's unsupported config overrides and
return a Boolean value indicating whether "local-with-fallback" should be
enabled, or an error if the override could not be parsed.
* pkg/operator/controller/ingress/load_balancer_service_test.go
(TestShouldUseLocalWithFallback): New test.  Verify that
shouldUseLocalWithFallback behaves as expected.
* pkg/operator/controller/ingress/nodeport_service.go
(ensureNodePortService): Check the error value from desiredNodePortService.
(desiredNodePortService): Add error return value.  Use the new
shouldUseLocalWithFallback function (which can return an error) to
determine whether to set the "local-with-fallback" annotation.
* pkg/operator/controller/ingress/nodeport_service_test.go
(TestDesiredNodePortService): Check the error value from
desiredNodePortService.
* test/e2e/operator_test.go
(TestLocalWithFallbackOverrideForLoadBalancerService)
(TestLocalWithFallbackOverrideForNodePortService): New tests.  Verify that
the operator does not set the "local-with-fallback" annotation on a
LoadBalancer or NodePort service if the localWithFallback unsupported
config override is set to "false".

Co-authored-by: candita <cholman@redhat.com>
@Miciah Miciah force-pushed the BZ1960284-set-the-local-with-fallback-service-annotation branch from 56685eb to fcc0f4a Compare June 3, 2021 19:56
@Miciah
Copy link
Contributor Author

Miciah commented Jun 4, 2021

/retest

@sgreene570
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 4, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 4, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Miciah, sgreene570

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Miciah
Copy link
Contributor Author

Miciah commented Jun 4, 2021

TestHTTPCookieCapture failed.
/test e2e-aws-operator

28/289 e2e-gcp-serial tests failed, nothing that looks related to ingress.
/test e2e-gcp-serial

@Miciah
Copy link
Contributor Author

Miciah commented Jun 4, 2021

/retest

@Miciah
Copy link
Contributor Author

Miciah commented Jun 5, 2021

/test e2e-gcp-serial
GCP's DNS seems to be broken.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

4 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit a535129 into openshift:master Jun 7, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 7, 2021

@Miciah: All pull requests linked via external trackers have merged:

Bugzilla bug 1960284 has been moved to the MODIFIED state.

Details

In response to this:

Bug 1960284: Set the "local-with-fallback" service annotation

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-urgent Referenced Bugzilla bug's severity is urgent for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants