Skip to content

OCPBUGS-10799: [release-4.13] Netpol performance: shared peer address sets, add metrics#1603

Merged
openshift-merge-robot merged 8 commits intoopenshift:release-4.13from
npinaeva:netpol-perf-4.13
Mar 27, 2023
Merged

OCPBUGS-10799: [release-4.13] Netpol performance: shared peer address sets, add metrics#1603
openshift-merge-robot merged 8 commits intoopenshift:release-4.13from
npinaeva:netpol-perf-4.13

Conversation

@npinaeva
Copy link
Copy Markdown
Contributor

@npinaeva npinaeva commented Mar 23, 2023

Bring required changes to 4.13
perf metrics ovn-kubernetes/ovn-kubernetes#3450
downstream #1556

shared address sets ovn-kubernetes/ovn-kubernetes#3329
downstream #1574

No conflicts, last commit fixes a "conflic" because k8s 1.26 bump is not yet merged to 4.13

ovn-k master to start watching every resource.
Add scale metrics for network policies, rename existing
enable-eip-scale-metrics flag to more general enable-scale-metrics, and
use it for network policy metric too.

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit 4a3f1dd)
Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit fbc9e4e)
required, to unlock other handlers from the same namespace.

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit f6f3ed8)
address sets for pod selector (network policy object is only
responsible for local pods and peer namespace-only handlers).
The locking mechanism is copied from networkPolicy.

Update deletre logic for completed pods: match collided pod
not only by podSeelctor, but also by namespace selector.

PeerPod functions were moved from policy.go and gress_policy.go.

Update syncNetworkPolicies to cleanup policies based on acls, and not
address sets, since address sets are not created for policies without
peers with selectors, and it doesn't cleanup default deny port groups.
New sync is based on acls, it will only skip empty policies without
any gress rules. This should be fixed later with proper ownership
indexing for port groups.

Rename metrics from peer/network_policy to pod_selector_address_set.

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit 120140f)
least one selector had peerAddressSet, and empty gress (allow all)
was identified by "gp.sizeOfAddressSet() > 0". Now we don't create
address sets like that, therefore a new hasPeerSelector field was added
to distinguish empty gress from the one that just doesn't have any
address sets added yet.

Previously l3Match for gress with namespace selector that doesn't select
anything looked like "ip4.src == {<empty address set>}", now it will be
"ip4.src == {}".

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>

update 2

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit ed34024)
Add new functions to FakeAddressSetFactory for more insight into
existing address sets.
Add tests for PodSelectorAddressSet

Update Netpol-owned address sets to be shared.
Add netpol test that verifies that default deny port groups and port
groups for policies without peer selectors (IpBlock) are cleaned up
on sync.

Move completed pod test from policy_test to pod_selector_address_set_test,
simplify the test and make sure ip will be removed when collided pod
is not selected by the address set.

Update policy sync tests with existing policy in every namespace
to make sure port groups won't be deleted as stale before being updated.

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit 7f4b409)
Previously "ip4.dst == {}" match was created, and ovn-controller
throws an error on such acl

Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
(cherry picked from commit 54b67a3)
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Mar 23, 2023
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@npinaeva: This pull request references Jira Issue OCPBUGS-10799, which is invalid:

  • expected Jira Issue OCPBUGS-10799 to depend on a bug targeting a version in 4.14.0 and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Bring required changes to 4.13
perf metrics ovn-kubernetes/ovn-kubernetes#3450
downstream #1556

shared address sets ovn-kubernetes/ovn-kubernetes#3329
downstream #1574

Clean cherry-pick

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 23, 2023

@npinaeva: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

OCPBUGS-10799: [release-4.13] Netpol performance: shared peer address sets, add metrics

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@npinaeva
Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@npinaeva: This pull request references Jira Issue OCPBUGS-10799, which is invalid:

  • expected dependent Jira Issue OCPBUGS-10800 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is Closed (Done) instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot requested review from JacobTanenbaum and abhat March 23, 2023 16:35
@npinaeva
Copy link
Copy Markdown
Contributor Author

/retest

@npinaeva
Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Mar 24, 2023
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@npinaeva: This pull request references Jira Issue OCPBUGS-10799, which is valid. The bug has been moved to the POST state.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.13.0) matches configured target version for branch (4.13.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • dependent bug Jira Issue OCPBUGS-10800 is in the state MODIFIED, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-10800 targets the "4.14.0" version, which is one of the valid target versions: 4.14.0
  • bug has dependents

Requesting review from QA contact:
/cc @anuragthehatter

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot requested a review from anuragthehatter March 24, 2023 14:47
Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@npinaeva: This pull request references Jira Issue OCPBUGS-10799, which is valid.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.13.0) matches configured target version for branch (4.13.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • dependent bug Jira Issue OCPBUGS-10800 is in the state MODIFIED, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-10800 targets the "4.14.0" version, which is one of the valid target versions: 4.14.0
  • bug has dependents

Requesting review from QA contact:
/cc @anuragthehatter

Details

In response to this:

Bring required changes to 4.13
perf metrics ovn-kubernetes/ovn-kubernetes#3450
downstream #1556

shared address sets ovn-kubernetes/ovn-kubernetes#3329
downstream #1574

No conflicts, last commit fixes a "conflic" because k8s 1.26 bump is not yet merged to 4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 24, 2023

@npinaeva: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Retaining the bugzilla/valid-bug label as it was manually added.

Details

In response to this:

OCPBUGS-10799: [release-4.13] Netpol performance: shared peer address sets, add metrics

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link
Copy Markdown
Contributor

@trozet trozet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/label backport-risk-assessed

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Mar 24, 2023
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 24, 2023
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 24, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: npinaeva, trozet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 24, 2023
@anuragthehatter
Copy link
Copy Markdown

/label cherry-pick-approved

@openshift-ci openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Mar 27, 2023
@npinaeva
Copy link
Copy Markdown
Contributor Author

/retest-required

@openshift-ci-robot
Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD ebb5dca and 2 for PR HEAD 8513b38 in total

@anuragthehatter
Copy link
Copy Markdown

/label cherry-pick-approved

@dcbw
Copy link
Copy Markdown
Contributor

dcbw commented Mar 27, 2023

/retest

@openshift-merge-robot openshift-merge-robot merged commit cb89e52 into openshift:release-4.13 Mar 27, 2023
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@npinaeva: Jira Issue OCPBUGS-10799: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-10799 has been moved to the MODIFIED state.

Details

In response to this:

Bring required changes to 4.13
perf metrics ovn-kubernetes/ovn-kubernetes#3450
downstream #1556

shared address sets ovn-kubernetes/ovn-kubernetes#3329
downstream #1574

No conflicts, last commit fixes a "conflic" because k8s 1.26 bump is not yet merged to 4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 27, 2023

@npinaeva: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-hypershift 8513b38 link false /test e2e-aws-ovn-hypershift
ci/prow/e2e-openstack-ovn 8513b38 link false /test e2e-openstack-ovn

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.