OCPBUGS-66980: Update NetworkPolicy egress to support HyperShift custom API ports#723
Conversation
|
@rashmigottipati: This pull request references Jira Issue OCPBUGS-66980, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Removing all egress restrictions... |
|
/jira refresh |
|
@rashmigottipati: This pull request references Jira Issue OCPBUGS-66980, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
Requesting review from QA contact: DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/lgtm |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: tmshort The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/test e2e-gcp |
|
Hi @kuiwang02 , could you help verify it? Thanks! |
|
/assign @kuiwang02 |
|
/retest-required |
|
@rashmigottipati @grokspawn (it seem you are invovled in discussion per jira ticket) 1, acutally the fix will not only impact hypershift hosted cluster, but also impact non-hypershift cluster. is it expected? here is the result: for non-hypershift cluster: kuiwang@kuiwang-mac kube-apiserver % oc get node
NAME STATUS ROLES AGE VERSION
ip-10-0-0-49.us-east-2.compute.internal Ready control-plane,master 158m v1.34.2
ip-10-0-39-239.us-east-2.compute.internal Ready control-plane,master 158m v1.34.2
ip-10-0-4-192.us-east-2.compute.internal Ready worker 141m v1.34.2
ip-10-0-54-255.us-east-2.compute.internal Ready worker 151m v1.34.2
ip-10-0-68-194.us-east-2.compute.internal Ready control-plane,master 158m v1.34.2
ip-10-0-77-176.us-east-2.compute.internal Ready worker 151m v1.34.2
kuiwang@kuiwang-mac kube-apiserver % oc get -n openshift-marketplace networkpolicy marketplace-operator -o yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
annotations:
capability.openshift.io/name: marketplace
include.release.openshift.io/hypershift: "true"
include.release.openshift.io/ibm-cloud-managed: "true"
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
creationTimestamp: "2026-02-05T03:17:18Z"
generation: 1
name: marketplace-operator
namespace: openshift-marketplace
ownerReferences:
- apiVersion: config.openshift.io/v1
controller: true
kind: ClusterVersion
name: version
uid: a40ebddd-1fb3-4ce3-8ef2-910f94b8a791
resourceVersion: "11145"
uid: 0f0ed9ea-48b0-4f99-903a-24cccf8c318d
spec:
egress:
- {}
ingress:
- ports:
- port: 8081
protocol: TCP
podSelector:
matchLabels:
name: marketplace-operator
policyTypes:
- Ingress
- Egress
kuiwang@kuiwang-mac kube-apiserver % oc get -n openshift-marketplace networkpolicy unpack-bundles -o yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
annotations:
capability.openshift.io/name: marketplace
include.release.openshift.io/hypershift: "true"
include.release.openshift.io/ibm-cloud-managed: "true"
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
creationTimestamp: "2026-02-05T03:17:18Z"
generation: 1
name: unpack-bundles
namespace: openshift-marketplace
ownerReferences:
- apiVersion: config.openshift.io/v1
controller: true
kind: ClusterVersion
name: version
uid: a40ebddd-1fb3-4ce3-8ef2-910f94b8a791
resourceVersion: "11170"
uid: 1997fdee-e503-43b3-b9ac-d7f2c1945d7a
spec:
egress:
- {}
podSelector:
matchExpressions:
- key: operatorframework.io/bundle-unpack-ref
operator: Exists
- key: olm.managed
operator: In
values:
- "true"
policyTypes:
- Ingress
- Egressfor hypershift hosted cluster: kuiwang@kuiwang-mac kube-apiserver % oc get node
NAME STATUS ROLES AGE VERSION
ip-10-0-138-34.us-east-2.compute.internal Ready worker 129m v1.34.2
ip-10-0-145-214.us-east-2.compute.internal Ready worker 129m v1.34.2
ip-10-0-165-232.us-east-2.compute.internal Ready worker 128m v1.34.2
kuiwang@kuiwang-mac kube-apiserver % oc get -n openshift-marketplace networkpolicy marketplace-operator -o yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
annotations:
capability.openshift.io/name: marketplace
include.release.openshift.io/hypershift: "true"
include.release.openshift.io/ibm-cloud-managed: "true"
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
creationTimestamp: "2026-02-05T04:00:10Z"
generation: 1
name: marketplace-operator
namespace: openshift-marketplace
ownerReferences:
- apiVersion: config.openshift.io/v1
controller: true
kind: ClusterVersion
name: version
uid: 14520b32-26d6-4b1c-bcc5-03e0a7e73bfd
resourceVersion: "1352"
uid: b4724ea9-b83c-4da5-935d-8134f20c2fd6
spec:
egress:
- {}
ingress:
- ports:
- port: 8081
protocol: TCP
podSelector:
matchLabels:
name: marketplace-operator
policyTypes:
- Ingress
- Egress
kuiwang@kuiwang-mac kube-apiserver % oc get -n openshift-marketplace networkpolicy unpack-bundles -o yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
annotations:
capability.openshift.io/name: marketplace
include.release.openshift.io/hypershift: "true"
include.release.openshift.io/ibm-cloud-managed: "true"
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
creationTimestamp: "2026-02-05T04:00:11Z"
generation: 1
name: unpack-bundles
namespace: openshift-marketplace
ownerReferences:
- apiVersion: config.openshift.io/v1
controller: true
kind: ClusterVersion
name: version
uid: 14520b32-26d6-4b1c-bcc5-03e0a7e73bfd
resourceVersion: "1396"
uid: 78bfe26c-716e-4a17-ae6a-332656163712
spec:
egress:
- {}
podSelector:
matchExpressions:
- key: operatorframework.io/bundle-unpack-ref
operator: Exists
- key: olm.managed
operator: In
values:
- "true"
policyTypes:
- Ingress
- Egress2, for hypershift hosted cluster which apiserver port is not 6443, the fix can only resolve the catalogsource in openshift-marketplace ns. it means if the customer create custom catalogsource in other ns (not in openshift-marketplace ns). it still restrict the port as 6443 because dynmiacally catalogsource code for network policy does not change. (FYI code is operator-framework/operator-lifecycle-manager#3568), so the customer still can not install operator from custom catalogsource which is not in openshift-marketplace ns on hypershift hosted cluster. do we need to change it or we restrict the customer not to create catalogsource in other ns on hypershif-hosted cluster which apiserver is not 6443? here is result kuiwang@kuiwang-mac kube-apiserver % oc create ns test
namespace/test created
kuiwang@kuiwang-mac kube-apiserver % cat /tmp/catsrctest.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: test
namespace: test
spec:
displayName: test operators
grpcPodConfig:
extractContent:
cacheDir: /tmp/cache
catalogDir: /configs
memoryTarget: 30Mi
image: quay.io/openshift-qe-optional-operators/aosqe-index:v4.22
publisher: OpenShift QE
sourceType: grpc
kuiwang@kuiwang-mac kube-apiserver % oc apply -f /tmp/catsrctest.yaml
catalogsource.operators.coreos.com/test created
kuiwang@kuiwang-mac kube-apiserver % oc -n test get networkpolicy
NAME POD-SELECTOR AGE
test-grpc-server olm.catalogSource=test,olm.managed=true 44s
test-unpack-bundles olm.managed in (true),operatorframework.io/bundle-unpack-ref 44s
kuiwang@kuiwang-mac kube-apiserver % oc -n test get networkpolicy test-unpack-bundles -o yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
creationTimestamp: "2026-02-05T06:25:00Z"
generation: 1
labels:
olm.catalogSource: test
olm.managed: "true"
name: test-unpack-bundles
namespace: test
ownerReferences:
- apiVersion: operators.coreos.com/v1alpha1
blockOwnerDeletion: false
controller: false
kind: CatalogSource
name: test
uid: 30650d77-863b-482f-ac43-3d0ed29a5ec1
resourceVersion: "49125"
uid: 90b938e3-8147-4734-9a0f-78ccbd9e5ca1
spec:
egress:
- ports:
- port: 6443
protocol: TCP
podSelector:
matchExpressions:
- key: operatorframework.io/bundle-unpack-ref
operator: Exists
- key: olm.managed
operator: In
values:
- "true"
policyTypes:
- Ingress
- Egress |
|
@rashmigottipati: This pull request references Jira Issue OCPBUGS-66980, which is valid. 3 validation(s) were run on this bug
Requesting review from QA contact: DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/test e2e-aws-ovn-serial |
|
@kuiwang02 great comments!
yes, this is intentional. Guidance from the networking team is that we use wildcard egress for kube-apiserver across the board. The updated guidance has been to change everything that talks to kube-apiserver "use wildcard allow all IPs/Ports" and for DNS, "use wildcard allow all IPs with ports 53 and 5353". I have updated this PR accordingly. PTAL. Let me know if you have any further questions. Regarding your 2nd comment, I've opened PR in Both functions now use wildcard egress {} for kube-apiserver instead of hardcoded port 6443. This means that when customers create custom CatalogSources in any namespace (not just openshift-marketplace), the dynamically generated NetworkPolicies will support hypershift's custom API ports. Let me know if this addresses your concerns. Thanks for helping verify this PR! |
|
/retest-required |
|
/test e2e-aws-ovn-serial |
|
/test e2e-aws-ovn-serial |
Signed-off-by: Rashmi Gottipati <rgottipa@redhat.com>
Signed-off-by: Rashmi Gottipati <rgottipa@redhat.com>
adafd48 to
b8bfcb5
Compare
|
/hold I will check it again per new change. |
|
/retest-required |
|
@rashmigottipati - kube-apiserver: allow all IPs/Ports
- DNS: allow all IPs with ports 53 and 5353 and the PR #723 and openshift/operator-framework-olm#1237 define the policy as egress:
- {}
- ports:
- port: 53
protocol: TCP
- port: 53
protocol: UDP
- port: 5353
protocol: TCP
- port: 5353
protocol: UDPActually it doesn't match the requirement precisely because NetworkPolicy egress rules use OR logic - if any rule matches, traffic is allowed. So, it does not match the expected. |
I already mentioned this in: operator-framework/operator-lifecycle-manager#3770 (comment) |
|
/retest |
rashmigottipati
left a comment
There was a problem hiding this comment.
@kuiwang02
You're correct that it's technically redundant: the wildcard {} already allows all traffic including DNS.
However, per guidance from the networking team, we're intentionally keeping both rules for documentation purposes:
- Wildcard egress: for kube-apiserver
- DNS rules (53, 5353): for DNS
The reasoning is that "if we're ever able to tighten apiserver rules or implement it some other way, we'll remember that DNS is separate."
This approach is consistent across:
- operator-marketplace (this PR)
- operator-lifecycle-manager: operator-framework/operator-lifecycle-manager#3770
Hope this clarifies your concerns. Could you please verify this PR and add the necessary labels if everything looks good to you? Thanks.
|
/hold cancel |
|
/lgtm |
|
/verified by @kuiwang02 |
|
@kuiwang02: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
8ab43bb
into
operator-framework:master
|
@rashmigottipati: Jira Issue Verification Checks: Jira Issue OCPBUGS-66980 Jira Issue OCPBUGS-66980 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/cherry-pick release-4.21 |
|
@rashmigottipati: new pull request created: #729 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/cherry-pick release-4.20 |
|
@rashmigottipati: new pull request created: #730 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Fix included in accepted release 4.22.0-0.nightly-2026-03-02-153725 |
Description of the change:
Replace hardcoded port 6443 with wildcard egress in NetworkPolicy manifests:
manifests/14_networkpolicy_marketplace-operator.yamlmanifests/15_networkpolicy_unpack-bundles.yamlThis allows egress to all hosts and ports, matching the pattern used by catalogd and operator-controller.
Motivation for the change:
The current NetworkPolicy rules hardcode port 6443 for Kubernetes API Server access, which breaks HyperShift deployments that customize the API server port via
hostedcluster.spec.networking.apiServer.port. The updated guidance recommends usingegress: [{}]for API server access rather than hardcoding specific ports. This PR aligns with the guidance.Fixes: OCPBUGS-66980
Reviewer Checklist
/docs