Skip to content

Bump to OSSM 3.0.3 and Istio 1.24.4#1239

Closed
Miciah wants to merge 6 commits intoopenshift:masterfrom
Miciah:NE-2022-bump-to-OSSM-3.0.3
Closed

Bump to OSSM 3.0.3 and Istio 1.24.4#1239
Miciah wants to merge 6 commits intoopenshift:masterfrom
Miciah:NE-2022-bump-to-OSSM-3.0.3

Conversation

@Miciah
Copy link
Contributor

@Miciah Miciah commented Jul 1, 2025

Bump to OSSM 3.0.3 and Istio 1.24.4

Bump from OSSM v3.0.0 to v3.0.3 and from Istio v1.24.3 to v1.24.4.

Enable Gateway only CA Bundles and custom CA CM name

Avoid conflict with a user control plane by setting a custom CA Bundle CM name for the Gateway Control plane and enable Istio to only inject CA Bundle CMs in namespaces where Gateways exist to avoid polluting the whole cluster.

Two new environment variables are set for the Istio control plane deployment CR:

PILOT_CA_CERT_CONFIGMAP
PILOT_ENABLE_GATEWAY_API_CA_CERT_ONLY

This change is related to OSSM-9076.

This change incorporates #1209.

Don't copy labels or annotations

Configure Istiod not to copy annotations or labels from gateways onto associated resources, such as the proxy deployment and load-balancer service for a gateway.

This copying behavior is Istio-specific, not part of the Gateway API spec, and could be used to inject unsupported configuration. For example, an end-user could set a service annotation on the gateway in order to configure a load-balancer. Setting annotations on the gateway to configure the load-balancer would not be portable to other Gateway API implementations and would complicate product support.

One new environment variable is set:

PILOT_ENABLE_GATEWAY_API_COPY_LABELS_ANNOTATIONS

This change is related to OSSM-8989.

Delete old controller-mode setting

Delete the obsolete PILOT_ENABLE_GATEWAY_CONTROLLER_MODE environment variable from the Istiod configuration. This environment variable is no longer recognized in OSSM 3, and the variable has been superseded by EnhancedResourceScoping.

assertDNSRecord: Increase timeout to 10m

Increase the timeout in assertDNSRecord for polling for the DNSRecord CR from 1 minute to 10 minutes.

The cloud provider can easily take over a minute to provision the load balancer, and the operator cannot create the DNSRecord CR before the load balancer has been provisioned and assigned a host name or address. Consequently, the polling loop could easily reach the 1-minute timeout just on account of the time that it takes to provision the load balancer.

testGatewayAPIManualDeployment: Increase timeout

Increase the timeout for polling the gateway, and dump the gateway if the test fails.

Miciah and others added 6 commits July 1, 2025 10:03
This commit resolves NE-2022.

https://issues.redhat.com/browse/NE-2022

* cmd/ingress-operator/start.go (defaultGatewayAPIOperatorVersion):
* manifests/02-deployment-ibm-cloud-managed.yaml
(GATEWAY_API_OPERATOR_VERSION):
* manifests/02-deployment.yaml
(GATEWAY_API_OPERATOR_VERSION): Bump from OSSM v3.0.0 to v3.0.3.
* pkg/operator/controller/gatewayclass/istio.go (desiredIstio): Bump
from Istio v1.24.3 to v1.24.4.
Avoid conflict with a user control plane by setting a custom CA Bundle CM name
for the Gateway Control plane and enable istio to only inject CA Bundle CMs in
namespaces where Gateways exist to avoid poluting the whole cluster.

Two new Env variables set for the Istio control plane deployment CR;
  PILOT_CA_CERT_CONFIGMAP
  PILOT_ENABLE_GATEWAY_API_CA_CERT_ONLY

Related to OSSM-9076
Configure Istiod not to copy annotations or labels from gateways onto
associated resources, such as the proxy deployment and load-balancer
service for a gateway.

This copying behavior is Istio-specific, not part of the Gateway API
spec, and could be used to inject unsupported configuration.  For
example, an end-user could set a service annotation on the gateway in
order to configure a load-balancer.  Setting annotations on the gateway
to configure the load-balancer would not be portable to other Gateway
API implementations and would complicate product support.

This commit is related to OSSM-8989.

https://issues.redhat.com/browse/OSSM-8989

* pkg/operator/controller/gatewayclass/istio.go (desiredIstio): Set the
"PILOT_ENABLE_GATEWAY_API_COPY_LABELS_ANNOTATIONS" to "false".
Delete the obsolete PILOT_ENABLE_GATEWAY_CONTROLLER_MODE environment
variable from the Istiod configuration.  This environment variable is
no longer recognized in OSSM 3, and the variable has been superseded
by EnhancedResourceScoping.

* pkg/operator/controller/gatewayclass/istio.go (desiredIstio): Delete
PILOT_ENABLE_GATEWAY_CONTROLLER_MODE.
Increase the timeout in assertDNSRecord for polling for the DNSRecord CR
from 1 minute to 10 minutes.

The cloud provider can easily take over a minute to provision the load balancer,
and the operator cannot create the DNSRecord CR before the load balancer has
been provisioned and assigned a host name or address.  Consequently, the polling
loop could easily reach the 1-minute timeout just on account of the time that it
takes to provision the load balancer.

* test/e2e/util_gatewayapi_test.go (assertDNSRecord): Increase timeout
for the DNSRecord CR polling loop from 1m to 10m.
Increase the timeout for polling the gateway, and dump the gateway if
the test fails.

* test/e2e/gateway_api_test.go (testGatewayAPIManualDeployment):
Increase the timeout for polling the gateway from 1m to 5m.  Dump the
gateway if the test fails.
@openshift-ci openshift-ci bot requested review from alebedev87 and knobunc July 1, 2025 14:07
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 1, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign rfredette for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 1, 2025

@Miciah: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-hypershift 0928412 link true /test e2e-hypershift
ci/prow/e2e-azure-operator 0928412 link true /test e2e-azure-operator
ci/prow/e2e-aws-operator 0928412 link true /test e2e-aws-operator
ci/prow/e2e-gcp-operator 0928412 link true /test e2e-gcp-operator
ci/prow/e2e-aws-ovn-single-node 0928412 link false /test e2e-aws-ovn-single-node
ci/prow/e2e-aws-ovn-hypershift-conformance 0928412 link true /test e2e-aws-ovn-hypershift-conformance
ci/prow/e2e-aws-ovn 0928412 link true /test e2e-aws-ovn
ci/prow/e2e-gcp-ovn 0928412 link false /test e2e-gcp-ovn
ci/prow/e2e-aws-operator-techpreview 0928412 link false /test e2e-aws-operator-techpreview
ci/prow/e2e-azure-ovn 0928412 link false /test e2e-azure-ovn
ci/prow/e2e-aws-ovn-techpreview 0928412 link false /test e2e-aws-ovn-techpreview

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@lihongan
Copy link
Contributor

Did pre-merge test with the PR but seems gateway Programmed status is false and pod complains configmap "istio-ca-root-cert" not found. Am I missing something ?

$ oc -n openshift-ingress get gateway
NAME    CLASS               ADDRESS   PROGRAMMED   AGE
gwapi   openshift-default             False        47m

$ oc -n openshift-ingress get pod
NAME                                       READY   STATUS              RESTARTS       AGE
gwapi-openshift-default-6d5c47849c-bkf29   0/1     ContainerCreating   0              47m

$ oc -n openshift-ingress describe pod gwapi-openshift-default-6d5c47849c-bkf29
<......>
Warning  FailedMount  77s (x31 over 48m)  kubelet            MountVolume.SetUp failed for volume "istiod-ca-cert" : configmap "istio-ca-root-cert" not found

And OSSM and istio looks good

$ oc -n openshift-operators get csv
NAME                          DISPLAY                            VERSION   REPLACES                      PHASE
servicemeshoperator3.v3.0.3   Red Hat OpenShift Service Mesh 3   3.0.3     servicemeshoperator3.v3.0.2   Succeeded

$ oc get istio
NAME                REVISIONS   READY   IN USE   ACTIVE REVISION     STATUS    VERSION   AGE
openshift-gateway   1           1       1        openshift-gateway   Healthy   v1.24.4   99m

$ oc get istio openshift-gateway -oyaml | yq .spec.values.pilot
cni:
  enabled: false
enabled: true
env:
  PILOT_CA_CERT_CONFIGMAP: openshift-gw-ca-root-cert
  PILOT_ENABLE_ALPHA_GATEWAY_API: "false"
  PILOT_ENABLE_GATEWAY_API: "true"
  PILOT_ENABLE_GATEWAY_API_CA_CERT_ONLY: "true"
  PILOT_ENABLE_GATEWAY_API_COPY_LABELS_ANNOTATIONS: "false"
  PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER: "true"
  PILOT_ENABLE_GATEWAY_API_GATEWAYCLASS_CONTROLLER: "false"
  PILOT_ENABLE_GATEWAY_API_STATUS: "true"
  PILOT_GATEWAY_API_CONTROLLER_NAME: openshift.io/gateway-controller/v1
  PILOT_GATEWAY_API_DEFAULT_GATEWAYCLASS_NAME: openshift-default
  PILOT_MULTI_NETWORK_DISCOVER_GATEWAY_API: "false"
podAnnotations:
  target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'

@lihongan
Copy link
Contributor

And connot find configmap istio-ca-root-cert in ns openshift-ingress

$ oc -n openshift-ingress get configmap
NAME                                       DATA   AGE
istio-gateway-status-leader                0      101m
istio-openshift-gateway                    2      101m
istio-sidecar-injector-openshift-gateway   2      101m
kube-root-ca.crt                           1      153m
openshift-gw-ca-root-cert                  1      53m
openshift-service-ca.crt                   1      153m
service-ca-bundle                          1      153m

@Miciah
Copy link
Contributor Author

Miciah commented Jul 16, 2025

Did pre-merge test with the PR but seems gateway Programmed status is false and pod complains configmap "istio-ca-root-cert" not found. Am I missing something ?

I was using this PR to explore the errors I was observing on #1227. I believe that the issue you are describing has been resolved in #1227; quoting #1227 (comment):

https://github.com/openshift/cluster-ingress-operator/compare/d3257317c7c0c8e14c8c0704b2a40bd81f95b527..ece76fddf9ce97b985fddc36aafcc73197c28f17 removes PILOT_CA_CERT_CONFIGMAP on @dgn's suggestion (setting trustBundleName should be sufficient).

#1227 is passing the testGatewayAPIObjects and testGatewayAPIManualDeployment tests now, so this testing PR is no longer necessary.

/close

@openshift-ci openshift-ci bot closed this Jul 16, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 16, 2025

@Miciah: Closed this PR.

Details

In response to this:

Did pre-merge test with the PR but seems gateway Programmed status is false and pod complains configmap "istio-ca-root-cert" not found. Am I missing something ?

I was using this PR to explore the errors I was observing on #1227. I believe that the issue you are describing has been resolved in #1227; quoting #1227 (comment):

https://github.com/openshift/cluster-ingress-operator/compare/d3257317c7c0c8e14c8c0704b2a40bd81f95b527..ece76fddf9ce97b985fddc36aafcc73197c28f17 removes PILOT_CA_CERT_CONFIGMAP on @dgn's suggestion (setting trustBundleName should be sufficient).

#1227 is passing the testGatewayAPIObjects and testGatewayAPIManualDeployment tests now, so this testing PR is no longer necessary.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments