Add docs for enabling Istio native nftables feature#1122
Add docs for enabling Istio native nftables feature#1122istio-testing merged 18 commits intoistio-ecosystem:mainfrom
Conversation
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1122 +/- ##
==========================================
+ Coverage 77.78% 80.51% +2.72%
==========================================
Files 44 44
Lines 2823 2299 -524
==========================================
- Hits 2196 1851 -345
+ Misses 521 328 -193
- Partials 106 120 +14 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
We don't have a |
| @@ -0,0 +1,13 @@ | |||
| apiVersion: sailoperator.io/v1 | |||
There was a problem hiding this comment.
While this looks good. In the longer run, we should use templates and update it based on the args.
There was a problem hiding this comment.
I dont see this file used anywhere in the PR.
docs/common/istio-nftables.md
Outdated
|
|
||
| This feature configures Istio to use the `nftables` backend instead of `iptables` for traffic redirection. | ||
|
|
||
| #### Installation on OpenShift |
There was a problem hiding this comment.
Regarding the installation on OCP for Sail Operator, I feel we should link to the existing documentation rather than duplicating the steps here once again. If the steps require some explicit config that can be added as a note.
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
11eb9e9 to
79a1d1b
Compare
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
docs/common/istio-nftables.md
Outdated
|
|
||
| ### Installation | ||
|
|
||
| The support for native nftables when using Istio sidecar mode was implemented in the upstream istio release-1.27 [release note](https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml). It is disabled by default. To enable it, you can set a feature flag as `values.global.nativeNftables=true`. For example, |
There was a problem hiding this comment.
| The support for native nftables when using Istio sidecar mode was implemented in the upstream istio release-1.27 [release note](https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml). It is disabled by default. To enable it, you can set a feature flag as `values.global.nativeNftables=true`. For example, | |
| The support for native nftables when using Istio sidecar mode was implemented in the upstream istio [release-1.27](https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml). It is disabled by default. To enable it, you can set a feature flag as `values.global.nativeNftables=true`. For example, |
docs/common/istio-nftables.md
Outdated
| Installation with Helm | ||
|
|
||
| ```sh | ||
| helm install istiod-canary istio/istiod \ |
There was a problem hiding this comment.
Since we are talking about nftables, we can avoid using "istiod-canary"
| helm install istiod-canary istio/istiod \ | |
| helm install istiod istio/istiod \ |
docs/common/istio-nftables.md
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.28-alpha.24646157 |
There was a problem hiding this comment.
We can use 1.27 (or whatever is the latest supported version) here.
docs/common/istio-nftables.md
Outdated
| updateStrategy: | ||
| type: InPlace | ||
| inactiveRevisionDeletionGracePeriodSeconds: 30 |
There was a problem hiding this comment.
Since the updateStrategy section is anyways using the defaults, we can avoid specifying it in the yamls.
docs/common/istio-nftables.md
Outdated
| ```sh | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: operators.coreos.com/v1alpha1 | ||
| kind: Subscription | ||
| metadata: | ||
| name: sailoperator | ||
| namespace: openshift-operators | ||
| spec: | ||
| channel: "1.27-nightly" | ||
| installPlanApproval: Automatic | ||
| name: sailoperator | ||
| source: community-operators | ||
| sourceNamespace: openshift-marketplace | ||
| EOF | ||
| ``` |
There was a problem hiding this comment.
As this document is for Sail Operator, lets avoid any OpenShift specific notes. Please use appropriate mechanism for Kind cluster.
docs/common/istio-nftables.md
Outdated
| inactiveRevisionDeletionGracePeriodSeconds: 30 | ||
| values: | ||
| global: | ||
| nativeNftables: true |
There was a problem hiding this comment.
Actually you can remove this whole section since we are capturing the same info below.
docs/common/istio-nftables.md
Outdated
| ``` | ||
|
|
||
| #### Install in Ambient Mode | ||
|
|
There was a problem hiding this comment.
yes, the Ambient mode part install and upgrade is in progress. When that part is clear, I will update the upgrade task JIRA and this PR draft.
docs/common/istio-nftables.md
Outdated
| kubectl apply -n test-ns -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml | ||
| ``` | ||
|
|
||
| Attach a debug container and you will get the following rules in the `istio-proxy` container: |
There was a problem hiding this comment.
| Attach a debug container and you will get the following rules in the `istio-proxy` container: | |
| Attach a debug container and you can see the nftable rules from the `istio-proxy` container: |
docs/common/istio-nftables.md
Outdated
|
|
||
| ### Upgrade | ||
|
|
||
| The migration of using the existing Istio iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application curl in a data plane namespace test-ns. |
There was a problem hiding this comment.
| The migration of using the existing Istio iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application curl in a data plane namespace test-ns. | |
| The migration from iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application in test-ns namespace. |
docs/common/istio-nftables.md
Outdated
| The migration of using the existing Istio iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application curl in a data plane namespace test-ns. | ||
|
|
||
| ```sh | ||
| istioctl install -y |
There was a problem hiding this comment.
As this is a Sail Operator doc, lets use the installation mechanisms supported in Sail Operator.
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
|
This PR is still WIP. I tried the Sail Operator 1.27.1 but still getting the following error IstioCNI default violates policy 299 - "unknown field "spec.values.global.nativeNftables"" The api field PR was merged in #1183 |
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
|
Here are two findings : The doc PR is ready for review. Thanks. |
| updateStrategy: | ||
| type: InPlace | ||
| inactiveRevisionDeletionGracePeriodSeconds: 30 |
There was a problem hiding this comment.
Lets remove these fields as they are anyways default.
docs/common/istio-nftables.adoc
Outdated
| When you install IstioCNI and Istio resources with Sail Operator, you can create an instance of them with `+spec.values.global.nativeNftables=true+`. | ||
| This feature configures Istio to use the `+nftables+` backend instead of `+iptables+` for traffic redirection. |
There was a problem hiding this comment.
| When you install IstioCNI and Istio resources with Sail Operator, you can create an instance of them with `+spec.values.global.nativeNftables=true+`. | |
| This feature configures Istio to use the `+nftables+` backend instead of `+iptables+` for traffic redirection. | |
| When installing `Istio` and `IstioCNI` resources with the Sail Operator, you can enable nftables by setting `spec.values.global.nativeNftables=true` in the resource. This option configures Istio to use the nftables backend for traffic redirection instead of iptables. |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| To enable the native nftables feature, using the following steps: | ||
|
|
||
| [arabic] |
There was a problem hiding this comment.
Not sure what this is for. Can you eloborate?
There was a problem hiding this comment.
That was automatically generated when I run make gen-api-docs . I can remove that asciidoc format mark.
docs/common/istio-nftables.adoc
Outdated
| When you install IstioCNI and Istio resources with Sail Operator, you can create an instance of them with `+spec.values.global.nativeNftables=true+`. | ||
| This feature configures Istio to use the `+nftables+` backend instead of `+iptables+` for traffic redirection. | ||
|
|
||
| To enable the native nftables feature, using the following steps: |
There was a problem hiding this comment.
We can remove this line as we already mentioned above how to enable nftables.
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.27.2 |
There was a problem hiding this comment.
This should be updated to version 1.28
docs/common/istio-nftables.adoc
Outdated
| ==== Validation | ||
|
|
||
| When using the `+nftables+` backend, you can verify the traffic redirection rules using the `+nft list ruleset+` command in a data plane application or sidecar container. | ||
| You can find all rules are in the `+inet+` table. The following example installs a sample application `+curl+` in a data plane namespace `+test-ns+`. |
There was a problem hiding this comment.
| You can find all rules are in the `+inet+` table. The following example installs a sample application `+curl+` in a data plane namespace `+test-ns+`. | |
| You can find all rules are in the `+inet+` table. The following example installs a sample application `+curl+` in a namespace `+test-ns+` that is part of the mesh. |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| When using Sail Operator, the upgraded `IstioCNI` and `Istio` resources `spec.version` value need to be different from the prior instances. Otherwise, the appended `spec.values` will not be configured. | ||
|
|
||
| ===== Upgrade with Sail Operator on OpenShift |
There was a problem hiding this comment.
| ===== Upgrade with Sail Operator on OpenShift | |
| ===== Upgrade using Sail Operator |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| ===== Upgrade with Sail Operator on OpenShift | ||
|
|
||
| To upgrade an iptable based Istio service mesh, using the following steps: |
There was a problem hiding this comment.
| To upgrade an iptable based Istio service mesh, using the following steps: | |
| To upgrade an iptable based Istio service mesh with nftables backend, use the following steps: |
docs/common/istio-nftables.adoc
Outdated
| . Upgrade the IstioCNI and Istio control plane with `+spec.values.global.nativeNftables=true+`. | ||
| More details about the Update Strategy are described in link:../update-strategy/update-strategy.adoc[update-strategy.adoc]. For example, | ||
|
|
||
| [source,sh] |
There was a problem hiding this comment.
The whole of the section is redundant, we are just repeating things again. Please re-work on the section below. We dont have to show the Yamls again, just include a note saying that spec.values.global.nativeNftables should be set to true in Istio and IstioCNI CRs.
docs/common/istio-nftables.adoc
Outdated
|
|
||
| [source,sh] | ||
| ---- | ||
| kubectl rollout restart deployment -n test-ns |
There was a problem hiding this comment.
hmm... we should find a way to support re-programming the new rules without having to restart the pods. Did you take a look at the code to understand what it does when config changes and the rules do not match?
Are you saying that when we update any spec.values (without modifying the version) the updated values are not reflected in the Istio deployment? |
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
|
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
When I run the latest Sail operator 1.27.3+ , I see Istio values overrides behavior works correctly in this upgrade case. My Quote finding was not correct. It could be caused by running an old Istio version which had not included nftables change there. |
Still work in progress about the re-programming rule changes without having to restart the application pods. |
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
|
The Safe migration/upgrade change was created in the upstream PR: istio/istio#58354 I have upgraded this doc PR and it's ready for review. |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| === Installation | ||
|
|
||
| The support for native nftables in Istio sidecar mode was implemented in the upstream istio https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml[release-1.27]. |
There was a problem hiding this comment.
Probably we can link the official release announcement here - https://istio.io/latest/news/releases/1.27.x/announcing-1.27/
docs/common/istio-nftables.adoc
Outdated
|
|
||
| ==== Install in Ambient Mode | ||
|
|
||
| The support for native nftables in Istio ambient mode was implemented in the upstream istio `+release-1.28+`. |
There was a problem hiding this comment.
docs/common/istio-nftables.adoc
Outdated
| [source,sh] | ||
| ---- | ||
| cat <<EOF | kubectl apply -f- | ||
| apiVersion: sailoperator.io/v1alpha1 |
There was a problem hiding this comment.
While using v1alpha1 is not an issue, its better to change all the docs to use v1
docs/common/istio-nftables.adoc
Outdated
| When an existing Istio Ambient deployment using the iptables backend is upgraded to the nftables backend, IstioCNI doesn't switch to nftables silently. | ||
| The IstioCNI initialization code detects any iptables artifacts on the host first. If it finds them, it overrides the nftables setting, keeps using the iptables backend, and logs a message telling users to reboot the node to complete the migration. | ||
| After a reboot, the old iptables artifacts are cleared and pods come up with clean namespaces, completing the upgrade and migration automatically. |
There was a problem hiding this comment.
Lets move this to the ambient section.
docs/common/istio-nftables.adoc
Outdated
| . Check existing `+Istio+` and `+IstioCNI+` resources’ state are Healthy. | ||
|
|
||
| [start=2] | ||
| . Enable nftables by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the same example manifests in the Installation section above. |
There was a problem hiding this comment.
| . Enable nftables by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the same example manifests in the Installation section above. | |
| . Enable nftables by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the sample manifests in the Installation section above. |
docs/common/istio-nftables.adoc
Outdated
| . Enable nftables by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the same example manifests in the Installation section above. | ||
|
|
||
| [start=3] | ||
| . Update the data plane namespace `+test-ns+` by restarting all deployments. For example, |
There was a problem hiding this comment.
| . Update the data plane namespace `+test-ns+` by restarting all deployments. For example, | |
| . Update the data plane by restarting the application workloads that are part of the mesh. For example, |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| ==== Upgrade in Ambient mode | ||
|
|
||
| . Enable nftables with `+ambient+` profile by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the same example manifests in the Installation section above. |
There was a problem hiding this comment.
| . Enable nftables with `+ambient+` profile by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the same example manifests in the Installation section above. | |
| . Enable nftables with `+ambient+` profile by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the sample manifests in the Installation section above. |
docs/common/istio-nftables.adoc
Outdated
| . Enable nftables with `+ambient+` profile by setting `spec.values.global.nativeNftables=true` in the `+Istio+` and `+IstioCNI+` resources. You can find the same example manifests in the Installation section above. | ||
|
|
||
| [start=2] | ||
| . If you get the following IstioCNI log message like `To complete migration to nftables, reboot the node`, reboot each node. You don't need restarting application deployment pods because the Reconcile flag was enabled in Ambient mode IstioCNI. |
There was a problem hiding this comment.
| . If you get the following IstioCNI log message like `To complete migration to nftables, reboot the node`, reboot each node. You don't need restarting application deployment pods because the Reconcile flag was enabled in Ambient mode IstioCNI. | |
| . Restart the cluster nodes to complete the migration from iptables backend to nftables backend. |
|
Please look into the review comments. I've added "/hold" as the corresponding upstream PR is not yet merged. |
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
|
Ok, I have updated all review updates and will hold this doc PR. |
| @@ -0,0 +1,13 @@ | |||
| apiVersion: sailoperator.io/v1 | |||
There was a problem hiding this comment.
I dont see this file used anywhere in the PR.
| @@ -0,0 +1,10 @@ | |||
| apiVersion: sailoperator.io/v1 | |||
ed3b33b to
c48d831
Compare
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
fjglira
left a comment
There was a problem hiding this comment.
Adding variable to be replaced in the rendered view. I think is safe to replace all the hardcoded Istio version with version: {istio_latest_tag} because currently it points to 1.28 and is already mentioned in the doc that the support for nft table start on 1,27 and the support for nft tables in Ambient starts in 1.28
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.28-latest |
There was a problem hiding this comment.
| version: v1.28-latest | |
| version: {istio_latest_tag} |
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.28-latest |
There was a problem hiding this comment.
| version: v1.28-latest | |
| version: {istio_latest_tag} |
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.28-latest |
There was a problem hiding this comment.
| version: v1.28-latest | |
| version: {istio_latest_tag} |
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.27-latest |
There was a problem hiding this comment.
| version: v1.27-latest | |
| version: {istio_latest_tag} |
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.27-latest |
There was a problem hiding this comment.
| version: v1.27-latest | |
| vversion: {istio_latest_tag} |
Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com>
sridhargaddam
left a comment
There was a problem hiding this comment.
Upstream PR is now merged.
|
/test e2e-kind-olm |
* upstream/main: Add docs for enabling Istio native nftables feature (istio-ecosystem#1122)
…1122) * update docs: add terminology Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * Add docs for istio nftables Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * Add installation steps on OpenShift Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * Update Istio nftables doc Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * update doc Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * update docs Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * add converted ascii istio-nftables.adoc Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * remove markdown doc file Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * update docs: fix typo and commit check Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * update docs Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * review updates Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * update istio-nftables doc Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * doc: update nftables doc upgrade part (istio-ecosystem#1122) Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * review updates Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * remove unused sample files Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> * update version tag Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> --------- Signed-off-by: Yuanlin Xu <yuanlin.xu@redhat.com> Signed-off-by: Daniel Grimm <dgrimm@redhat.com>
This PR adds new doc and samples about the Istio native nftables feature. That feature was merged in the upstream release-1.27. This doc outlines the configuration steps using Sail Operator.
Related issue: #1123