Add containerd runtime support for antrea agent on windows #4279

XinShuYang · 2022-10-11T07:32:56Z

Add new containerd yaml file for windows agent because of the different cni workflow.

Signed-off-by: Shuyang Xin [email protected]

codecov · 2022-10-11T07:50:39Z

Codecov Report

Merging #4279 (3d08fa2) into main (330a7de) will increase coverage by 2.80%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4279      +/-   ##
==========================================
+ Coverage   64.01%   66.81%   +2.80%     
==========================================
  Files         400      403       +3     
  Lines       56888    57690     +802     
==========================================
+ Hits        36418    38547    +2129     
+ Misses      17783    16401    -1382     
- Partials     2687     2742      +55

Flag	Coverage Δ
integration-tests	`34.63% <ø> (+0.04%)`	⬆️
kind-e2e-tests	`41.07% <ø> (-0.30%)`	⬇️
unit-tests	`56.71% <ø> (+6.84%)`	⬆️

Impacted Files	Coverage Δ
...g/agent/apiserver/handlers/addressgroup/handler.go	`5.00% <0.00%> (-35.00%)`	⬇️
...agent/apiserver/handlers/appliedtogroup/handler.go	`5.00% <0.00%> (-35.00%)`	⬇️
...nt/apiserver/handlers/serviceexternalip/handler.go	`3.70% <0.00%> (-25.93%)`	⬇️
...uster/commonarea/labelidentityimport_controller.go	`54.32% <0.00%> (-20.99%)`	⬇️
...agent/flowexporter/connections/deny_connections.go	`66.66% <0.00%> (-18.28%)`	⬇️
pkg/agent/apiserver/handlers/multicast/handler.go	`69.76% <0.00%> (-9.31%)`	⬇️
pkg/agent/flowexporter/connections/connections.go	`82.22% <0.00%> (-5.56%)`	⬇️
.../flowexporter/connections/conntrack_connections.go	`72.85% <0.00%> (-4.29%)`	⬇️
pkg/controller/ipam/antrea_ipam_controller.go	`75.25% <0.00%> (-4.02%)`	⬇️
pkg/agent/util/iptables/iptables.go	`45.07% <0.00%> (-3.63%)`	⬇️
... and 78 more

XinShuYang · 2022-10-17T15:37:26Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-17T15:46:36Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T05:20:46Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T05:37:47Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T06:00:25Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T06:42:51Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T11:16:11Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T11:45:00Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T12:56:48Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T13:31:54Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T14:15:07Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T15:25:36Z

/test-windows-containerd-e2e

XinShuYang · 2022-10-18T23:33:56Z

/test-windows-containerd-e2e

wenyingd · 2022-12-09T02:08:32Z

@XinShuYang @wenyingd should this be included in milestone 1.10?

Yes, it is better if we can include it.

luolanzone

LGTM

XinShuYang · 2022-12-15T13:15:51Z

/test-windows-containerd-e2e
/test-windows-containerd-networkpolicy
/test-windows-containerd-conformance
/test-windows-all

tnqn

LGTM overall, some minor comments

build/yamls/windows/base/conf/Install-WindowsCNI-Containerd.ps1

ci/jenkins/test.sh

tnqn · 2022-12-15T13:46:18Z

ci/jenkins/test.sh

+        govc snapshot.revert -vm ${WORKER_NAME} win-initial
+        # If Windows VM fails to power on correctly in time, retry several times.
+        winVMIPs=""
+        for i in `seq 10`; do
+            winVMIPs=$(govc vm.ip -wait=2m -a ${WORKER_NAME})
+            if [[ $winVMIPs != "" ]]; then
+                echo "Windows VM ${WORKER_NAME} powered on"
+                break
+            fi
+            echo "Windows VM ${WORKER_NAME} failed to power on"
+            govc vm.power -on ${WORKER_NAME} || true
+        done
+        if [[ $winVMIPs == "" ]]; then
+            echo "Windows VM ${WORKER_NAME} didn't power on after 3 tries, exiting"
+            exit 1
+        fi
+        IP=$(kubectl get node "${WORKER_NAME}" -o jsonpath='{.status.addresses[0].address}')
+        # Windows VM is reverted to an old snapshot so computer date needs updating.
+        for i in `seq 24`; do
+            sleep 5
+            ssh -o StrictHostKeyChecking=no -n Administrator@${IP} "W32tm /resync /force" | grep successfully && break
+        done


this seems same as the code for jumper VM, perhaps extract a function to reduce redundancy?

exacted a function revert_snapshot_windows

tnqn · 2022-12-15T13:52:17Z

ci/jenkins/test.sh

+            # Compress antrea repo and copy it to a Windows node
+            mkdir -p jenkins
+            tar --exclude='./jenkins' -czf jenkins/antrea_repo.tar.gz -C "$(pwd)" .
+            for i in `seq 2`; do
+                timeout 2m scp -o StrictHostKeyChecking=no -T jenkins/antrea_repo.tar.gz Administrator@${IP}: && break
+            done
+            echo "=== Build Windows on Windows Node==="
+            ssh -o StrictHostKeyChecking=no -n Administrator@${IP} "docker pull ${DOCKER_REGISTRY}/antrea/golang:${GO_VERSION}-nanoserver && docker tag ${DOCKER_REGISTRY}/antrea/golang:${GO_VERSION}-nanoserver golang:${GO_VERSION}-nanoserver"
+            ssh -o StrictHostKeyChecking=no -n Administrator@${IP} "rm -rf antrea && mkdir antrea && cd antrea && tar -xzf ../antrea_repo.tar.gz > /dev/null && NO_PULL=${NO_PULL}; DOCKER_NETWORK=host make build-windows && docker save -o antrea-windows.tar antrea/antrea-windows:latest" || true
+            for i in `seq 2`; do
+                timeout 2m scp -o StrictHostKeyChecking=no -T Administrator@${IP}:antrea/antrea-windows.tar.gz . && break
+            done
+        else
+            for i in `seq 2`; do
+                timeout 2m scp -o StrictHostKeyChecking=no -T antrea-windows.tar.gz Administrator@${IP}: && break
+            done
+            ssh -o StrictHostKeyChecking=no -n Administrator@${IP} "gzip -d antrea-windows.tar.gz && ctr -n k8s.io images import antrea-windows.tar"
+        fi


Haven't it been executed once after L522?

Removed. If building fails on jumper node, the test will exit.

tnqn · 2022-12-15T13:55:58Z

ci/jenkins/test.sh

+        ginkgo -timeout=2h --no-color $E2ETEST_PATH -- --provider=skeleton --ginkgo.focus="$WINDOWS_NETWORKPOLICY_FOCUS" --ginkgo.skip="$WINDOWS_NETWORKPOLICY_CONTAINERD_SKIP" > windows_conformance_result_no_color.txt || true
+    else
+        ginkgo --no-color $E2ETEST_PATH -- --provider=skeleton --node-os-distro=windows --ginkgo.focus="$WINDOWS_CONFORMANCE_FOCUS" --ginkgo.skip="$WINDOWS_CONFORMANCE_SKIP" > windows_conformance_result_no_color.txt || true


Doesn't windows-containerd-networkpolicy need to specify --node-os-distro=windows?

We can not specify --node-os-distro=windows in windows-containerd-networkpolicy or windows-networkpolicy, it is because most of the network policy test cases we expect to run in kubernetes test suite have the mark "[LinuxOnly]", and the parameter will lead the cases are passed. Although kubernetes mark these case to run on Linux only, they are supposed to work on Windows since Antrea Windows supports NP functions.

better to add a comment for above explanation.

Added comments.

XinShuYang · 2022-12-16T02:09:03Z

/test-windows-containerd-e2e
/test-windows-containerd-networkpolicy
/test-windows-containerd-conformance
/test-windows-all

XinShuYang · 2022-12-16T08:27:01Z

/test-windows-e2e
/test-windows-conformance
/test-windows-containerd-networkpolicy

XinShuYang · 2022-12-16T08:27:31Z

/test-windows-proxyall-e2e

XinShuYang · 2022-12-16T08:27:39Z

/test-all

XinShuYang · 2022-12-17T07:37:22Z

/test-windows-conformance
/test-windows-containerd-networkpolicy
/test-windows-proxyall-e2e

tnqn · 2022-12-18T16:39:54Z

ci/jenkins/test.sh

+        ginkgo -timeout=2h --no-color $E2ETEST_PATH -- --provider=skeleton --ginkgo.focus="$WINDOWS_NETWORKPOLICY_FOCUS" --ginkgo.skip="$WINDOWS_NETWORKPOLICY_CONTAINERD_SKIP" > windows_conformance_result_no_color.txt || true
+    else
+        ginkgo --no-color $E2ETEST_PATH -- --provider=skeleton --node-os-distro=windows --ginkgo.focus="$WINDOWS_CONFORMANCE_FOCUS" --ginkgo.skip="$WINDOWS_CONFORMANCE_SKIP" > windows_conformance_result_no_color.txt || true


better to add a comment for above explanation.

ci/jenkins/test.sh

XinShuYang · 2022-12-19T15:59:07Z

/test-windows-containerd-e2e
/test-windows-containerd-networkpolicy
/test-windows-containerd-conformance
/test-windows-all
/test-all

tnqn

LGTM

XinShuYang · 2022-12-19T17:01:08Z

/test-windows-containerd-e2e
/test-windows-containerd-networkpolicy
/test-windows-containerd-conformance
/test-windows-all
/test-all

Add e2e,conformance and networkpolicy tests for windows containerd testbed. ginkgo:v2.1.6 kubernetes:v1.25 containerd:v1.6.6 Signed-off-by: Shuyang Xin <[email protected]>

XinShuYang · 2022-12-20T03:32:49Z

/test-windows-containerd-e2e
/test-windows-containerd-networkpolicy
/test-windows-containerd-conformance
/test-windows-all
/test-all

tnqn

LGTM

…#4279) Add new containerd yaml file for windows agent because of the different cni workflow. Add e2e,conformance and networkpolicy tests for windows containerd testbed. Signed-off-by: Shuyang Xin <[email protected]>

XinShuYang force-pushed the wincontainerd branch from 6adf201 to 4eb3c2c Compare October 13, 2022 01:10

XinShuYang changed the title ~~Support containerd runtime support for antrea agent on windows~~ Add containerd runtime support for antrea agent on windows Oct 13, 2022

XinShuYang force-pushed the wincontainerd branch from 4eb3c2c to d834c78 Compare October 17, 2022 15:30

XinShuYang marked this pull request as ready for review October 17, 2022 15:34

XinShuYang force-pushed the wincontainerd branch 2 times, most recently from 365645b to e7bddbe Compare October 18, 2022 05:19

XinShuYang force-pushed the wincontainerd branch from e7bddbe to 1d1cb7d Compare October 18, 2022 06:20

XinShuYang force-pushed the wincontainerd branch 2 times, most recently from 80bc54b to 30330bb Compare October 18, 2022 11:10

XinShuYang force-pushed the wincontainerd branch from 30330bb to 6eb00d6 Compare October 18, 2022 11:42

XinShuYang force-pushed the wincontainerd branch from 6eb00d6 to 39516d3 Compare October 18, 2022 13:31

XinShuYang force-pushed the wincontainerd branch from 39516d3 to bb42577 Compare October 18, 2022 14:14

XinShuYang force-pushed the wincontainerd branch 2 times, most recently from e8813f6 to 2e8b776 Compare October 18, 2022 15:25

XinShuYang force-pushed the wincontainerd branch 2 times, most recently from 6a6fbd2 to b949f0d Compare October 18, 2022 23:33

luolanzone added this to the Antrea v1.10 release milestone Dec 9, 2022

luolanzone previously approved these changes Dec 9, 2022

View reviewed changes

XinShuYang dismissed stale reviews from luolanzone and wenyingd via 4f17371 December 14, 2022 08:43

XinShuYang force-pushed the wincontainerd branch 3 times, most recently from 34e8716 to cdf7952 Compare December 14, 2022 10:14

tnqn reviewed Dec 15, 2022

View reviewed changes

tnqn added the action/release-note Indicates a PR that should be included in release notes. label Dec 15, 2022

XinShuYang force-pushed the wincontainerd branch from cdf7952 to 7fd3868 Compare December 16, 2022 02:04

tnqn reviewed Dec 18, 2022

View reviewed changes

XinShuYang force-pushed the wincontainerd branch from 7fd3868 to 3dc6b87 Compare December 19, 2022 15:39

tnqn previously approved these changes Dec 19, 2022

View reviewed changes

XinShuYang dismissed tnqn’s stale review via 7208658 December 19, 2022 17:00

XinShuYang force-pushed the wincontainerd branch from 3dc6b87 to 7208658 Compare December 19, 2022 17:00

Enable containerd CI pipeline for antrea windows

3d08fa2

Add e2e,conformance and networkpolicy tests for windows containerd testbed. ginkgo:v2.1.6 kubernetes:v1.25 containerd:v1.6.6 Signed-off-by: Shuyang Xin <[email protected]>

XinShuYang force-pushed the wincontainerd branch from 7208658 to 3d08fa2 Compare December 20, 2022 03:31

tnqn approved these changes Dec 20, 2022

View reviewed changes

tnqn merged commit 721fc9e into antrea-io:main Dec 20, 2022

Add containerd runtime support for antrea agent on windows #4279

Add containerd runtime support for antrea agent on windows #4279

Conversation

XinShuYang commented Oct 11, 2022

codecov bot commented Oct 11, 2022 • edited Loading

Codecov Report

XinShuYang commented Oct 17, 2022

XinShuYang commented Oct 17, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

XinShuYang commented Oct 18, 2022

wenyingd commented Dec 9, 2022

luolanzone left a comment

Choose a reason for hiding this comment

XinShuYang commented Dec 15, 2022

tnqn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XinShuYang commented Dec 16, 2022

XinShuYang commented Dec 16, 2022

XinShuYang commented Dec 16, 2022

XinShuYang commented Dec 16, 2022

XinShuYang commented Dec 17, 2022

Choose a reason for hiding this comment

XinShuYang commented Dec 19, 2022

tnqn left a comment

Choose a reason for hiding this comment

XinShuYang commented Dec 19, 2022

XinShuYang commented Dec 20, 2022

tnqn left a comment

Choose a reason for hiding this comment

codecov bot commented Oct 11, 2022 •

edited

Loading