Use the docker.io registry for "latest" antrea images #4235

antoninbas · 2022-09-16T19:45:46Z

It has been observed that pulling
projects.registry.vmware.com/antrea/antrea-ubuntu:latest will often fail. This is because the Harbor registry uses a CDN for caching, which doesn't work well at all when tags are being re-used (in our case, we re-use the latest tag all the time). While most users should be installing a released version of antrea (with an immutable tag) and thus should not suffer from this issue, it can be frustrating for users who want to install the latest antrea built from the main branch.

As a result, we switch to the docker.io registry whenever the latest tag is used, and in particular for checked-in manifest files. The Harbor registry (projects.registry.vmware.com) is still used for all release assets, and more generally for all images with an immutable tag.

We are able to simplify some infrastructure code because of that, as built images no longer need to be tagged with
projects.registry.vmware.com.

codecov · 2022-09-16T20:03:25Z

Codecov Report

Merging #4235 (be38393) into main (4caa8c1) will decrease coverage by 1.28%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4235      +/-   ##
==========================================
- Coverage   64.29%   63.00%   -1.29%     
==========================================
  Files         361      396      +35     
  Lines       52203    54859    +2656     
==========================================
+ Hits        33563    34564    +1001     
- Misses      16186    17792    +1606     
- Partials     2454     2503      +49

Flag	Coverage Δ
e2e-tests	`41.29% <ø> (?)`
integration-tests	`34.91% <ø> (+0.08%)`	⬆️
kind-e2e-tests	`47.83% <ø> (-0.73%)`	⬇️
unit-tests	`43.81% <ø> (-1.20%)`	⬇️

Impacted Files	Coverage Δ
pkg/agent/cniserver/ipam/ipam_service.go	`76.40% <0.00%> (-12.36%)`	⬇️
pkg/agent/controller/trafficcontrol/controller.go	`77.52% <0.00%> (-5.98%)`	⬇️
pkg/ovs/openflow/ofctrl_builder.go	`77.06% <0.00%> (-3.61%)`	⬇️
pkg/agent/memberlist/cluster.go	`72.29% <0.00%> (-3.19%)`	⬇️
pkg/util/k8s/node.go	`86.36% <0.00%> (-1.82%)`	⬇️
...gent/controller/networkpolicy/status_controller.go	`79.83% <0.00%> (-1.69%)`	⬇️
pkg/agent/flowexporter/exporter/exporter.go	`80.44% <0.00%> (-1.24%)`	⬇️
pkg/agent/util/net_linux.go	`32.25% <0.00%> (-1.21%)`	⬇️
pkg/agent/controller/networkpolicy/reconciler.go	`70.27% <0.00%> (-1.14%)`	⬇️
...g/agent/cniserver/interface_configuration_linux.go	`26.45% <0.00%> (-0.98%)`	⬇️
... and 79 more

XinShuYang · 2022-09-17T11:44:19Z

/test-all
/test-windows-all

luolanzone · 2022-09-19T01:20:02Z

ci/jenkins/test.sh

@@ -316,7 +316,7 @@ function deliver_antrea_windows {
    fi

    cp -f build/yamls/*.yml $WORKDIR
-    docker save -o antrea-ubuntu.tar projects.registry.vmware.com/antrea/antrea-ubuntu:latest
+    docker save -o antrea-ubuntu.tar antrea/antrea-ubuntu:latest


The default registry is DOCKER_REGISTRY=$(head -n1 "${WORKSPACE}/ci/docker-registry"), there are quite a few scripts are using this default registry to build images. I feel you may need to double check it.
Another usage I see is DOCKER_REGISTRY="projects.registry.vmware.com".

could you clarify your concern? Tagged images, and in particular images used for testing (e.g., nginx) or tagged base images (e.g. antrea/openvswitch) should still be pulled from projects.registry.vmware.com. The Jenkins test scripts build the antrea image (antrea/antrea-ubuntu:latest) as part of the test (image is not pulled) and there is no reason to tag it with the registry (projects.registry.vmware.com/antrea/antrea-ubuntu:latest) any more.

I mean there are some images' prefix is ${DOCKER_REGISTRY} instead of projects.registry.vmware.com directly. E.g: https://github.com/antrea-io/antrea/blob/main/ci/jenkins/test-mc.sh#L268-L273.

Thanks. As far as I can tell, only test-mc.sh needs to be updated.

antoninbas · 2022-09-19T17:31:20Z

/test-multicluster-e2e

It has been observed that pulling projects.registry.vmware.com/antrea/antrea-ubuntu:latest will often fail. This is because the Harbor registry uses a CDN for caching, which doesn't work well at all when tags are being re-used (in our case, we re-use the latest tag all the time). While most users should be installing a released version of antrea (with an immutable tag) and thus should not suffer from this issue, it can be frustrating for users who want to install the latest antrea built from the main branch. As a result, we switch to the docker.io registry whenever the latest tag is used, and in particular for checked-in manifest files. The Harbor registry (projects.registry.vmware.com) is still used for all release assets, and more generally for all images with an immutable tag. We are able to simplify some infrastructure code because of that, as built images no longer need to be tagged with projects.registry.vmware.com. Signed-off-by: Antonin Bas <[email protected]>

Signed-off-by: Antonin Bas <[email protected]>

antoninbas · 2022-09-20T17:05:19Z

/test-all
/test-multicluster-e2e

antoninbas · 2022-09-20T22:05:11Z

/test-all

XinShuYang · 2022-09-21T00:54:30Z

/test-all
/test-multicluster-e2e

tnqn

LGTM

antoninbas · 2022-09-22T17:33:40Z

@XinShuYang the tests are still not running, I'll try again

antoninbas · 2022-09-22T17:33:44Z

/test-all
/test-multicluster-e2e

XinShuYang · 2022-09-22T23:35:14Z

/test-windows-all
/test-ipv6-e2e
/test-ipv6-only-e2e

XinShuYang

LGTM

antoninbas · 2022-09-23T16:50:25Z

/test-e2e

antoninbas · 2022-09-23T20:27:33Z

@luolanzone any further comment?

luolanzone

LGTM, thanks.

It has been observed that pulling projects.registry.vmware.com/antrea/antrea-ubuntu:latest will often fail. This is because the Harbor registry uses a CDN for caching, which doesn't work well at all when tags are being re-used (in our case, we re-use the latest tag all the time). While most users should be installing a released version of antrea (with an immutable tag) and thus should not suffer from this issue, it can be frustrating for users who want to install the latest antrea built from the main branch. As a result, we switch to the docker.io registry whenever the latest tag is used, and in particular for checked-in manifest files. The Harbor registry (projects.registry.vmware.com) is still used for all release assets, and more generally for all images with an immutable tag. We are able to simplify some infrastructure code because of that, as built images no longer need to be tagged with projects.registry.vmware.com. Signed-off-by: Antonin Bas <[email protected]>

antoninbas requested review from tnqn, luolanzone, edwardbadboy and XinShuYang September 16, 2022 19:46

antoninbas mentioned this pull request Sep 16, 2022

kind: fix antrea-ubuntu image in build/yamls #4147

Closed

antoninbas added area/build-release Issues or PRs related to building and releasing action/release-note Indicates a PR that should be included in release notes. and removed action/release-note Indicates a PR that should be included in release notes. labels Sep 16, 2022

antoninbas added this to the Antrea v1.9 release milestone Sep 16, 2022

antoninbas force-pushed the use-docker.io-registry-for-latest-images branch from d512969 to 1ee04b0 Compare September 16, 2022 20:19

luolanzone reviewed Sep 19, 2022

View reviewed changes

antoninbas added 2 commits September 20, 2022 10:04

Update ci/jenkins/test-mc.sh

be38393

Signed-off-by: Antonin Bas <[email protected]>

antoninbas force-pushed the use-docker.io-registry-for-latest-images branch from 1ee04b0 to be38393 Compare September 20, 2022 17:04

tnqn approved these changes Sep 22, 2022

View reviewed changes

XinShuYang approved these changes Sep 23, 2022

View reviewed changes

luolanzone approved these changes Sep 24, 2022

View reviewed changes

antoninbas merged commit 7216f2a into antrea-io:main Sep 26, 2022

antoninbas deleted the use-docker.io-registry-for-latest-images branch September 26, 2022 18:05

XinShuYang mentioned this pull request Oct 20, 2022

Recover cloud jenkins by fixing ctr image tag issue #4320

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use the docker.io registry for "latest" antrea images #4235

Use the docker.io registry for "latest" antrea images #4235

antoninbas commented Sep 16, 2022

codecov bot commented Sep 16, 2022 •

edited

Loading

XinShuYang commented Sep 17, 2022

luolanzone Sep 19, 2022 •

edited

Loading

antoninbas Sep 19, 2022

luolanzone Sep 20, 2022

antoninbas Sep 20, 2022

antoninbas commented Sep 19, 2022

antoninbas commented Sep 20, 2022

antoninbas commented Sep 20, 2022

XinShuYang commented Sep 21, 2022

tnqn left a comment

antoninbas commented Sep 22, 2022

antoninbas commented Sep 22, 2022

XinShuYang commented Sep 22, 2022

XinShuYang left a comment

antoninbas commented Sep 23, 2022

antoninbas commented Sep 23, 2022

luolanzone left a comment

Use the docker.io registry for "latest" antrea images #4235

Use the docker.io registry for "latest" antrea images #4235

Conversation

antoninbas commented Sep 16, 2022

codecov bot commented Sep 16, 2022 • edited Loading

Codecov Report

XinShuYang commented Sep 17, 2022

luolanzone Sep 19, 2022 • edited Loading

Choose a reason for hiding this comment

antoninbas Sep 19, 2022

Choose a reason for hiding this comment

luolanzone Sep 20, 2022

Choose a reason for hiding this comment

antoninbas Sep 20, 2022

Choose a reason for hiding this comment

antoninbas commented Sep 19, 2022

antoninbas commented Sep 20, 2022

antoninbas commented Sep 20, 2022

XinShuYang commented Sep 21, 2022

tnqn left a comment

Choose a reason for hiding this comment

antoninbas commented Sep 22, 2022

antoninbas commented Sep 22, 2022

XinShuYang commented Sep 22, 2022

XinShuYang left a comment

Choose a reason for hiding this comment

antoninbas commented Sep 23, 2022

antoninbas commented Sep 23, 2022

luolanzone left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 16, 2022 •

edited

Loading

luolanzone Sep 19, 2022 •

edited

Loading