kubelet: add setting for configuring registryPullQPS and registryBurst

[gthao313]

Issue number:
#1495

Description of changes:
Adds two new settings kubernetes.registry-qps and kubernetes.resgitry-burst for configuring

Testing done:

registry-qps

Setting registry-qps to lowest value (1). and lauching/pulling a number of pods/containers that exceed the image pulling QPS to trigger Error ErrImagePull - pull QPS exceeded

Fist step: Test Default value (5)
Test with default value to make sure test result only can be affected by registry-qps

Lauching 12 pods/containers simultaneously

kubectl apply -f amazonlinux.yaml -f debian.yaml -f ubuntu.yaml
			  -f nginx.yaml -f python.yaml -f busybox.yaml -f golang.yaml
			  -f httpd.yaml -f node.yaml -f redis.yaml -f mysql.yaml
			  -f postgres.yaml

All pods/containers have been created succesfully, which is expected.

[tianhg@ip-172-31-35-96 ~]$ kubectl get pods
NAME                READY   STATUS             RESTARTS   AGE
amazonlinux-96gg7   1/1     Running            1          63m
busybox-k4pp7       1/1     Running            0          63m
debian-2qpwt        1/1     Running            1          63m
....

Second step: Test Lowest value (1)
Test with lowest value (1) to makre sure Error ErrImagePull - pull QPS exceeded will be triggered because of inadequate QPS value

Setting registry-qps to lowest value, which is 1.

Userdata
[settings.kubernetes]
registry-qps = 1

Lauching 12 pods/containers simultaneously

kubectl apply -f amazonlinux.yaml -f debian.yaml -f ubuntu.yaml 
			  -f nginx.yaml -f python.yaml -f busybox.yaml -f golang.yaml
			  -f httpd.yaml -f node.yaml -f redis.yaml -f mysql.yaml
			  -f postgres.yaml 

Error ErrImagePull - pull QPS exceeded has been triggerred, which means registry-qps = 1 succussfully limits pulling images.

redis-vl72j         0/1     ErrImagePull        0          13s
httpd-d7mtb         0/1     ContainerCreating   0          13s
node-9xvzk          0/1     ContainerCreating   0          14s
mysql-86jwv         0/1     ContainerCreating   0          13s

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  44s                default-scheduler  Successfully assigned default/redis-bjz2x to ip-192-168-4-68.us-west-2.compute.internal
  Warning  Failed     36s                kubelet            Failed to pull image "redis:5.0.12": pull QPS exceeded
  Warning  Failed     36s                kubelet            Error: ErrImagePull

registry-burst

Setting registry-burst to lowest value (1). and lauching/pulling a number of pods/containers that exceed the image pulling QPS to trigger Error ErrImagePull - pull QPS exceeded

Fist step: Test Default value (10)
Test with default value to make sure test result only can be affected by registry-burst

Lauching 7 pods/containers simultaneously

kubectl apply -f amazonlinux.yaml -f debian.yaml -f redis.yaml -f ubuntu.yaml
			  -f nginx.yaml -f python.yaml -f busybox.yaml

All pods/containers have been created succesfully, which is expected.

NAME                READY   STATUS    RESTARTS   AGE
amazonlinux-dmc2j   1/1     Running   0          2m28s
amazonlinux-q5qhj   1/1     Running   0          2m28s
busybox-7mr77       1/1     Running   0          2m28s
busybox-bccx9       1/1     Running   0          2m28s
....

Second step: Test Lowest value (1)
Test with lowest value (1) to makre sure Error ErrImagePull - pull QPS exceeded will be triggered because of inadequate QPS value

Setting registry-burst to lowest value, which is 1.

Userdata
[settings.kubernetes]
registry-burst = 1

Lauching 7 pods/containers simultaneously

kubectl apply -f amazonlinux.yaml -f debian.yaml -f redis.yaml -f ubuntu.yaml
			  -f nginx.yaml -f python.yaml -f busybox.yaml 

Error ErrImagePull - pull QPS exceeded has been triggerred, which means registry-burst = 1 succussfully limits pulling images.

redis-lr7lg         0/1     ErrImagePull        0          7s
debian-tdgj2        0/1     ErrImagePull        0          8s
redis-2b4zg         0/1     ErrImagePull        0          8s
amazonlinux-55s6b   0/1     ErrImagePull        0          9s
debian-rcf2k        0/1     ErrImagePull        0          10s

Events:
  Type     Reason     Age               From               Message
  ----     ------     ----              ----               -------
  Normal   Scheduled  20s               default-scheduler  Successfully assigned default/debian-tdgj2 to ip-192-168-12-158.us-west-2.compute.internal
  Warning  Failed     18s               kubelet            Failed to pull image "debian:latest": pull QPS exceeded
  Warning  Failed     18s               kubelet            Error: ErrImagePull
  Normal   BackOff    18s               kubelet            Back-off pulling image "debian:latest"

Migration test

upgrade and downgrade

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[gthao313]

I realized that 1.20 had been added, and I will update it

[gthao313]

Pushing above add settings to kubernetes 1.20.

[gthao313]

Push above fix rebase problem.

[zmrow]

Provided the i32/u32 bit is fixed - this looks good!

🐝

[gthao313]

Push above update data type from u32 to i32 according to Ben's comment.

[zmrow]

🍭