Skip to content

Commit 4d8c9fe

Browse files
committed
add blog document for feature ServiceNodePortStaticSubrange
1 parent 5a97479 commit 4d8c9fe

File tree

2 files changed

+94
-0
lines changed

2 files changed

+94
-0
lines changed
Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
---
2+
layout: blog
3+
title: "Kubernetes 1.27: Avoid Collisions Assigning ports to Nodeport Services"
4+
date: 2023-03-08
5+
slug: nodeport-dynamic-and-static-allocation
6+
---
7+
8+
**Author:** xuzhenglun
9+
10+
In Kubernetes, the Service can be used to provide a unified traffic entry point for applications running on a set of Pods. Clients can use the virtual IP address provided by the Service for access, and Kubernetes provides load balancing for traffic accessing different back-end Pods, but ClusterIP-type Service is limited to providing access to nodes within the cluster, and traffic from outside the cluster cannot be routed. One way to solve this problem is to use NodePort-type Service, which maps a VIP's port to a specific port of all nodes in the cluster, thus redirecting traffic from the outside to the inside of the cluster.
11+
12+
## How NodePort's ports are allocated?
13+
14+
When a NodePort-type Service is created, its corresponding port is allocated in two ways.
15+
16+
**Dynamic** : If the Service type is NodePort and the corresponding `service.spec.ports.nodePort` is not specified, the Kubernetes controlpalne will automatically allocat a unused port to it at creation time.
17+
18+
**Static** : In addition to the dynamic auto-assignment described above, you can also explicitly assign a port that is within the nodeport port configuration.
19+
20+
the `service.spec.ports.nodePort` must be unique for each NodePort-type Service across the cluster. Attempting to create a NodePort-type Service with a port already allocated will return an error.
21+
22+
## Why do you need to reserve ports of NodePort-type Service?
23+
Sometimes, you may want to have the NodePort-type Service running on well-known ports so that other components and users inside and outside the cluster can use them.
24+
25+
In some complex cluster deployments with a mix of K8S nodes and non-K8S nodes, it may be necessary to rely on some pre-defined port for communicating. In particular, some fundamental components cannot rely on the VIPs which provided by LoadBalancer-type services, because the load balancing itself that provides the VIPs may also rely on these fundamental components.
26+
27+
Now suppose we need to expose the Minio object storage service on K8S to services located on non-K8S nodes, and the agreed port is `30009`, we need to create a Service as follows:
28+
29+
```yaml
30+
apiVersion: v1
31+
kind: Service
32+
metadata:
33+
labels:
34+
k8s-app: minio
35+
name: minio
36+
namespace: kube-system
37+
spec:
38+
ports:
39+
- name: http
40+
nodePort: 30009
41+
port: 9000
42+
protocol: TCP
43+
targetPort: 9000
44+
selector:
45+
app: minio
46+
type: NodePort
47+
```
48+
49+
However, as mentioned before, if the port `30009` required for the minio Service is not reserved, and another NodePort Service is created and dynamically allocated before or concurrently with minio Service, `30009` can be allocated to those NodePort Service, and minio Service will fail to be created due to a port conflict.
50+
51+
## How can you avoid NodePort-type Service port conflicts?
52+
In the Kubernetes 1.24, Service ClusterIP had already divided the CIDRs into two blocks, using different allocation policies to reduce the risk of conflicts. In Kubernetes 1.27, a similar policy can be adopted for NodePort. You can enable a new featuregate `ServiceNodePortStaticSubrange`. Turning this on allows you to use a different port allocation strategy for NodePort-type Services, and reducing the risk of collision.
53+
54+
The port range for `NodePort` will be divided, based on the formula `min(max(16, nodeport-size / 32), 128)`, which can be described as _never less than 16 or more than 128 with a graduated step between them_.
55+
56+
Dynamic port assignment will use the upper band by default, once this has been exhausted it will use the lower range. This will allow users to use static allocations on the lower band with a low risk of collision.
57+
58+
## Examples:
59+
60+
### default range
61+
service-node-port-range: 30000-32767
62+
Band Offset: min(max(16, 2768/32), 128) = min(max(16, 86), 128) = min(86, 128) = 86
63+
Static band start: 30000
64+
Static band end: 30085
65+
Range end: 32767
66+
67+
### very small range
68+
service-node-port-range: 30000-30015
69+
Band Offset: 0
70+
Static band start: 30000
71+
Static band end: 30000
72+
Range end: 30015
73+
74+
### small(lower boundary) range
75+
service-node-port-range: 30000-30127
76+
Band Offset: min(max(16, 128/32), 128) = min(max(16, 4), 128) = min(16, 128) = 16
77+
Static band start: 30000
78+
Static band end: 30015
79+
Range end: 30127
80+
81+
### large(upper boundary) range
82+
service-node-port-range: 30000-34095
83+
Band Offset: min(max(16, 4096/32), 128) = min(max(16, 128), 128) = min(128, 128) = 128
84+
Static band start: 30000
85+
Static band end: 30127
86+
Range end: 34095
87+
88+
### very large range
89+
service-node-port-range: 30000-38191
90+
Band Offset: min(max(16, 8192/32), 128) = min(max(16, 256), 128) = min(256, 128) = 128
91+
Static band start: 30000
92+
Static band end: 30127
93+
Range end: 38191

content/en/docs/reference/command-line-tools-reference/feature-gates.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,7 @@ For a reference to old feature gates that are removed, please refer to
186186
| `SeccompDefault` | `true` | Beta | 1.25 | |
187187
| `ServerSideFieldValidation` | `false` | Alpha | 1.23 | 1.24 |
188188
| `ServerSideFieldValidation` | `true` | Beta | 1.25 | |
189+
| `ServiceNodePortStaticSubrange` | `false` | Alpha | 1.27 | |
189190
| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | 1.21 |
190191
| `SizeMemoryBackedVolumes` | `true` | Beta | 1.22 | |
191192
| `StatefulSetAutoDeletePVC` | `false` | Alpha | 1.22 | |

0 commit comments

Comments
 (0)