Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document lack of support for Pod traffic shaping on Photon OS 3 #1548

Merged
merged 1 commit into from
Nov 13, 2020
Merged

Document lack of support for Pod traffic shaping on Photon OS 3 #1548

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/os-issues.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ notes](https://coreos.com/releases/).
| Issues |
| ------ |
| [#591](https://github.com/vmware-tanzu/antrea/issues/591) |
| [#1516](https://github.com/vmware-tanzu/antrea/issues/1516) |

If your K8s Nodes are running Photon OS 3.0, you may see error messages in the
antrea-agent logs like this one: `"Received bundle error msg: [...]"`. These
Expand Down Expand Up @@ -94,3 +95,21 @@ the Pod network:
```
iptables -A INPUT -i antrea-gw0 -j ACCEPT
```

### Pod Traffic Shaping

Antrea provides support for Pod [Traffic Shaping](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#support-traffic-shaping)
by leveraging the open-source [bandwidth plugin](https://github.com/containernetworking/plugins/tree/master/plugins/meta/bandwidth)
maintained by the CNI project. This plugin requires the following Kernel
modules: `ifb`, `sch_tbf` and `sch_ingress`. It seems that at the moment Photon
OS 3.0 is built without the `ifb` Kernel module, which you can confirm by
running `modprobe --dry-run ifb`: an error would indicate that the module is
indeed missing. Without this module, Pods with the
`kubernetes.io/egress-bandwidth` annotation cannot be created successfully. Pods
with no traffic shaping annotation, or which only use the
`kubernetes.io/ingress-bandwidth` annotation, can still be created successfully
as they do not require the creation of an `ifb` device.

If Photon OS is patched to enable `ifb`, we will update this documentation to
reflect this change, and include information about which Photon OS version can
support egress traffic shaping.
6 changes: 4 additions & 2 deletions test/e2e/bandwidth_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ func TestPodTrafficShaping(t *testing.T) {
// Test is flaky on dual-stack clusters: https://github.com/vmware-tanzu/antrea/issues/1543.
// So we disable it except for IPv4 single-stack clusters for now.
skipIfIPv6Cluster(t)
nodeName := masterNodeName()
skipIfMissingKernelModule(t, nodeName, []string{"ifb", "sch_tbf", "sch_ingress"})
data, err := setupTest(t)
if err != nil {
t.Fatalf("Error when setting up test: %v", err)
Expand Down Expand Up @@ -138,7 +140,7 @@ func TestPodTrafficShaping(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
clientPodName := fmt.Sprintf("client-a-%d", i)
serverPodName := fmt.Sprintf("server-a-%d", i)
if err := data.createPodOnNode(clientPodName, masterNodeName(), perftoolImage, nil, nil, nil, nil, false, func(pod *v1.Pod) {
if err := data.createPodOnNode(clientPodName, nodeName, perftoolImage, nil, nil, nil, nil, false, func(pod *v1.Pod) {
pod.Annotations = map[string]string{
"kubernetes.io/egress-bandwidth": fmt.Sprintf("%dM", tt.clientEgressBandwidth),
}
Expand All @@ -149,7 +151,7 @@ func TestPodTrafficShaping(t *testing.T) {
if err := data.podWaitForRunning(defaultTimeout, clientPodName, testNamespace); err != nil {
t.Fatalf("Error when waiting for the perftest client Pod: %v", err)
}
if err := data.createPodOnNode(serverPodName, masterNodeName(), perftoolImage, nil, nil, nil, []v1.ContainerPort{{Protocol: v1.ProtocolTCP, ContainerPort: iperfPort}}, false, func(pod *v1.Pod) {
if err := data.createPodOnNode(serverPodName, nodeName, perftoolImage, nil, nil, nil, []v1.ContainerPort{{Protocol: v1.ProtocolTCP, ContainerPort: iperfPort}}, false, func(pod *v1.Pod) {
pod.Annotations = map[string]string{
"kubernetes.io/ingress-bandwidth": fmt.Sprintf("%dM", tt.serverIngressBandwidth),
}
Expand Down
15 changes: 15 additions & 0 deletions test/e2e/fixtures.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,21 @@ func skipIfNotIPv6Cluster(tb testing.TB) {
}
}

func skipIfMissingKernelModule(tb testing.TB, nodeName string, requiredModules []string) {
for _, module := range requiredModules {
// modprobe with "--dry-run" does not require root privileges
cmd := fmt.Sprintf("modprobe --dry-run %s", module)
rc, stdout, stderr, err := RunCommandOnNode(nodeName, cmd)
if err != nil {
tb.Skipf("Skipping test as modprobe could not be run to confirm the presence of module '%s': %v", module, err)
}
if rc != 0 {
tb.Skipf("Skipping test as modprobe exited with an error when trying to confirm the presence of module '%s' - stdout: %s - stderr: %s", module, stdout, stderr)
}
}
tb.Logf("The following modules have been found on Node '%s': %v", nodeName, requiredModules)
}

func ensureAntreaRunning(tb testing.TB, data *TestData) error {
tb.Logf("Applying Antrea YAML")
if err := data.deployAntrea(); err != nil {
Expand Down