Akka cluster bootstrapping with Istio

[blublinsky]

Versions used

Akka management 1.0.8

Akka version: 2.6.8

Expected Behavior

Documentation on the Istio usage https://doc.akka.io/docs/akka-management/current/bootstrap/istio.html is wrong. Documentation suggest allowing inbound communication as follows by using:
annotations:
traffic.sidecar.istio.io/includeInboundPorts: "8080"

where in fact, it should be excluding inbound ports:

annotations:
traffic.sidecar.istio.io/excludeInboundPorts: "2552, 8558"

Note that here port numbers are based of default configuration and can change, if configuration specifies alternative ports

Even with this change this solution only works for Istio version 1.4.x and below.

Actual Behavior

Creation of the Akka cluster fails

Proposed changes:

For Istio 1.4.x and below use the following annotation:

annotations:
traffic.sidecar.istio.io/excludeOutboundPorts: "2552, 8558"
traffic.sidecar.istio.io/excludeInboundPorts: "2552, 8558"

for Istio 1.5.x and beyond, the following is necessary:

1. Change Akka management configuration:

akka.management {
cluster.bootstrap {
contact-point-discovery {
discovery-method = kubernetes-api

 required-contact-point-nr = ${REQUIRED_CONTACT_POINT_NR}

}
}
http {
port = ${MANAGEMENT_PORT}
bind-hostname = "0.0.0.0"
bind-port = ${MANAGEMENT_PORT}
}
}

2. Modify Istio annotations
   There are 2 options here - exclude outbound traffic on port 443

annotations:
traffic.sidecar.istio.io/excludeOutboundPorts: "2552, 8558, 443"
traffic.sidecar.istio.io/excludeInboundPorts: "2552, 8558"

or exclude outbound IP for API server

annotations:
traffic.sidecar.istio.io/excludeOutboundPorts: "2552, 8558"
traffic.sidecar.istio.io/excludeInboundPorts: "2552, 8558"
traffic.sidecar.istio.io/excludeOutboundIPRanges: "/32"

Both solutions work and both have their advantages and disadvantages. For more information look at this writeup https://docs.google.com/document/d/1BOg7zA0vBv644f6TFDI8RKU6q7gmUBWwNk2hDoQ9PmI/edit#. the code example for this (tested with Istio 1.5.x and 1.6.x) is here: https://github.com/blublinsky/akka-sample-cluster-kubernetes-scala

[chbatey]

Thanks Boris, a PR would be great as itsio isn't something on our roadmap atm

[blublinsky]

Thanks Chris, there is no code changes, its just documentation. I did a small writeup https://docs.google.com/document/d/1BOg7zA0vBv644f6TFDI8RKU6q7gmUBWwNk2hDoQ9PmI/edit#heading=h.7548fujn5bom which explains all relevant topics and references a simple project https://github.com/blublinsky/akka-sample-cluster-kubernetes-scala, which is a clone of yours. Let me know what else I can do.

[fernandoiury]

I was able to workaround this issue on the istio side by setting rewriteAppHTTPProbe: false in istio config:
Istio Version: 1.9.5
Command to generate manifests: istioctl manifest generate --set values.sidecarInjectorWebhook.rewriteAppHTTPProbe=false