[ENHANCEMENT] Proposal: New Way of Handling Ports

[iwilltry42]

New Way of Handling Ports (Proposal)

Target Version: v5.0.0
Milestone: https://github.com/rancher/k3d/milestone/27

Background

One of the current main issues involves exposing ports on a cluster after it has been created.
The only way to do this with docker is to re-create the affected containers with the new port-mapping.
This is not really feasible to do with a node container on a running cluster as it could confuse Kubernetes when the node goes down unexpectedly.
#615 already adds some super basic functionality to circumvent this by simply mapping the ports to the stateless loadbalancer/serverlb running side-by-side with the cluster.
The only downside of this is that it breaks cluster-external connections going via the loadbalancer for a second or two (the moment we shutdown the old container and start the new one).

Proposal

By default, map all ports to the loadbalancer instead of directly to the nodes to allow for easier changes to the port-mappings.
This should be almost 100% transparent to the user, as the loadbalancer/proxy is a simple NGINX with a super basic configuration, that shouldn't affect the traffic flow in any way.

Effect

Command: k3d cluster create mycluster --port 1234:4321@server[0]

Before

Container k3d-mycluster-server-0 is created with host port 1234 mapped to container port 4321.
Container k3d-mycluster-serverlb is created with only the default API Port mapping.

After

Container k3d-mycluster-server-0 is created with no ports mapped.
Container k3d-mycluster-serverlb is created with one additional port mapping from host port 1234 to container port 4321 and an additional internal configuration to proxy all traffic incoming on 4321 to container k3d-mycluster-server-0.

Implementation

As this should be the new default, we'd need to re-purpose the existing --port flag to do what's described in the previous section.
Now we should also keep a way of using the old/current behavior. This can be implemented in various ways:

1. new flag --port-direct
2. extended syntax (suffix) for existing --port flag: host:port:port/proto@group[index]:direct

Vote: If you're ok with the overall change and only want to give your opinion on this part, please leave a comment with 1️⃣ or 2️⃣ corresponding to your favorite option.

Follow-Up Proposal: Nodefilter Syntax Cleanup

The existing syntax of nodefilters (@group[index]) caused some issues already, e.g. due to some shells (like zsh) trying to interpret square brackets used for indices [index].

Thus, combined with the preceding proposal, I'd propose the following syntax change for nodefilters: @identifier:index:suffix with the following details/options:

• identifier (required)
  • groups: servers or agents
  • individuals: full names, e.g. serverlb or agent-3
  • reserved identifiers: loadbalancer
• index (optional)
  • single integer, e.g. 0 or 2
  • range, e.g. 0-2
  • list of integers, e.g. 0,1,2,3
  • wildcard: *
• suffix (optional)
  • implementation specific and interpreted by the receiver (e.g. the function implementing the port mapping behavior or the one doing the volume mounts)
  • e.g for --port, the suffix can be direct for direct, non-proxied port-mappings, while proxy will be the default, indicating that the port is mapped to the loadbalancer and proxied through to the target container

The main benefits of this are

• less typework
• more flexibility
• less issues with shells misinterpreting the various symbols (proposed syntax at least does not cause issues on tested shells including sh, bash and zsh)

[Max-Schubert]

The suffix version looks good. Makes things simpler and bundles the logic into one command. 2️⃣