Docker Networking in bridge mode with multiple public IP Addresses

[kess78]

Background (please complete the following information):

• Panel or Daemon: Daemon
• Version of Panel/Daemon: 0.6.8
• Server's OS: Debian 9
• Your Computer's OS & Browser: Windows 10

Describe the bug
Problem with MASQUERADE when running multiple servers on the same node with multiple Public IP Addresses.
Docker/Wings using default configuration with pterodactyl_nw
The problem is that outgoing traffic generated from inside containers, always goes out using the primary IP Address, instead of using the assigned IP Address cause of docker iptables MASQUERADE.
Incoming traffic is OK because DNAT rules are applied correctly.
The only solution (partial solution) I've found, is to:

1. docker.policy.network.enable_ip_masquerade to false
2. MANUALLY adding iptables rule as follows:
   Server 1: iptables -t nat -A POSTROUTING -s 172.18.0.2 ! -o pterodactyl_nw -j SNAT --to-source 1.1.1.1
   Server 2: iptables -t nat -A POSTROUTING -s 172.18.0.3 ! -o pterodactyl_nw -j SNAT --to-source 2.2.2.2

Panel: PHP 7.2.14-1+020190113100742.14+stretch1.gbpd83c69 (cli) (built: Jan 13 2019 10:07:43) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.14-1+020190113100742.14+stretch1.gbpd83c69, Copyright (c) 1999-2018, by Zend Technologies
Daemon:
Linux node01 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux

To Reproduce
Steps to reproduce the behavior:

1. Normally configure a node with 2 or more Public IP Addresses
2. Assign allocations in panel
3. Create a new gameserver on IP 1.1.1.1
4. Create a new gameserver on IP 2.2.2.2
5. on node shell:

• docker exec -it container_id_1 curl http://checkip.dyndns.com
• docker exec -it container_id_2 curl http://checkip.dyndns.com

Expected behavior
The correct result should be 1.1.1.1 on container_id_1 and 2.2.2.2 on container_id_2 but cause of MASQUERADE the result is 1.1.1.1 in both cases.

The correction is to manage SNAT rules directly with the daemon instead of using masquerade.
A full description can be found here: https://medium.com/@havloujian.joachim/advanced-docker-networking-outgoing-ip-921fc3090b09

This is why some Source games are not working when not in host mode. Fixing that also fixes the issue with these Source games by mantaining the docker network layer security.

Thx for your time :-)

[parkervcp]

These kinds of settings are covered in the docs... https://pterodactyl.io/daemon/configuration.html#custom-network-interfaces

[kess78]

Yes, they are covered until MASQUERADE disable.
But then the correct iptabled rules should be entered MANUALLY in order to reflect the expected behavior.

Please have a look at the article on medium.com, or if you prefer test it yourself… It's fast to see what happens

[parkervcp]

We aren't touching the iptables rules in any way... ever. It's to easy to overlap a rule or something. I get that docker messes with things but that is how it is.

[kess78]

I'm sorry to read your answer, you admit the problem existance but you state that you're not going to fix it or even to look into it.
I know that iptables could create overlaps, but in that case it won't be so "serious". It's just a single SNAT rule for each container that tells which IP Address the container should use in internet.

If you can tell where these things are managed at present (daemon part), I'll try to test something from my part, and if I find a good solution I'll be pleased to share.