Putting docker container behind a treafik reverse proxy

[phirestalker]

I have many services working on docker with traefik as my reverse proxy. I have tried to set this up pointing traefik at port 8000. I can get to the interface, but the games list is blank.

I am not sure where to look for errors. I have tried looking in docker logs for this container and for the traefik docker. I have also looked at the developer console in Chrome. I found not errors in any of them.

Has anyone gotten this to run behind a reverse proxy (hopefully traefik)?
Where else should I look for errors?

[sergystepanov]

If you look at the Docker Compose example:

cloud-game/docker-compose.yml

Lines 8 to 19 in 167071a

	environment:
	- DISPLAY=:99
	- MESA_GL_VERSION_OVERRIDE=4.5
	- CLOUD_GAME_WEBRTC_SINGLEPORT=8443
	# - CLOUD_GAME_WEBRTC_ICEIPMAP=127.0.0.1
	- CLOUD_GAME_COORDINATOR_DEBUG=true
	- CLOUD_GAME_WORKER_DEBUG=true
	# - PION_LOG_TRACE=all
	ports:
	- "8000:8000"
	- "9000:9000"
	- "8443:8443/udp"

In addition to one HTTP port 8000 of coordinator, there is one UDP port 8443 for worker (port 9000 doesn't matter).
WebRTC connection and streaming won't be possible without it, so make sure this additional port is reachable on your server.

All the logs should be visible in the console if CLOUD_GAME_COORDINATOR_DEBUG and CLOUD_GAME_WORKER_DEBUG environment variables are set to true. And with PION_LOG_TRACE=all it will show everything about WebRTC connections.

Also I added some error reporting when WebRTC ICE stuff fails after a 30 sec timeout.

[phirestalker]

I added an ingress for UDP port 8443, then I added a router and service to the docker-compose.yml file for the cloud-game container. I turned all those logging options on. The only error I can find is sender_interceptor WARNING: 2023/05/17 14:25:17 failed sending: the DTLS transport has not started yet. I do not see the error you mentioned.

Let me know what logs or config files you need.

[phirestalker]

Also, I am running this on my local network only. Do I need to remove the google stun server reference in the config?

[sergystepanov]

I added an ingress for UDP port 8443, then I added a router and service to the docker-compose.yml file for the cloud-game container. I turned all those logging options on. The only error I can find is sender_interceptor WARNING: 2023/05/17 14:25:17 failed sending: the DTLS transport has not started yet. I do not see the error you mentioned.

Let me know what logs or config files you need.

Could you show me Docker logs, browser console output, and what's in the ICE candidates table on the tab in Chrome: chrome://webrtc-internals/ or Firefox: about:webrtc?

Also, I am running this on my local network only. Do I need to remove the google stun server reference in the config?

STUN allows public IP detection and use, so it is not mandatory for a local network.

[phirestalker]

web console log

chrome console output.txt

docker logs

docker_logs.txt

screenshot of table from webrtc-internals

What is the correct environment variable to set public host/ip?

[sergystepanov]

Maybe try it with CLOUD_GAME_WEBRTC_ICEIPMAP=127.0.0.1? Your browser can't connect to 127.0.0.1:8443...

[phirestalker]

same result. Can I put a domain name in there?

EDIT:
nope, the ICE candidate table is empty with the FQDN put in there.

[sergystepanov]

same result. Can I put a domain name in there?

IP adresses only.
Domains only work with TLS:

worker/coordonator:
  server:
    tls:
      domain:

I wrote this thing some time ago https://github.com/sergystepanov/webrtc-troubleshooting. Basically, this is WebRTC part of the retro but all those options can be changed on the fly.
Maybe try it? In Docker there are two ports 3000/TCP (for interface) and 4000/UDP (for WebRTC). And on the page just set "Use a single port" to 4000.

w3t

[sergystepanov]

Try disabling this feature in Chrome: chrome://flags/#enable-webrtc-hide-local-ips-with-mdns. This will allow you to see the IP address being resolved in the browser, instead of obfuscated xxxxxx.local addresses. If this IP is unreachable from your server, then, I think, local STUN may help...

[phirestalker]

Will do. When I enter an IP address for ICEIPMAP, what is it using internally for the type? Is it host? If not, maybe allowing a setting for that will help.

EDIT:
oh, and it is reachable from my browser. My concern was that the STUN servers can not see it because it is a local LAN address. Your, and I think, my successful connection with w3t was using some public address figured out by the STUN server. It was using a port that was not 4000. If it would just try to connect locally without the STUN server (if that's even possible), then it should work.

[sergystepanov]

When I enter an IP address for ICEIPMAP, what is it using internally for the type? Is it host?

Yes, it doesn't matter.

I got a successful test with server offer and the local ip of the server /host. However, I can not seem to get cloud-game to connect to its webrtc port. I set ip map with the local server ip, but it still is unable to connect. I will keep trying different things. Is there a way to make cloud-game server do the offer?

In cloud-game it's the server who makes an offer (for historical reasons due to old Firefox problems).
Strange. It should work similar to w3t.

I'm out of ideas. Last thing which you could try, is to run Docker in your Host network dropping all explicit port mappings. Like we do in cloudretro.io.

cloud-game/.github/workflows/cd/docker-compose.yml

Lines 6 to 7 in 624eecd

	network_mode: "host"
	privileged: true

The idea is that you can leave a single port for WebRTC and proxy it trough your proxy, or just leave your UDPs outta Traefik in singleport/auto-port modes, removing one layer of networking (Docker bridge). ?

[phirestalker]

I got it working. I have the local IP in ICEIPMAP, but I forgot to forward the udp port. I kept forwarding and not forwarding for each thing I tried. I must have forgotten to forward it again for the last tweak.

I see a few issue with the few minutes of gameplay on super mario bros. Should I discuss them here?

[sergystepanov]

Cool.
Would be better not to discuss unrelated issues in a single discussion. Also we have Discord.
Just add a brief comment about how you solved it as a reply on the last message (#399 (comment)) in this discussion + mark it as answer, that would help others.

[sergystepanov]

In short, I assume, there is a problem in your UDP ingress config what makes this port (or IP) unreachable from the client. And the only theoretically useful option here is CLOUD_GAME_WEBRTC_ICEIPMAP, which forces to override, I think all, IP-addresses announced by the worker. This is useful in cases, for example, if worker inside Docker gathers only internal IPs 172.17.x.x and such.
Try to look with that tool what IPs:ports it gathers and what's up with their reachability (set Trace level instead of Debug in the options on the page for that). 🤷‍♂️

[phirestalker]

I was able to solve this by using ICEIPMAP of my server's LAN address, and forwarding the UDP port in the docker compose file. I think that is all. I made a lot of back and forth changes.