Bug: Lowering MTU drastically improves the Wireguard upload bandwidth

[invernizzi]

Is this urgent?

None

Host OS

No response

CPU arch

x86_64

VPN service provider

AirVPN

What are you using to run the container

docker-compose

What is the version of Gluetun

v3.34.3

What's the problem 🤔

Hello! Thanks for the wonderful project.

I’ve noticed that my upload across a custom site-to-site Wireguard connection was pretty bad, so I’ve replicated the issue on two commercial VPNs: Mullvad and AirVpn.
After a bit of investigation, I've identified the culprit: the MTU is a smidge too high. Lowering it slightly (1420 -> 1400) solves the issue.

How to replicate:

• Start a wireguard vpn container
• Run the following script (where $1 is the name of the container). It runs a speed test, lowers the mtu, and runs the same speed test.

# Install speedtest-cli.
docker-compose exec $1  /bin/sh -c "apk add speedtest-cli" > /dev/null
# Test with MTU=1420 (the default).
docker-compose exec $1  /bin/sh -c "speedtest-cli"
# Lower the MTU.
echo "Lowering MTU"
docker-compose exec $1  /bin/sh -c "ip link set dev tun0  mtu 1400"
# Test again.
docker-compose exec $1  /bin/sh -c "speedtest-cli"

The output I get:

• Airvpn: upload went from 0.36 Mbit/s to 69.71 Mbit/s

Retrieving speedtest.net configuration...
Testing from M247 Ltd (185.156.175.171)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by teliko GmbH (Limburg) [335.16 km]: 62.993 ms
Testing download speed................................................................................
Download: 185.52 Mbit/s
Testing upload speed......................................................................................................
Upload: 0.36 Mbit/s
Lowering MTU
Retrieving speedtest.net configuration...
Testing from M247 Ltd (185.156.175.171)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by KEVAG Telekom GmbH (Koblenz) [335.53 km]: 45.062 ms
Testing download speed................................................................................
Download: 249.50 Mbit/s
Testing upload speed......................................................................................................
Upload: 69.71 Mbit/s

• Mullvad: upload goes from from 0.21 Mbit/s to 86.61 Mbit/s

Retrieving speedtest.net configuration...
Testing from 31173 Services AB (45.83.220.212)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by TDC Group (Copenhagen) [27.97 km]: 63.497 ms
Testing download speed................................................................................
Download: 113.53 Mbit/s
Testing upload speed......................................................................................................
Upload: 0.21 Mbit/s
Lowering MTU
Retrieving speedtest.net configuration...
Testing from 31173 Services AB (45.83.220.212)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Bahnhof AB (Malmo) [3.50 km]: 35.916 ms
Testing download speed................................................................................
Download: 143.11 Mbit/s
Testing upload speed......................................................................................................
Upload: 86.61 Mbit/s

This is likely due to packet fragmentation - I'm happy to confirm that through tcpdump/wireshark if needed.

Once you've confirmed this behavior, is it possible to either:

• lower the default MTU
• provide a mechanism to set it (e.g., env variable MTU=1400)
  ?

Thanks for the attention and the great project!

Share your logs

N/A

Share your configuration

vpnairvpn:
    image: qmcgaw/gluetun:v3
    cap_add:
      - NET_ADMIN
    restart: unless-stopped
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      - VPN_SERVICE_PROVIDER=airvpn
      - VPN_TYPE=wireguard
      - WIREGUARD_PRIVATE_KEY=
      - WIREGUARD_PRESHARED_KEY=
      - WIREGUARD_ADDRESSES=
      - SERVER_COUNTRIES=Switzerland


  vpnmullvad:
    image: qmcgaw/gluetun:v3
    cap_add:
      - NET_ADMIN
    restart: unless-stopped
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      - VPN_SERVICE_PROVIDER=mullvad
      - VPN_TYPE=wireguard
      - WIREGUARD_PRIVATE_KEY=
      - WIREGUARD_PRESHARED_KEY=
      - WIREGUARD_ADDRESSES=
      - SERVER_CITIES=Malmö

[normanu]

Two different servers are benched both times, thats not a fare comparison?

[invernizzi]

I've reproduced this behavior consistently across multiple servers, and the delta is substantial every time.

The script and configuration are all there, so anyone can reproduce it with fixed benchmark servers.

[qdm12]

Thanks to @LarsHaalck and since PR #1571 you can now use WIREGUARD_MTU. I just added it to the wiki.

Also 2ec9293 changes the default MTU from 1420 to 1400, let's see how it does.

[invernizzi]

Awesome, thanks for the quick fix!

…

On Thu, Jun 8, 2023, 11:55 AM Quentin McGaw ***@***.***> wrote: Closed #1650 <#1650> as completed. — Reply to this email directly, view it on GitHub <#1650 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABCM7PLJBHXXPSJCT4TKADXKGOP7ANCNFSM6AAAAAAYYLCGLQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Ditti4]

I just spent a few hours trying to figure out why only DNS, IMCP and (some) HTTP traffic seemed to make it through my tunnel, but not HTTPS and SSH. Lowering the MTU even further to 1320 fixed all of my issues.

AirVPN seems to recommend a MTU of 1320. They also seem to use this MTU in their official application (Eddie). Would it make sense to use this MTU value for AirVPN specifically? If so, should I open a new issue for this?

[qdm12]

@Ditti4 please reply to #2044 (comment)
I pushed commit bdc594c to set mssfix to 1320-28 by default as well.

[Ditti4]

@qdm12 Thanks for the response! #2044 seems to be about OpenVPN though, right? My issue was with the Wireguard configuration.

[qdm12]

Slightly related, now all MTU default to 1320, since other users faced the same issue (TLS handshake errors mostly).
I'm working on a MTU discovery mechanism, although since it's rather low level (need to fiddle with ip packet header), that may not be possible in Go (despite it being a system programming language!). See #2570