introduce XLAT, native IPv4 for clients #1808

christf · 2019-09-04T19:20:54Z

This introduces native ipv4 support for clients in a babel-based, ipv6-only network. There is a jool instance expected somewhere on the network for the plat part. This setup is used in Magdeburg and working well.

I do not think the ipv4 connections survive roaming with the current implementation but at least android-clients do not disconnect from the network any more due to missing ipv4 connectivity.

@CodeFetch I do hope you do not disapprove raising a PR partly containing your work. Let me know if you do and we can work something out.

provide usage documentation including the backend
In our tests we are still seeing performance issues that are not fully understood. Those have to be resolved.
ensure nextnode.ipv4 address is defined

package/gluon-ddhcpd/Makefile

package/gluon-ddhcpd/files/lib/gluon/upgrade/317-ddhcp-init-remove

package/gluon-l3roamd/files/etc/init.d/gluon-l3roamd

package/gluon-ddhcpd/files/etc/init.d/gluon-ddhcpd

package/gluon-xlat464-clat/Makefile

package/gluon-xlat464-plat/Makefile

package/gluon-xlat464-plat/check_site.lua

package/gluon-xlat464-plat/files/xlat464plat.sh

package/gluon-xlat464-plat/luasrc/lib/gluon/upgrade/440-gluon-xlat464-plat

CodeFetch · 2019-09-08T21:57:00Z

@christf Thank you for raising this PR. I think PLAT support should be removed, because it is not usable without N2N/S2N support. Rotanid's remarks regarding indentation and double newlines are correct. I didn't bother at the time of writing it.

christf · 2019-09-10T06:49:50Z

yeah, I will rework it to make it mergable. For now I just wanted to share that there is such a set of packages while still commencing the tests. I am still experiencing performance issues in my test network related to MTU problems. This PR will remain WIP until those are resolved.

CodeFetch · 2019-09-10T12:20:39Z

@christf Indeed the MTU issue is not handled correctly. From the comments the NAT46 device is intended to be used with a big MTU (as much as the WLAN interface plus IPv6 overhead in our case). But that contradicts the RFC which says for the IPv4 network a very small MTU should be set (IPv6-MTU minus IP-header-size-difference - e.g. 1280 - 60 = 1220). Because the problem is that the resulting IPv6 packets won't get fragmented as the default for IPv6 is "don't fragment". So it needs to happen on the IPv4-side with MSS clamping enabled. The NAT46 modules does not check whether packets are too big at the moment https://github.com/ayourtch/nat46/blob/master/nat46/modules/nat46-core.c#L1843.

Performance-wise the NAT46 module might be heavy as it processes each skb directly instead of using a separate worker which is correct in general, but it can take up much memory on high traffic flows, because it reallocates a bigger skb as far as I remember and it takes time until the old one gets freed. This could be optimized. I think the newer kernels have a function to put in a page at the beginning for the IPv6 header and leave the rest of the data untouched and for unlinking references to the resulting skb. So we could possibly save most of the copying process.

T-X · 2019-09-10T16:54:22Z

@CodeFetch:

Performance-wise the NAT46 module might be heavy as it processes each skb directly instead of using a separate worker which is correct in general

Is there an accidental mixup in this sentence? It's the other way round, queuing something onto a kworker with queue_work() is a performance killer. You don't want to use that on the fast-path for your payload packet flow. The only reason you might want to queue something on a kworker is if you have some packet that needs a longer time to be processed and therefore should be processed in a sleepable/interruptable handler.

But lifting onto such a handler is heavy.

CodeFetch · 2019-09-10T17:19:35Z

@T-X Yes, sorry I changed the sentence and didn't read it again. Performance-wise it's ordinarily better, but not memory-performance wise. The module uses skb_copy/reallocates the skbs instead of skb_cloning them and adding/removing the headers with push-pull and removing the references, but this would require a worker which checks on the references on the original one and only starts changing the headers when the old skb has been released by everything else. Do you know what I mean?

Edit:
Maybe one just needs to check on the reference count... I don't know what could still hold a reference when the skb is handed over to the netdev... Thus a worker might not be needed at all. At least for IPv6 to IPv4 translation copying the whole skb seems to be overkill. I ran OOM with this module when using iperf on a 64 MB device.

Edit2: @T-X I assume pskb_expand_head won't give us a real improvement for linear skb's translation from IPv4 to IPv6 as the 60 additional bytes won't be within the margin of the IPv4 max headroom (do you remember where this value is being defined?).
Another option could be to set dev->needed_headroom = 60;, but this will likely also result in a call to pskb_expand_head, but I'm not sure...

christf · 2019-09-20T20:10:20Z

actually outgoing traffic is ok but inbound is problematic. This is not menat to be a high-performance solution. It just has to work barely enough to allow the few remaining clients that refuse to work without ipv4 use the network. When larger scale applications are required, the best performance optimization is to stop the dns server to hand out A records...

CodeFetch · 2019-09-21T15:51:25Z

@christf I agree. I'd still be happy to hear T-X's point, because I don't want to have the same problems with NAT426, but it's OT.
In the worst case (if rebooting nodes happen often) we can have a second look (maybe there is just a bug we haven't found, yet) or do throttling to keep the nodes stable.

christf · 2019-11-24T00:25:52Z

I have this integrated now into our gluon builds. PMTU discovery works with the additional clat rule.
My TL-WR740 with all of its 400Mhz was able to process 17Mbit through wireguard and the clat module. Hence upgrading this PR.

rotanid

i noted a few additional things.
also, i noticed that your PR is using different indentations, even inside the same shell script.
Please try to use at most two, one for lua, one for shell and not mix them, maybe one can look at gluon-core stuff to see which one is preferred

documentation is still missing.

package/gluon-ddhcpd/Makefile

package/gluon-xlat464-clat/Makefile

…ess is assigned

This provides native ipv4 to clients on a network having an ipv6-only backend, relying on external plat. This allows to use clients in ipv6-only networks that would otherwise refuse to connect without a valid ipv4 route. Limitations: * External plat must be provided, for example by jool - for example by https://github.com/FreifunkMD/jool-docker.git * This implementation will break ipv4 connections when clients roam.

christf · 2020-01-22T23:42:38Z

The following changes were performed:

rename of the package, update title
fixing indentation, using tabs now as mentined in the developers guide
added docs
removal of plat references that are not actually used anywhere
instead of removing ddhcp init script, ddhcpd is disabled

package/gluon-464xlat-clat/Makefile

lemoer · 2020-02-03T18:54:08Z

I am using the Magdeburg firmware and therefore this PR since a couple of weeks and did not see any issues in that while.

package/gluon-464xlat-clat/Makefile

neocturne · 2020-02-03T20:00:53Z

package/gluon-ddhcpd/files/lib/gluon/upgrade/317-ddhcpd-disable

@@ -0,0 +1,3 @@
+#!/bin/sh
+
+uci set ddhcpd.settings.enabled="0"


This can be merged into 316-ddhcp

done, please check if it was done correctly

package/gluon-ddhcpd/luasrc/lib/gluon/upgrade/315-dhcp-config-firewall

docs/package/gluon-464xlat-clat.rst

mweinelt

Looks mostly okay, a few nitpicks and then we can get this merged in time for v2020.2.

mweinelt · 2020-03-24T13:35:08Z

docs/package/gluon-464xlat-clat.rst

+Example::
+
+  {
+	clat_range = 'fdff:ffff:ffff::/48', 


Indentation

mweinelt · 2020-03-24T13:37:15Z

docs/package/gluon-464xlat-clat.rst

+
+clat_range : mandatory
+    - infrastructure net (ULA) from which a /96 CLAT prefix will be generated.
+    - This must be a /48 prefix.


A short subclause, explaining why a /48, e.g. how it is partitioned, would go a long way.

mweinelt · 2020-03-24T13:55:04Z

package/gluon-464xlat-clat/luasrc/lib/gluon/upgrade/450-gluon-xlat464-clat

+		end
+	end
+	f:close()
+end


This could reuse gluon.util.file_contains_line.

https://github.com/freifunk-gluon/gluon/pull/1876/files#diff-43f6fea0241b380e57e45be323c68d30

mweinelt · 2020-03-24T14:01:37Z

docs/package/gluon-464xlat-clat.rst

+clat_range : mandatory
+    - infrastructure net (ULA) from which a /96 CLAT prefix will be generated.
+    - This must be a /48 prefix.
+    - This can be the same for each site and is pre-registered at https://wiki.freifunk.net/IP-Netze#IPv6 as part of fdff:ffff:ff00::/40


I don't think we should reference such a badly maintained and outdated wiki page. And if it's all the same we might also default to some prefix, no?

mweinelt · 2020-03-24T14:04:01Z

package/gluon-ddhcpd/check_site.lua

@@ -0,0 +1,2 @@
+


empty line?

mweinelt · 2020-03-24T14:07:26Z

package/gluon-ddhcpd/Makefile

+
+define Package/gluon-ddhcpd
+  TITLE:=Distributed DHCP Daemon for Gluon
+  DEPENDS:=+gluon-core +ddhcpd


Where does ddhcpd come from?

mweinelt · 2020-03-24T14:11:51Z

package/gluon-ddhcpd/Makefile

+
+define Package/gluon-ddhcpd
+  TITLE:=Distributed DHCP Daemon for Gluon
+  DEPENDS:=+gluon-core +ddhcpd


If gluon-ddhcpd depends on mmfd that dependency is missing here.

ddhcpd depends on a means to distribute multicast throughout the network. In babel networks this capability is provided by mmfd. in batman networks this is provided by the l2-property of batman. It could also be provided by bier or any other means. There is no way to model that currently and depending on mmfd is not the right thing to do for batman - as such a dependency was not placed.

mweinelt · 2020-03-24T14:13:36Z

package/gluon-ddhcpd/luasrc/lib/gluon/upgrade/315-dhcp-config-firewall

@@ -0,0 +1,22 @@
+#!/usr/bin/lua


WIth the mmfd rule it's probably appropriate to call the file ddhcpd-firewall.

mweinelt · 2020-03-24T14:14:15Z

package/gluon-ddhcpd/luasrc/lib/gluon/upgrade/316-ddhcp

@@ -0,0 +1,10 @@
+#!/usr/bin/lua


There's no need to sort this behind the firewall rules, both can be put at position 315.

mweinelt · 2020-03-24T14:17:01Z

package/gluon-mesh-babel/check_site.lua

@@ -1,6 +1,8 @@
 need_string_match(in_domain({'node_prefix6'}), '^[%x:]+/64$')
 need_string_match(in_domain({'node_client_prefix6'}), '^[%x:]+/64$')

+need_string_match(in_domain({'clat_range'}), '^[%x:]+/48$', false)


Why does babel require this? And why is this check more specific that the one in gluon-464xlat-clat? Shouldn't gluon-mesh-babel just depend on gluon-464xlat-clat instead?

Babel needs to know which routes (from which IP ranges) should be redistributed.

But the clat functionality is only there, when the clat package is actually included, no?

If a node does not use the CLAT (whyever) it should still redistribute the route. I agree that the site-check fits better in the clat-package itself. There is no clear solution for this, but I guess nobody will be mixing firmwares with and without clat-support. Based on that it is fine to move it to the clat-package. I don't think gluon-mesh-babel should depend on the clat package.

mweinelt · 2020-04-17T01:50:45Z

shellcheck:

+ make lint-sh
Checking contrib/depdot.sh
Checking contrib/lsupgrade.sh
Checking contrib/sign.sh
Checking contrib/sigtest.sh
Checking contrib/actions/install-dependencies.sh
Checking contrib/actions/run-build.sh
Checking package/gluon-464xlat-clat/files/lib/netifd/proto/xlat464clat.sh
package/gluon-464xlat-clat/files/lib/netifd/proto/xlat464clat.sh:82:5: warning: debug is referenced but not assigned. [SC2154]
package/gluon-464xlat-clat/files/lib/netifd/proto/xlat464clat.sh:157:36: warning: local_ip is referenced but not assigned (did you mean 'local_v4'?). [SC2154]
make: *** [Makefile:136: lint-sh] Error 1
script returned exit code 2

rotanid · 2020-04-19T18:36:10Z

Hey @christf , your PR got a review 4 weeks ago and is waiting to be updated :-)

T-X · 2022-07-10T21:54:34Z

The Linux kernel and Babel got support for routing IPv4 packets via IPv6 routes:

Which sounds like a neat idea and overall a lot simpler than XLAT?

CodeFetch · 2022-08-10T08:10:05Z

@T-X Yes, indeed it looks promising. As a sidenote: I've kind of lost interest in Babel, because there is more future potential in Batman as it is much easier to implement technologies like multipath bonding. Maybe we can have a discussion about it at the end of the year. Btw rhashtables was something to keep in mind for improving Batman performance and possibly fixing the scaling issues.

blocktrron · 2024-01-07T22:34:13Z

Closed because of #3105

T-X · 2024-09-20T21:38:27Z

Just in case someone stumbles over this PR again, in case some renewed interest might come up:

Someone at Freifunk Franken has recently implemented SIIT in eBPF, which seems to include code for handling many of the annoying parts, like checksums and fragmentation handling, too: https://git.freifunk-franken.de/jkimmel/siit-bpf
And secondly, there is also work on upstreaming some SIIT implementation into the Linux kernel: https://nlnet.nl/project/IPv6-monostack/

christf added 0. type: enhancement The changeset is an enhancement 3. topic: babel Topic: Babel Layer 3 Routing labels Sep 4, 2019

christf mentioned this pull request Sep 4, 2019

Xlat für das babel Netz bauen, damit ipv4 möglich wird freifunk-ffm/ToDo-Liste#65

Closed

10 tasks

rotanid requested changes Sep 5, 2019

View reviewed changes

christf added the 2. status: waiting-on-author Waiting on some action from the author label Sep 7, 2019

christf changed the title ~~introduce XLAT, native IPv4 for clients~~ WIP: introduce XLAT, native IPv4 for clients Sep 7, 2019

christf force-pushed the xlat branch from 5ee4c9e to 6b6e64c Compare November 24, 2019 00:33

christf changed the title ~~WIP: introduce XLAT, native IPv4 for clients~~ introduce XLAT, native IPv4 for clients Nov 24, 2019

christf force-pushed the xlat branch 3 times, most recently from ab96d5c to 96c1df1 Compare November 24, 2019 13:11

rotanid requested changes Jan 11, 2020

View reviewed changes

package/gluon-ddhcpd/Makefile Outdated Show resolved Hide resolved

package/gluon-xlat464-clat/Makefile Outdated Show resolved Hide resolved

package/gluon-xlat464-clat/Makefile Outdated Show resolved Hide resolved

christf force-pushed the xlat branch 2 times, most recently from 4ab9359 to 223e80d Compare January 12, 2020 20:07

rotanid added this to the 2020.1 milestone Jan 20, 2020

christf added 4 commits January 23, 2020 00:27

gluon-ddhcpd: add new package to configure ddhcpd for gluon

277d1bf

gluon-mesh-babel: utilize clat_range for babel config

78f09e9

gluon-l3roamd: place hook to be called by ddhcpd when a new ipv4 addr…

35a0e3f

…ess is assigned

christf force-pushed the xlat branch from 223e80d to ebbd407 Compare January 22, 2020 23:39

christf removed the 2. status: waiting-on-author Waiting on some action from the author label Jan 22, 2020

rotanid reviewed Jan 24, 2020

View reviewed changes

package/gluon-464xlat-clat/Makefile Show resolved Hide resolved

neocturne reviewed Feb 3, 2020

View reviewed changes

package/gluon-464xlat-clat/Makefile Outdated Show resolved Hide resolved

neocturne reviewed Feb 3, 2020

View reviewed changes

neocturne requested changes Feb 3, 2020

View reviewed changes

rotanid added 2. status: waiting-on-author Waiting on some action from the author and removed 2. status: waiting-on-author Waiting on some action from the author labels Mar 14, 2020

rotanid added 2 commits March 14, 2020 16:39

xlat: docs fixes

5569a6d

xlat: some fixes suggested by neoraider

b6616b6

rotanid force-pushed the xlat branch from 362f985 to b6616b6 Compare March 14, 2020 15:40

rotanid requested a review from neocturne March 14, 2020 15:41

mweinelt requested changes Mar 24, 2020

View reviewed changes

rotanid added the 2. status: waiting-on-author Waiting on some action from the author label Apr 5, 2020

mweinelt mentioned this pull request Apr 11, 2020

Gluon v2020.2 Tracking Issue #1979

Closed

2 tasks

mweinelt modified the milestones: 2020.2, 2020.3 May 2, 2020

rotanid mentioned this pull request Jun 1, 2020

RFC: Provide CLAT on Routers (464XLAT) #1408

Open

rotanid removed this from the 2020.3 milestone Aug 8, 2020

AiyionPrime added the 2. status: merge conflict The merge has a conflict and needs rebasing label Jan 21, 2023

AiyionPrime mentioned this pull request Jan 21, 2023

check existing PRs for linting issues #2765

Closed

25 tasks

blocktrron closed this Jan 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

introduce XLAT, native IPv4 for clients #1808

introduce XLAT, native IPv4 for clients #1808

christf commented Sep 4, 2019 •

edited

Loading

CodeFetch commented Sep 8, 2019

christf commented Sep 10, 2019

CodeFetch commented Sep 10, 2019

T-X commented Sep 10, 2019 •

edited

Loading

CodeFetch commented Sep 10, 2019 •

edited

Loading

christf commented Sep 20, 2019

CodeFetch commented Sep 21, 2019

christf commented Nov 24, 2019

rotanid left a comment •

edited

Loading

christf commented Jan 22, 2020 •

edited

Loading

lemoer commented Feb 3, 2020

neocturne Feb 3, 2020

rotanid Mar 14, 2020

mweinelt left a comment

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020 •

edited

Loading

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020

christf May 6, 2020

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020

mweinelt Mar 24, 2020

CodeFetch Mar 24, 2020

mweinelt Mar 24, 2020

CodeFetch Mar 24, 2020

mweinelt commented Apr 17, 2020

rotanid commented Apr 19, 2020

T-X commented Jul 10, 2022 •

edited

Loading

CodeFetch commented Aug 10, 2022

blocktrron commented Jan 7, 2024

T-X commented Sep 20, 2024

introduce XLAT, native IPv4 for clients #1808

introduce XLAT, native IPv4 for clients #1808

Conversation

christf commented Sep 4, 2019 • edited Loading

CodeFetch commented Sep 8, 2019

christf commented Sep 10, 2019

CodeFetch commented Sep 10, 2019

T-X commented Sep 10, 2019 • edited Loading

CodeFetch commented Sep 10, 2019 • edited Loading

christf commented Sep 20, 2019

CodeFetch commented Sep 21, 2019

christf commented Nov 24, 2019

rotanid left a comment • edited Loading

Choose a reason for hiding this comment

christf commented Jan 22, 2020 • edited Loading

lemoer commented Feb 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweinelt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweinelt Mar 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweinelt commented Apr 17, 2020

rotanid commented Apr 19, 2020

T-X commented Jul 10, 2022 • edited Loading

CodeFetch commented Aug 10, 2022

blocktrron commented Jan 7, 2024

T-X commented Sep 20, 2024

christf commented Sep 4, 2019 •

edited

Loading

T-X commented Sep 10, 2019 •

edited

Loading

CodeFetch commented Sep 10, 2019 •

edited

Loading

rotanid left a comment •

edited

Loading

christf commented Jan 22, 2020 •

edited

Loading

mweinelt Mar 24, 2020 •

edited

Loading

T-X commented Jul 10, 2022 •

edited

Loading