net: 15.4 network interfaces use incorrect MTU setting of 127

[dleach02]

Commit 4718dac has broken net_pkt_append().

The change added code to detect max_len overruns of the submitted transmit frame but the logic path ends up setting max_len to the smaller mtu size of the underlying transport if there is a context associated with the packet which then in turn resets the submitted "len" to this smaller size. In my test scenario, I'm testing echo_client over 802.15.4 and the logic ends up resetting len to 65 from its original value of 512. So the function that called net_pkt_append() is testing to ensure that the amount of data appended equaled the amount requested which now it fails every time in my test scenario.

Not sure why the change was added but the splitting up of the packet happens down at a lower layer to ensure that the packet fits the transport. So in the case of 15.4, the 512 byte packet request would be properly broken up into multiple 6LoWPAN packets. I suspect this is true of other transports but I'm not sure.

[pfalcon]

Not sure why the change was added but the splitting up of the packet happens down at a lower layer to ensure that the packet fits the transport.

Which shows that the problem is not with how net_pkt_append() works, but with something else.

Specifically, with setting of MTU for 15.4 interfaces, which is set to 127 or so. This is not a correct value for IPv6 network interface, as IPv6 requires minimum MTU of 1280. Consequently, that's the value which should be used for MTU of 15.4 too, because L2 fragmentation is handled on different level. Let me update ticket title accordingly.

[tbursztyka]

@pfalcon if commit 4718dac changes the behaviour that used to be fine before then yes this commit is the culprit.

You are mixing up HW MTU vs IPv6 layer "MTU". I made that comment actually in your PR afaik, as well as in various places where we've been talking about MTU. 15.4 hw MTU is 127 bytes, it's the right value. Now if IPv6 is used on top, the core should be aware of it and use the IPv6 setting (there is a Kconfig option setting this minimal length actually). Proper mtu handling should be ready to manage the difference transparently.

Handling MTU properly is not trivial, and I approved that commit it because for some reason http new stuff needed it (?). Ok then, but we have a problem that needs to be solved. And afaik, that commit is not that critical now.

@jukkar Shall we revert commit 4718dac ? I don't see MTU handling being properly fixed until 1.10 release.

[pfl]

To clarify a bit more, IEEE 802.15.4 uses 6LoWPAN fragment headers to assemble the 127-byte 15.4 payload into an IPv6 datagram with a minimum size of 1280 bytes. Thus the MTU for 15.4 is 127, while still being 1280 for IPv6.

[pfalcon]

if commit 4718dac changes the behaviour that used to be fine before then yes this commit is the culprit.

Sorry, but there never was "fine". IP stack is full of bugs, and it takes multiple iterations to get it right. It's quite expected that solving (well, making progress on) one issue makes evident other issues, like in this case.

You are mixing up HW MTU vs IPv6 layer "MTU". I made that comment actually in your PR afaik, as well as in various places where we've been talking about MTU.

My argument is that, no, I don't mix them up, they actually were mixed before. We're talking about generic network interfaces, those work on IP level. If there's a different underlying technology, with different maximum frame size, this is different level, which doesn't relate to IP network interface.

This dichotomy is very evident in Linux 15.4 handling. Here's my transcript, based on Stefan Schmidt's (Linux 15.4 maintainer) presentations: https://github.com/pfalcon/zephyr/wiki/Linux802.15.4 .

wpan0 is an 15.4 interface, not suitable for IP networking directly (one sign of that is that it has 15.4 MTU, ~ 127). For IPv6 networking, we need to wrap it in 6LoWPAN interface: ip link add link wpan0 name lowpan0 type lowpan. Created lowpan0 interface will have the standard IPv6 MTU (1280), can have IPv6 addresses configured, etc.

So, my argument is that we have "lowpan0"-style network interface, which has IPv6 address directly assigned, and suitable for IP networking. And apparently, there's no point for Zephyr to mirror Linux and have 2 network interfaces (note that for BLE 6lowpan, Linux has only one, normal netif with normal IPv6 MTU).

[tbursztyka]

It was fine in the context of MTU not being handled. I repeatedly said what are the issue about this, and how to handle properly the hw MTU vs the IPv6 packet size, it's all clear, all known, and it's not a small task at all. No they were not mixed up for the good reason that HW MTU was never handled at all. I don't see why you are discussing this: your commit added a bug it seems. One more on top of all the "full of bugs".

[pfalcon]

@tbursztyka: I re-read your comment #119 (review) . Unfortunately, that's the best reference to your solution I have. There, you cover various "lower-level" points of the problem, but at the high-level, the solution is not POSIX compatible, which leads to issues. Namely, you write:

Better: send would update protocol header and stuff, all according to MTU. net_context_send would then return the amount of data it could send, and the net_pkt would be updated so the un-sent data would be still there (aka: allocated, etc..) ready to be sent as well by the user (or with some more data appended to it etc...)

That's all nice, but it's the same question I posed to Jukka: I have 1MB of data to send, I call Z's pkt data filling function (net_pkt_append), what happens? In your and Jukka's solution, the system exhausts all data fragment buffers to add as much data as possible, to psot-process later, which requires allocating at least one data frag, which thus fail. I proposed to adopt POSIX-based solution as the "high-level" solution to a problem.

It may have some problems at lower-level with some netiface type, 15.4. I would need your help to resolve them, and propose a tentative solution with arguments behind it. This solution would probably be a one-line patch.

If we revert that commit, we're back to the noticeable difference in handling of data send-out between native net API and sockets API, and not clear path/understanding how to deal with that. My working on this issue was actually to minimize such differences, to back @jukkar's statement at #4821 (review): "As the net_context APIs is modelled after BSD socket API (==it provides similar functions) and thus mapping BSD sockets over net_context API is quite straightforward." With all the similarities, data sending process becomes a noticeable divergence point.

[dleach02]

I think we may have an overuse of terms. The "MTU" is typically associated with IP terminology but we are also using it in relation to the 802.15.4 transport size. The definition of an "MTU" is the Maximum Transmission Unit of the largest network layer protocol data unit. IPv4/IPv6 are network layer protocols while IEEE802.15.4 is a data link layer.

If you look at the IEEE802.15.4-2015 specification, it doesn't use the term "MTU" because it is a data link layer protocol. Instead, it has maximum physical layer packet sizes of which the useful part depends on what is enabled (link-layer security, addressing overhead, etc...). So if this code was trying to prevent the application layer of submitting a frame larger then the IPv6 MTU size we probably need a different function to return what the current MTU size is instead of using net_if_get_mtu().

Note that given this naming overload, net_if_get_mtu() really should be named something else instead of continuing the overuse of MTU. Maybe net_if_get_max_len() although the change in the title of this bug probably captures the flip side of this which is that the net_if_get_max_len() is returning the wrong value which means that internal to the IPv6/6LoWPAN there will need to be a different function to return the link-layer max packet size?

[tbursztyka]

@dleach02 the net_if_get_mtu() was there from the start of the net stack rewrite and is meant to be about hw mtu, and basically unused anywhere until now. Thing is, both hw mtu and ip layer mtu are necessary. The later one however does not need to be stored per-interface and can be, I think, assessed from the context: ipv6, ipv4, tcp etc...

Naming is not the issue anyway. We used MTU as hw mtu to fit all bearers. All have their way to name basically the same thing.

@pfalcon I understand your issue in socket API area, and your commit was to make your life easier there. I had doubts about that commit, as I commented there. But what is done is done. Now we need to mitigate. Unfortunately I don't see a quick fix that would fit your issue and ours, in a nice way. So, either we revert and you will have to have a hack in your socket interface (which is not nice either), or ... or I don't see. We don't have time to fix this whole MTU stuff, moreover with the current memory/buffer management (thus why I count a lot on the net_buf split with data work from Johan), do we? We are already going to 1.10 rc1.
The socket behavior vs what net_pkt/net_context does discussion is a bit out of topic here*, in the end the problem would still exist about having 2 worlds, hw and IP layer, that need to work together. Whatever mtu values are found in both sides.

*: didn't you have a thread about that in the ml? Actually, for me net_pkt should represent one IP packet (it has metadata for 1 so), as I said there. However, as I basically never use the user API as I work it way much lower level, it's just my opinion, maybe I am wrong about it.

[dleach02]

@tbursztyka, I understand that the naming problem has been around from the beginning. I'm just pointing out that MTU gets used a lot and it can be confusing. On the surface, the only function for retrieving a MTU value was this net_if_get_mtu() but the part that is missed by @pfalcon is that the "if" in this function is the HW link layer physical packet size and not the IPv6 MTU.

Maybe what is needed is an API that returns the current "network layer" MTU of IP (net_get_mtu() ??). I see in the code that this is a constant NET_IPV6_MTU, but even then I'm not sure what @pfalcon is trying to protect against because doesn't IPv6 support fragmenting up a large packet request? i.e. couldn't we submit a 5000 byte packet and the IPv6 would properly fragment that up to NET_IPV6_MTU size packets which then the 6LoWPAN will fragment up to fit the link layer physical packet sizes?