Can TCP MultiPath be used to create multiple TCP Connections over the same Interface?

[dbaddorf]

I am looking to try to take a single TCP connection between two SAN's used for replication, and split that TCP stream into multiple TCP Connections over a WAN. I would like to try to see if I can get more throughput with multiple TCP connections than just a single TCP connection (but only using a single Path between destinations). Can TCP MultiPath be used with a single Ethernet port (creating multiple connections) instead of using multiple interfaces (with different paths)?

SAN --- tcpreen (to redirect TCP endpoint) --- TCP Multipath === TCP Multipath --- tcpreen --- SAN

This may not work at all, but I'd appreciate any thoughts about it.

[matttbe]

Hi,

Can TCP MultiPath be used with a single Ethernet port (creating multiple connections) instead of using multiple interfaces (with different paths)?

Yes, you can. The path-manager module is responsible to create and accept subflows, e.g.

• The "fullmesh" path manager is acting per address: you can have multiple IP addresses per network interface
• the "ndiffport" PM is creating multiple subflows by creating subflows to different ports: it can use the same network interface

Probably best to look at scientific papers about that.

For your use-case, you might want something with ndiffport or having in the middle -- between the two tcpreen elements -- a TCP to MPTCP proxy.

[dbaddorf]

Good point about allowing multiple IP's on a single NIC. Thanks for reminding me of that.
I was hoping to use the tcpreen as a sort of "tcp-flow redirector", where I could point SAN1 to the tcpreen server and it would automatically send the TCP traffic to the next tcpreen server, which would then redirect the traffic to the intended destination SAN2. Then I put MPTCP on the two Linux tcpreen servers to multi-path the traffic between these two servers to gain throughput.
I could be entirely misunderstanding both tcpreen and/or MPTCP but I was hoping that I could "insert" the combination of tcpreen and MPTCP in between two devices which I have no control over (other than changing the replication partner IP's).

[matttbe]

You need both end-host to "discuss MPTCP" to be able to benefit from MPTCP.
If you have hosts in the middle that can create/accept MPTCP connections but only forward the TCP traffic, this traffic will remain TCP. If these hosts intercept TCP traffic and proxy the data using MPTCP (new connections), then you can benefit from MPTCP.

If tcpreen is proxying the traffic to new TCP connections, these new connections can use MPTCP.

[dbaddorf]

Matthieu, Thanks so much for your help. I'm hoping that you'll be willing to point me in the right direction, because I am missing something important. I can't seem to find a lot of information on how to configure MPTCP. Here is the setup that I have: SAN1 -- Ubuntu1 [running tcpreen and MPTCP kernel] -- Ubuntu2 [running tcpreen and MPTCP kernel] -- SAN2 I can get SAN1 to talk to SAN2 through tcpreen (which is configured to proxy the TCP connection to the next hop). That works. And my Ubuntu VM's are running MPTCP (according to http://amiusingmptcp.de/). It sounds like using ndiffport would be great because I could have multiple TCP connections between the two Ubuntu VM's while each VM uses a single IP. (Although I am open to configuring each VM to have multiple IP's on their respective subnets). But my confusion is how do you configure the MPTCP on either end to agree to use the same set of ports? In other words, how would I configure Ubuntu1 to send traffic to Ubuntu2 on ports 2000, 2001, and 2002? How would I get tcpreen to send traffic via MPTCP? And how would I configure Ubuntu2 to listen on ports 2000, 2001, and 2002 and then pass that TCP traffic to tcpreen? I certainly appreciate any help that are willing to provide. Dave Dave Baddorf Sr. Systems Engineer Phone: 717.704.1000 Direct 717.925.7865 For Support you can email us here: ***@***.******@***.***> [https://www.icepts.com/wp-content/uploads/2019/07/iceptsemaillogo2019.png] <http://www.icepts.com/> [cid:iCeptsMSGoldLogo2020foremailsignature_466e4634-3503-44dc-8c96-8918679523dd.png]

…

________________________________ From: Matthieu Baerts ***@***.***> Sent: Monday, March 15, 2021 1:29 PM To: multipath-tcp/mptcp ***@***.***> Cc: Dave Baddorf ***@***.***>; Author ***@***.***> Subject: Re: [multipath-tcp/mptcp] Can TCP MultiPath be used to create multiple TCP Connections over the same Interface? (#410) You need both end-host to "discuss MPTCP" to be able to benefit from MPTCP. If you have hosts in the middle that can create/accept MPTCP connections but only forward the TCP traffic, this traffic will remain TCP. If these hosts intercept TCP traffic and proxy the data using MPTCP (new connections), then you can benefit from MPTCP. If tcpreen is proxying the traffic to new TCP connections, these new connections can use MPTCP. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#410 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATHVIBYRIUAHMXJW3P75OTTTDY7W5ANCNFSM4ZGUJXFA>.

[matttbe]

It sounds like using ndiffport would be great because I could have multiple TCP connections between the two Ubuntu VM's while each VM uses a single IP. (Although I am open to configuring each VM to have multiple IP's on their respective subnets).

Why do you need MPTCP between the two Ubuntu hosts? It makes sense if each path of the MPTCP connections is not sharing the same bottleneck (not taking the same path).

A typical use-case of ndiffport is in data centres where each host can reach others via different paths and each connection is load balanced through different paths. In this case, using the same IPs but not the same ports is enough to have each TCP subflow of an MPTCP connection taking different paths.
Are you in this case?

In many other cases, you have different network interfaces connected to different networks. You then use multiple IPs.
It might be easier to route the traffic differently with multiple IPs but I don't know your setup.

But my confusion is how do you configure the MPTCP on either end to agree to use the same set of ports? In other words, how would I configure Ubuntu1 to send traffic to Ubuntu2 on ports 2000, 2001, and 2002? How would I get tcpreen to send traffic via MPTCP? And how would I configure Ubuntu2 to listen on ports 2000, 2001, and 2002 and then pass that TCP traffic to tcpreen?

With ndiffport, on the client side, you only have to configure the number of subflows you want to have: /sys/module/mptcp_ndiffports/parameters/num_subflows.
The client will create X subflows across the same pair of IP-addresses but only modifying the source-port. No need to bind on different ports.

Can we close the ticket?

[dbaddorf]

Matthieu, Thanks for getting back to me. The reason that I would like to experiment with multiple paths is that my traffic is a single TCP stream between two SAN's over an IPSec tunnel. There is some packet loss and that packet loss causes slowdowns in the TCP stream. My hope is to try MPTCP to create different sub-flows to increase the overall bandwidth between the two endpoints. Basically, I'm trying to do what Internet Speedtests do with using multiple streams to maximize bandwidth rather than using just a single TCP stream. In other words, all the sub flows will share the same path but get more throughput because intermittent packet loss will slow down only a single sub flow instead of all the traffic. I really struggling to get my brain around how the ndiffport works on the server side. Yes, the client creates different outbound requests on different ports. But how does the listening side (server) know how to listen to each subflow? There needs to be some coordination between MPTCP on the client and the server to manage the subflows. So if I have Ubuntu VM 1 running tcpreen listening to traffic on 4214/TCP which forwards that traffic to Ubuntu VM 2 on 4214/TCP, how do I get ndiffport/MPTCP to split that single flow into multiple sub-flows between the two Ubuntu VM's? I could be totally misunderstanding what MPTCP is supposed to do or how to make it work with existing applications that aren't making the calls to use MPTCP in code. I do appreciate the help! Dave Dave Baddorf Sr. Systems Engineer Phone: 717.704.1000 Direct 717.925.7865 For Support you can email us here: ***@***.******@***.***> [https://www.icepts.com/wp-content/uploads/2019/07/iceptsemaillogo2019.png] <http://www.icepts.com/> [cid:iCeptsMSGoldLogo2020foremailsignature_466e4634-3503-44dc-8c96-8918679523dd.png]

…

________________________________ From: Matthieu Baerts ***@***.***> Sent: Tuesday, April 27, 2021 1:06 PM To: multipath-tcp/mptcp ***@***.***> Cc: Dave Baddorf ***@***.***>; Author ***@***.***> Subject: Re: [multipath-tcp/mptcp] Can TCP MultiPath be used to create multiple TCP Connections over the same Interface? (#410) It sounds like using ndiffport would be great because I could have multiple TCP connections between the two Ubuntu VM's while each VM uses a single IP. (Although I am open to configuring each VM to have multiple IP's on their respective subnets). Why do you need MPTCP between the two Ubuntu hosts? It makes sense if each path of the MPTCP connections is not sharing the same bottleneck (not taking the same path). A typical use-case of ndiffport is in data centres where each host can reach others via different paths and each connection is load balanced through different paths. In this case, using the same IPs but not the same ports is enough to have each TCP subflow of an MPTCP connection taking different paths. Are you in this case? In many other cases, you have different network interfaces connected to different networks. You then use multiple IPs. It might be easier to route the traffic differently with multiple IPs but I don't know your setup. But my confusion is how do you configure the MPTCP on either end to agree to use the same set of ports? In other words, how would I configure Ubuntu1 to send traffic to Ubuntu2 on ports 2000, 2001, and 2002? How would I get tcpreen to send traffic via MPTCP? And how would I configure Ubuntu2 to listen on ports 2000, 2001, and 2002 and then pass that TCP traffic to tcpreen? With ndiffport, on the client side, you only have to configure the number of subflows you want to have: /sys/module/mptcp_ndiffports/parameters/num_subflows. The client will create X subflows across the same pair of IP-addresses but only modifying the source-port. No need to bind on different ports. Can we close the ticket? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#410 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATHVIB2V2VBO7M4WKWNHVUTTK3VIXANCNFSM4ZGUJXFA>.