LAN synchronization

[nicoruti]

We should enable a LAN-synchronization mode for speeding up the file transfers. Instead of downloading a file chunk from anywhere in the network, clients of the same user in the same LAN should be used.
This serves as

• Performance improvement (bandwidth is usually higher in LAN)
• Fallback when network is not available

This issue relates with #73 because it's a direct file transfer.

[nicoruti]

The main task is to identify peer addresses that are in the same subnet than the downloader. This may be found out (heuristically) when comparing the external IP addresses (should equal).

TomP2P also provides a functionality to broadcast messages in the same LAN in order to detect other peers. Since we are already aware of the locations of peers, we don't need this by now.

I will implement this as soon as I know more how the external IP's can be retrieved from the TomP2P framework. If it's not possible, we may need to extend our 'Locations' and add a filed 'external IP'.

[ryanjaeb]

I don't know much about your project, but matching an external IP won't be 100% reliable for this. There's nothing stopping 2 devices from sharing an external IP while being completely isolated from each other.

[nicoruti]

@ryanjaeb Yes, you're right. They could be in different subnets, separated by a firewall. But as a heuristical approach, clients with the same external IP are asked first for the data. When the other client can be reached successfully, chances for a better bandwidth than over the Internet are high.

[ryanjaeb]

@nicoruti That makes perfect sense. For some stupid reason I was thinking you'd check for a matching external IP and then attempt to connect the devices using their private IPs. Now I realize you'll simply use the external IP and there's a good chance you'll get better throughput between those devices.

Do you think that approach could inadvertently cause some ISPs to count that traffic against bandwidth caps?

[nicoruti]

Do you think that approach could inadvertently cause some ISPs to count that traffic against bandwidth caps?

Can you explain more, what you mean with that?

[ryanjaeb]

I don't know exactly how the traffic counters used by ISPs work, so that was definitely more of a question than a suggestion that it could happen.

One naive way to count traffic on a SOHO firewall would be to sit there and count anything that has the external IP as the source or destination address. Alternatively you could count anything that goes across the WAN interface. In both cases it could create false positives for a network with isolated LANs, right?

      ISP
       |
  --- WAN ---
  |         |
LAN_1     LAN_2

Even though you're never going to touch the WAN gateway, a poorly implemented traffic counter watching the WAN interface might just assume that anything touching the WAN interface is "internet traffic" even though the traffic gets routed back to an internal network. That also assumes the counting is done on the same device as the WAN interface.

Some linux firewalls will complain about this type of traffic (in the logs) since it's easy to tell there a more efficient route between LAN_1 and LAN_2 than via the WAN interface. However, they still redirect the traffic back to the correct LAN. I suspect this traffic would show up as aggregate usage in RRD traffic graphs for the WAN interface on those devices, but don't have one to test.

Other devices, like pfSense, can't redirect that traffic back onto the LAN without intentionally being configured for it. This is done via some advanced NAT reflection settings that I've never played with (I use split DNS instead). For example, I did a quick test on my pfSense firewall:

• Port forward port 22 to 192.168.0.10
• From my LAN, SSH to my WAN IP

Using pfSense (with the default settings) it's not possible to make this connection.

None of that affects the way you'd implement this feature though. It's just something I was curious about and I might not even have the thought process correct since, as mentioned, I don't have a good way of testing it.

[JasperWallace]

We are running out of ipv4 addresses so some broadband networks won't be doing NAT in the end users home, they will be doing it somewhere deep in the ISP's network - search for Carrier Grade Nat for more info.

If you want to do local peer discovery imho the best way will be to use something like bonjour/rendevous/mdns (or upnp). That way you can be sure the peers are on the same lan segment.