Skip to content

Commit

Permalink
Merge pull request #41 from dweb-camp-2019/site
Browse files Browse the repository at this point in the history
Site
  • Loading branch information
benhylau authored Nov 29, 2019
2 parents e9f3616 + 927cfbb commit d1f45dd
Show file tree
Hide file tree
Showing 67 changed files with 1,711 additions and 282 deletions.
File renamed without changes.
31 changes: 1 addition & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,4 @@
Meshnet @ DWeb Camp 2019
========================

The 2019 Decentralized Web ~Summit~ Camp will take place at a beautiful piece of coastal land an hour west of San Jose from July 18 to 21.
This area currently has very minimal network infrastructure and Internet connectivity.
Let's use this opportunity to build, test, and use our decentralized systems together,
in a connectivity environment that more closely resembles large parts of the world,
and co-imagine the social processes and local technologies that will help us organize day-to-day,
and make inclusive spaces to learn from each other and build capacity around human-empowering technologies.

![internet](research/site-survey/images/internet.jpg?raw=true)

If you would like to get involved in creating this meshnet, join our meshnet chat at [#dweb-camp-2019-mesh:matrix.org](https://riot.im/app/#/room/#dweb-camp-2019-mesh:matrix.org) to share your ideas. You may also file Issues and send Pull Requests, but first please check out our [FAQ](FAQ.md).

## Goals

This document is the proposal to build a local mesh network with the following goals in mind:

- Reliable local network with 802.11ac speeds
- Access points in key areas for client devices (e.g. phones and laptops) to connect
- Allow Camp participants to extend or run locally-hosted services on the network
- Carry traffic to Internet gateway for client devices to access the Internet with limited bandwidth and reliability
- Document setup and learnings for future events or other groups

## Research Notes & Updates

- [Initial Plan for Meshnet](research/initial-plan/README.md)
- [Client Device Roaming on Meshnets](research/client-device-roaming-on-meshnets.md)
- [Babel + WireGuard Mesh](research/babel-wireguard-mesh.md)
- [Meshnet Site Survey @ The Farm](research/site-survey/README.md)
- [Testnet for DWeb Camp Applications](testnet/README.md)
- [Prodnet for DWeb Camp Applications](prodnet/README.md)
- [Mesh Network Equipment](HARDWARE.md)
See project website on: [https://dweb-camp-2019.github.io/meshnet/](https://dweb-camp-2019.github.io/meshnet/)
5 changes: 5 additions & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
*.gem
.bundle
.sass-cache
_site
Gemfile.lock
15 changes: 15 additions & 0 deletions docs/2.0-network-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
layout: page
title: 2.0 Network Design
navigation: 20
---

## 2.0 Network Design

![network-diagram](images/network-diagram.jpg)

TODO

### Credits

This network is designed with contributions by members of [Toronto Mesh](https://tomesh.net), [People's Open](https://peoplesopen.net), and [Althea](https://althea.org), and draws from MikroTik radio documentations published by [NYC Mesh](https://nycmesh.net).
76 changes: 76 additions & 0 deletions docs/2.1-espressobin-router.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
layout: page
title: 2.1 ESPRESSObin Router
navigation: 21
---

## 2.1 ESPRESSObin Router

The router is responsible for three things:

1. Distribute IP addresses to wired and wireless client devices on the LAN
1. Distribute routes to its LAN across the mesh network using Babel
1. Route IP packets across the mesh network using Babel

It is the central point of each node where all other devices are connected to.

### ESPRESSObin SBUD102 V5

We will configure the ESPRESSObin board so `wan` is used to connect point-to-point mesh radios, and `lan0` and `lan1` are bridged as `lan` for the local wired and wireless network:

```
+-ESPRESSOBIN---------+P+-+
| O |
| W |
| E |
| L L W R |
| A A A |
| +-N-+ +-N-+ +-N-+ |
+----| 1 |-| 0 |-| |----+
+---+ +---+ +---+
```

1. Flash SD card with [Armbian for ESPRESSObin](https://www.armbian.com/espressobin/), then insert it into the ESPRESSObin with Internet access through one of its ethernet ports, then power on (never connect two ESPRESSObin devices to the same network until after you run this install script, the ethernet interface on all devices have the same MAC address and it will packet storm your network, nobody wants that)

1. [Connect via serial interface](http://wiki.espressobin.net/tiki-index.php?page=Serial+connection+-+Linux) to the ESPRESSObin's micro-USB port and run something like `minicom` on your computer to update the boot script (you probably need to paste a few lines at a time and make sure there are no spaces before and after each line):

```
env default -a
setenv fdt_addr 0x6000000
setenv kernel_addr 0x7000000
setenv loadaddr 0x8000000
setenv initrd_size 0x2000000
setenv initrd_addr 0x1100000
setenv scriptaddr 0x6d00000
setenv initrd_image uInitrd
setenv boot_targets 'usb sata mmc1 mmc0'
setenv boot_prefixes '/ /boot/'
setenv bootcmd_mmc0 'setenv devnum 0; setenv boot_interface mmc; run scan_dev_for_boot;'
setenv bootcmd_mmc1 'setenv devnum 1; setenv boot_interface mmc; run scan_dev_for_boot;'
setenv bootcmd_sata 'setenv devnum 0; scsi scan; scsi dev 0; setenv boot_interface scsi; run scan_dev_for_boot;'
setenv bootcmd_usb 'setenv devnum 0; usb start;setenv boot_interface usb; run scan_dev_for_boot;'
setenv bootcmd 'for target in ${boot_targets}; do run bootcmd_${target}; done'
setenv scan_dev_for_boot 'for prefix in ${boot_prefixes}; do echo ${prefix};run boot_a_script; done'
setenv boot_a_script 'ext4load ${boot_interface} ${devnum}:1 ${scriptaddr} ${prefix}boot.scr;source ${scriptaddr};'
saveenv
```
then run `boot` to boot into the SD card
1. Login as `root` / `1234` then run [espressobin/install](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/espressobin/install) and go through the first-run menus.
1. Figure out what type of node this will be:
| Type | Description |
|:--------|:-----------------------------------------------------------------------------------|
|`edge` | Node with only one _Point-to-point Mesh Radio_ |
|`relay` | Node with multiple _Point-to-point Mesh Radios_ and relays traffic for other nodes |
|`gateway`| Node that routes the local network to the Internet |
then run with a `TYPE` from above and a `NODE_ID` between 0 to 99:
```
# wget https://raw.githubusercontent.com/dweb-camp-2019/meshnet/master/prodnet/espressobin/install
# chmod +x install
# ./install TYPE NODE_ID
```
29 changes: 29 additions & 0 deletions docs/2.2-vlan-managed-switch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
layout: page
title: 2.2 VLAN Managed Switch
navigation: 22
---

## 2.2 VLAN Managed Switch

TODO

### NETGEAR GS305E Gigabit Managed Switch

This device is only necessary for `relay` and `gateway` nodes.

We configured the ESPRESSObin to have only one `wan` port. Babel needs to distinguish different network interfaces in order to compute route metrics between the different links and make routing decisions accordingly. So we will make virtual interfaces by tagged each mesh radio with a different VLAN ID, essentially multiplexing differently tagged packets into `wan` which then seperates them out on `wan.1` `wan.2` `wan.3` and `wan.4` on the ESPRESSObin, and what Babel sees are seperate network interfaces as if the ESPRESSObin has 4 `wan` ports.

1. Connect to the GS305E default IP and login to the admin interface

1. Enable `Basic 802.1Q VLAN Status` then configure the following VLAN ID settings:

| port | 1 | 2 | 3 | 4 | 5 |
|:--------|:-:|:-:|:-:|:-:|:-:|
| VLAN ID | 1 | 2 | 3 | 4 |all|

1. Apply the configurations, then wait 30 seconds to ensure the changes are saved

1. Connect `port 5` to the ESPRESSObin `wan` port, and use the VLAN tagged ports for point-to-point radios or ethernet cables that are connect mesh nodes

1. If this is an Internet Gatway node, connect your Internet backhaul to `port 4` (since we have `wan.4` configured to be the Internet route for Internet Gateway ESPRESSObins)
35 changes: 35 additions & 0 deletions docs/2.3-point-to-point-mesh-radios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
layout: page
title: 2.3 Point-to-Point Mesh Radios
navigation: 23
---

## 2.3 Point-to-Point Mesh Radios

Directional radios that make a point-to-point link are put into bridge mode to serve as a wireless replacement of an ethernet cable. These high gain radios need to be aligned carefully to be pointed at one another within about 20 degrees to get optimal speeds.

### MikroTik SXTsq 5 ac (5 GHz)

1. Download the latest release of [RouterOS for the SXTsq 5 ac](https://mikrotik.com/product/sxtsq_5_ac) (`v6.44.3 (stable)` is the version used)

1. Connect your computer to the ethernet port of the MikroTik device and configure the static IP `192.168.88.100` on the local network interface

1. Connect to `192.168.88.1` and login to the web interface as `admin` without password, upload the `.npk` file and reboot the device, then verify RouterOS is upgraded to the latest

1. SSH into the device with `ssh [email protected]`

1. Ensure the device has fresh configurations, run `/system reset-configuration` if needed

1. Run [sxtsq/sxtsq-ap.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/sxtsq/sxtsq-ap.rsc)

1. Reboot the device with `/system reboot` and it will acquire the new IP address `192.168.88.2`

1. Repeat the above steps for the second (client) device, then run [sxtsq/sxtsq-client.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/sxtsq/sxtsq-client.rsc), and it will acquire the new IP address `192.168.88.3` after reboot

### MikroTik Wireless Wire (60 GHz)

These are pre-configured devices that operate at 60 GHz to form a gigabit wireless link at distances ~100 m. They between 700-900 Mbps even when alignment is a little off and have LEDs to indicate link quality. The pair has management IP addresses of `192.168.88.2` and `192.168.88.3`, when can be accessed via the ESPRESSObin via SSH as `admin` user, but this usually isn't necessary as they should "just work" as if it is an actual ethernet cable.

### Ethernet Cable

Yes. If distances allow, you can just use an ethernet cable to link two nodes. Remember the maximum distance for ethernet cables to work reliably is 100 m. I have run into trouble at smaller distances when also doing PoE.
72 changes: 72 additions & 0 deletions docs/2.4-access-point-radios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
layout: page
title: 2.4 Access Point Radios
navigation: 24
---

## 2.4 Access Point Radios

We use three models of radios depending on whether we need wireless coverage all around the antenna (omnidirectional) or an area within a 120 degree cone (sector). We will configure each radio to provide a 5 GHz-only network with:

```
SSID: dwebcamp
Password: dwebcamp
```

There will be two users: `admin` with `ADMIN_PASSWORD`, and `me` with no password and read-only access to the access point web UI.

### MikroTik OmniTIK 5 PoE ac (Outdoor Omnidirectional)

1. Connect your computer to one of the LAN ethernet ports of the MikroTik device and configure the static IP `192.168.88.100` on the local network interface

1. SSH into the device with `ssh [email protected]`

1. Ensure the device has fresh configurations, run `/system reset-configuration` if needed

1. Run [omnitik/omnitik-ap.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/omnitik/omnitik-ap.rsc) after replacing the `ADMIN_PASSWORD`

### MikroTik mANTBox 15s (Outdoor Sector)

1. Connect the MikroTik device to a router with a DHCP server

1. Connect your computer to the router (not to the access point of the MikroTik device, otherwise configuration scripts will not fully execute whenever a command resets the access point)

1. Scan for the IP address of the MikroTik device with a tool like `nmap` or `arp-scan`

1. SSH into the device with `ssh admin@IP_ADDRESS`

1. Ensure the device has fresh configurations, run `/system reset-configuration` if needed

1. Run [mant15s/mant15s-ap.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/mant15s/mant15s-ap.rsc) after replacing the `ADMIN_PASSWORD`

### MikroTik cAP ac (Indoor Dual-band Omnidirectional)

1. Connect the MikroTik device to a router with a DHCP server

1. Connect your computer to the router (not to the access point of the MikroTik device, otherwise configuration scripts will not fully execute whenever a command resets the access point)

1. Scan for the IP address of the MikroTik device with a tool like `nmap` or `arp-scan`

1. SSH into the device with `ssh admin@IP_ADDRESS`

1. Ensure the device has fresh configurations, run `/system reset-configuration` if needed

1. Run [cap/cap-ap.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/cap/cap-ap.rsc) after replacing the `ADMIN_PASSWORD`

### MikroTik wAP ac 3x3 (Indoor Dual-band Omnidirectional)

1. Connect the MikroTik device to a router with a DHCP server

1. Connect your computer directly to the MikroTik device using wireless (not ethernet). There should be 2 new SSIDs for it.

1. Scan for the IP address of the MikroTik device with a tool like `nmap` or `arp-scan`

1. SSH into the device with `ssh admin@IP_ADDRESS`

1. Run the commands in [wap/wap-ap.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/wap/wap-ap.rsc) in stages:
1. Run the first command after replacing the `ADMIN_PASSWORD`.
1. Run the next commands upto and including line 54, this will change the password and break your admin connection.
1. Reconnect with the wAP using the new password from line 54
1. Run `nmap` again to get device's new IP
1. SSH into the device again with it's new IP. (`ssh admin@IP_ADDRESS`)
1. Continue running the rest of the commands in [wap/wap-ap.rsc](https://github.com/dweb-camp-2019/meshnet/blob/master/prodnet/wap/wap-ap.rsc). depending on whether you connected to the 5GHz or 2.4GHz SSID you might break the admin connection one more time, or you will complete the last command and then it will break the connection. (Commands 56-67 are for 2.4GHz, 69-80 are for 5GHz). Keep reconnecting until you've run all of the commands.
11 changes: 11 additions & 0 deletions docs/2.5-client-poe-switches.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
layout: page
title: 2.5 Client PoE Switches
navigation: 25
---

## 2.5 Client PoE Switches

![network-switch](images/network-switch.jpg)

Two models of Linksys PoE switches are used, SRW208P (8-port) and SRW224P (24-port), with 10/100 ports and two gigabit ports on each. 802.3af PoE splitters of 5V (micro-USB) and 12V (barrel-jack) are used to provide power and 10/100 networking to client devices, such as Raspberry Pis and laptops. They are on the same `10.X.0.0/24` LAN as client devices that are wirelessly connected through Access Points.
27 changes: 16 additions & 11 deletions research/initial-plan/README.md → docs/3.0-initial-plan.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
Initial Plan for Meshnet
========================
---
layout: page
title: 3.0 Initial Plan
navigation: 30
---

![access-point-positions](images/access-point-positions.png?raw=true)
## 3.0 Initial Plan

## Hardware
![access-point-positions](images/access-point-positions.png)

### Hardware

The diagram above is the area map, with preliminary positions of _mesh nodes_ that together form the local mesh network.
Each node has a 100 m radius circle around it, representing the region where client devices are expected to form a good connection with the node's access point.
Expand All @@ -19,22 +24,22 @@ Each node is an aggregate of multiple pieces of equipment, which may include:
Some nodes will have interfaces that allow other devices to be plugged in to offer locally-hosted services on the network.
One or more nodes will also serve as the Internet gateway for this local network.

### Radio for directional link
#### Radio for directional link

Each green dotted line represents a directional link formed by a pair of [MikroTik SXTsq 5 ac](https://mikrotik.com/product/sxtsq_5_ac), [used in NYC Mesh](https://docs.nycmesh.net/hardware/sxtsqg5acd/), supporting link speeds at 800+ Mbps.
These devices will run stock firmware.
Where possible, we can also link two nodes with a long ethernet cable.

One alternative option is the 60 GHz [MikroTik Wireless Wire](https://mikrotik.com/product/wireless_wire) or [MikroTik Wireless Wire Dish](https://mikrotik.com/product/wireless_wire_dish), but we will start with the SXTsq 5 ac devices at lower costs.

### Radio for access point
#### Radio for access point

Omnidirectional access points are what client devices will connect to, we will use a [MikroTik OmniTIK 5 PoE ac](https://mikrotik.com/product/rbomnitikpg_5hacd), [used in NYC Mesh](https://docs.nycmesh.net/hardware/mikrotikomnitik5ac/), as the standard device to connect 50+ clients at a time and deliver 800+ Mbps aggregate speeds.
These devices will run stock firmware.

An alternative device we can use as access point is the [MikroTik mANTBox 15s](https://mikrotik.com/product/RB921GS-5HPacD-15S) sector, in areas where only one side of the mount point needs service.

### Network switch, router, and application server
#### Network switch, router, and application server

The [SolidRun ClearFog Pro](https://www.solid-run.com/product/SRM6828S00D01GE000P01CE/) will be used as gigabit network switch and router.
It has 7 gigabit ports, used to connect with the directional and access point radios, Internet gateways, as well as devices offering local services to the network (e.g. Raspberry Pi running a [Scuttlebutt](https://www.scuttlebutt.nz) Pub Server or [GUN](https://github.com/amark/gun) peer, or devices running [Dat](https://datproject.org) and [IPFS](https://ipfs.io) to provide the network with distributed storage capacity).
Expand All @@ -44,18 +49,18 @@ The ClearFog Pro has a 1.6 GHz ARM processor and 1 GB RAM, so we can run some ap
We also hope to connect [LibreRouter](https://librerouter.org) devices to the network.
It is a 802.11n device and does not have the same application processing power, but it includes two 5 GHz directional radios and a 2.4 GHz access point, and it is an all-in-one and open-source device perfect for the environment we are working with.

### Power equipment, mounts, and cables
#### Power equipment, mounts, and cables

Some areas may not have a source of power.
These areas will have to be identified when we survey the site, then we can plan for alternatives.
If no high point is available, we may have to install 5-6 m tall poles to mount the wireless equipment.
We will also bring large spools of ethernet cables, and tools to make them so everyone can participate in building from basic components.

## Prototype
### Prototype

We will create a prototype setup over April and May to validate the devices we chose, and try running various mesh routing software on them.

### Equipment list
#### Equipment list

| Device | Quantity |
|:------------------------------------------------------------------------------------|:---------:|
Expand All @@ -68,7 +73,7 @@ We will create a prototype setup over April and May to validate the devices we c
| Raspberry Pi 3B+ | available |
| Orange Pi Zero | available |

### Routing protocol
#### Routing protocol

We will start with [Althea's flavour of Babel](https://github.com/althea-mesh/babeld) (with [WireGuard](https://www.wireguard.com/)).
There are other mesh protocols such as [batman-adv](https://www.kernel.org/doc/html/v4.15/networking/batman-adv.html) and [BMX6](https://bmx6.net/projects/bmx6) that may also be tested.
Expand Down
Loading

0 comments on commit d1f45dd

Please sign in to comment.