Consolidate/improve IPC

[merlinran]

Currently pipe, zMQ and shared memory are used. Pipe is feature limited and caused some issues.

[merlinran]

Some thoughts on communication in the large. A robot needs to communicate with either the server / control application, other robots in the fleet, or optionally, 3rd party services, directly or forwarded by the server. The network environment is inherently unreliable. The Wifi signal can be weak; The target robot / server may be temporarily loss power or out of signal coverage. Weather and electromagnetic noise can happen.

The types of messages can be categorized as follows:

1. State update. Where it is, what it is doing, what's the navigation target, power level, etc. Fellow robots or servers can consume the information. As long as a robot frequently updates its state, no reliable delivery is required.
2. Events such as alerts or domain specific data. They may need to be kept re-sending with various priorities until the acknowledgements are got.
3. Access to 3rd party services - weather forecast, DGPS service, plant identification service, etc. Because of the likelihood of no Internet access, such features need to be designed as asynchronous and optional to avoid affecting normal operations.
4. Commands and feedback. An application or a management server asks certain robots to do something. The message may not be able to deliver, or the response may not be able to get back to the sender. They can be treated as events in the opposite direction, but the human operator wants to get the feedback promptly, so the UX impact needs to be considered.

If we use zMQ, PUB/SUB may be a better fit than REQ/REP patterns, and possibly with multicast. An alternative is https://github.com/libp2p/py-libp2p, which has message propagation and security features built in, if we want to use in conjunction with wireless ad-hoc network.

For now, given that we introduced SocketIO for realtime update of herd data, it makes sense to replace zMQ based robot-server communication with SocketIO as well, to greatly reduce code complexity. If we ever want to use zMQ again, the PUB/SUB pattern needs a different implementation from the current REQ/REP one.

[samuk]

You might like https://github.com/markqvist/Reticulum once C port is complete I'm hoping meshtastic.org will use it as network stack.