Rate-limiting

[Xopherus]

For some microservices it might be beneficial to rate-limit the number of active goroutines based on time. For example, if I have a Server which interacts with a third-party API and that is limited to 10 calls/sec, I would only want to serve 10 messages/second (assuming each message uses 1 API call).

With the current primitives, this rate limiting would have to be done at the Receiver level, which would introduce blocking by means of mutexes or channels - that's probably not the most efficient. We should consider adding this capability to the Server so we can limit the number of active Receivers based on time.

[Xopherus]

One strategy we've tried with #21 is to allow the Receiver to return a specific error message to the Server to indicate when to throttle. This works really well in situations where it's more likely that another resource needs to be rate-limited, such an upstream API, or database. In that scenario it's unlikely that a per-Server throttle will suffice - you need to have a shared throttling mechanism.

For now I think we'll accept #21 as the default mechanism for throttling, though happy to revisit this in the future if necessary!