FuturePut fails with churn and low peer numbers

[tbocek]

I observed a case where I get after a DHT put request a success = false, but the storage succeeded (could load it later from another peer).
failedReason is: Future (compl/canc):true/false, FAILED, Expected 3 result, but got 2
That happened after I shut down one peer, so it seems that the client expects 3 peers but gets returned only 2 at the future result and therefore reports success=false.

[tbocek]

The number of peers for requests in small networks is based on the local neighbor map. If I have 3 neighbors, I can get at most 3 answers, if now one goes offline and the local neighbor map is not updated yet, then the future will fail.

I think a proper way to deal with this is to get the number of neighbors from the routing itself and not from the neighbor map. However, this only happens for small networks.