Add lock free data structures to wsv

[i1i1]

https://jira.hyperledger.org/browse/IR-1065
https://jira.hyperledger.org/browse/IR-1066
Signed-off-by: i1i1 [email protected]

Description of the Change

Remove lock from WSV and make maps and sets mutable concurrently. It uses dashmap crate for that.

Benefits

This change removes racing for whole wsv and adds locks for its subparts.

Possible Drawbacks

Lockfree actually caused several panics and needs for rework.

Usage Examples or Tests [optional]

Alternate Designs [optional]

dashmap is also good, it offers entry api and some lock free functions, but there are cautions about deadlocks. It is probably a good candidate, but should be used wisely.
lockfree is very generic and good crate, but it seems very unsafe to use it, as it can panic. Also it is quiete slow
flurry is a port of java hashmap, so it seems to be very robust, but it requires using crossbeam epoch in api, so we need to drastically change it in order to use it.

[e-ivkov]

@i1i1 I remember you were doing performance tests for both of the crates, can you publish the results here?

[i1i1]

Benchmark of lockfree vs dashmap:

Insert scenario (50000 Inserts per thread):

Number of threads	Lockfree time(ms)	Dashmap time(ms)	Mutexmap* (ms)
1	27.926	4.7712	5.5090
2	60.090	6.3390	21.185
3	98.009	7.9950	36.675
4	116.96	8.6376	67.014
5	152.93	16.902	130.57
6	191.50	19.881	206.81
7	229.63	23.266	282.35
8	267.37	28.643	328.59

Generally dashmap 5-10x faster than lockfree.

More benchmarks here:
https://github.com/i1i1/dashmap_vs_lockfree

• Mutexmap is actually async_std::sync::Mutex<std::collections::HashMap>

[e-ivkov]

@i1i1 in the benchmarks mutexmap name is a bit confusing. Consider changing it to std HashMap behind async_std Mutex

[e-ivkov]

Or you can mark it with * and describe it below.

[i1i1]

@eadventurous looked through every RwLock in iroha_data_model. Unfortunately using HashSet/HashMap there would require removing Ord everywhere. I guess it is better to leave BTreeMap/BTreeSet for this small cases.

[i1i1]

Possible design for wsv (the same as before but with some lifetime wrangling)
https://gist.github.com/i1i1/cd84e1c5d5f29a1175033ff17ce4fe1c

[e-ivkov]

I was concerned about using dashmap in the async context because of the spinlocks (I thought they might steal computation time from other tasks as they are not cooperating). But I investigated the issue more and it seems they do cooperate. I was testing with ASYNC_STD_THREAD_COUNT=1 - the most extreme case, and other tasks can still proceed even if one of the tasks was locked there, waiting for write access.

Also there is an issue for async support for dashmap, and it is mentioned there that it is fully compatible xacrimon/dashmap#25

[i1i1]

@eadventurous Yeah it won't make sense as it would require locking and removing spinlocks from datastructure. Let's just keep that in mind and never take mutable refence for dashmap as it might deadlock whole datastructure.

[e-ivkov]

Mutable references will sometimes be taken, as we do mutate WSV it is just a matter of for how long, but anyway, I checked this and it seems working even in this extreme case, and it does not block the whole thread.

[i1i1]

@eadventurous we don't need to take mutable reference (&self for rust) as we can mutate it through datastructures like rwlock or dashmap/dashset. We have variety of options

[e-ivkov]

Finally :)

[i1i1]

@eadventurous It will live here https://jira.hyperledger.org/browse/IR-1078