chore(deps): timed-map migration

[shamardy]

Complete timed-map migration from TimeCache, DuplicateCache, ExpirableMap and ExpirableEntry.

[onur-ozkan]

Picked BTreeMap intentionally to process entries in order.

[mariocynicys]

wdym by processing entries in order? what processing?

[onur-ozkan]

wdym by processing entries in order? what processing?

Making sure to process responses in the FIFO way.

[mariocynicys]

yeah where is that 😂

[onur-ozkan]

Fell into this terrible naming (there is no separation at all at the first glance)

https://github.com/KomodoPlatform/komodo-defi-framework/blob/405bcb7daf316477497eb6925cba0ca9640ab383/mm2src/coins/utxo/rpc_clients/electrum_rpc/connection.rs#L340-L344

thought this was the loop iterating over the responses map..

[onur-ozkan]

Honestly I still believe we should prefer BTreeMap here to avoid difficult-to-debug magical future bugs. If we add any logic with the expectation of running FIFO processing, it will accidentally run in a random order and that will be super annoying to catch on runtime. The fact that I already falled in this trap makes it very likely to happen again to me or anyone in the team.

[mariocynicys]

aha cool, im fine with BTree or others, i only nitted to make sure im not missing something with the in-order comment.

[mariocynicys]

Thanks!
One comment inline. LGTM otherwise.

[shamardy]

lp_ordermatch::ordermatch_tests::test_orderbook_order_pairs_trie_state_history_updates_expiration_on_insert and lp_ordermatch::ordermatch_tests::test_orderbook_sync_trie_diff_time_cache are both failing after the recent changes

[mariocynicys]

labeled the never checked maps below

[onur-ozkan]

lp_ordermatch::ordermatch_tests::test_orderbook_order_pairs_trie_state_history_updates_expiration_on_insert and lp_ordermatch::ordermatch_tests::test_orderbook_sync_trie_diff_time_cache are both failing after the recent changes

The order trie logic highly depends on timecache internals, which is covered in the test_orderbook_order_pairs_trie_state_history_updates_expiration_on_insert test.

We have 3 options:

1. Change the order trie expiration logic (timecache has quite different expiration logic (e.g., bulk mode expirations) and removes expired entries on remove and entry functions, not on inserts. Also, reading functions do not consider expirations.)
2. Change the expectations on test_orderbook_order_pairs_trie_state_history_updates_expiration_on_insert flow.
3. Stay with timecache for order trie (this requires to keep time_cache module with ExpirableEntry in the expirable_map module)

I am ok with any of them.

[shamardy]

I think the problem with test_orderbook_order_pairs_trie_state_history_updates_expiration_on_insert is that we don't update expiration of existing entries like before https://github.com/KomodoPlatform/komodo-defi-framework/blob/6aa5d66a91833d36d68a18e30f049bdee95ea7f5/mm2src/common/time_cache.rs#L132-L134 https://github.com/KomodoPlatform/komodo-defi-framework/blob/6aa5d66a91833d36d68a18e30f049bdee95ea7f5/mm2src/common/time_cache.rs#L85
TimedMap doesn't have update_expiration function, maybe if we added it and used it like the below, the problem will be fixed

Index: mm2src/mm2_main/src/lp_ordermatch.rs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/mm2src/mm2_main/src/lp_ordermatch.rs b/mm2src/mm2_main/src/lp_ordermatch.rs
--- a/mm2src/mm2_main/src/lp_ordermatch.rs	(revision 5563e9d6c99366dd19d256d52ef6ba49c7481225)
+++ b/mm2src/mm2_main/src/lp_ordermatch.rs	(date 1737460248340)
@@ -2550,7 +2550,10 @@
 
         if prev_root != H64::default() {
             let history = match pubkey_state.order_pairs_trie_state_history.get_mut(&alb_ordered) {
-                Some(t) => t,
+                Some(t) => {
+                    pubkey_state.order_pairs_trie_state_history.update_expiration(alb_ordered, Duration::from_secs(TRIE_STATE_HISTORY_TIMEOUT));
+                    t
+                },
                 None => {
                     pubkey_state.order_pairs_trie_state_history.insert_expirable(
                         alb_ordered.clone(),

P.S. I am going with option 1 if we can

[onur-ozkan]

TimedMap doesn't have update_expiration function, maybe if we added it and used it like the below, the problem will be fixed

I tried that too but it didn't work either; like I said, time-cache internal logic is tightly coupled with the test expectations.

[onur-ozkan]

trie test expectation logic is 1:1 ported now

[shamardy]

@onur-ozkan please resolve conflicts so that I can approve the PR. It LGTM now.

[mariocynicys]

LGTM