Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reorg cache fix #3495

Merged
merged 3 commits into from
Nov 25, 2020
Merged

reorg cache fix #3495

merged 3 commits into from
Nov 25, 2020

Conversation

deevope
Copy link
Contributor

@deevope deevope commented Nov 22, 2020

Related to #3489

  • reorg cache period configurable : Allow everyone to configure themselves the reorg cache period, so they could increase it easily in function of their constraint. To repopulate their local mempool.

  • treat cached txs as "new" during reconciliation : Allow our nodes to broadcast txs to their peers.

For every tx in the reorg cache that is successfully added back to the mempool during reconciliation should also be (re)broadcast out to our peers. We should treat these txs as "new" in this scenario.

  • increase max pool size to 300K txs : To prevent for the future, in case of large reorg with empty blocks, against stale full blocks, nodes mempool should be able to handle 6 hours of transactions of full blocks.

  • Evict duplicate txs in reorg cache : Given we add transactions back transaction to the pool, and in the pool we add them back in the reorg cache, we remove them from the reorg cache before they are sent in the pool.

  • Evict only txs accepted to pool and sourced from reorg cache when rec… : Fix of commit 3d8df3a, It will evict only from reorg cache, transactions sourced from reorg cache, accepted by the pool to be broadcasted and added to the reorg_cache again. As we want to keep every transactions that were not accepted by the pool in the reorg_cache in the case where a reorg with larger depth happen again.

test:

  • reorg on testnet made, node rebroadcasted successfully the transactions to other peers

  • I plan also to make a stress test on testnet with 50k transactions - not done yet.

@deevope deevope changed the title reorg cache fix [WIP] reorg cache fix Nov 22, 2020
@deevope deevope changed the title [WIP] reorg cache fix reorg cache fix Nov 22, 2020
@antiochp antiochp self-requested a review November 23, 2020 10:17
// Reorg cache retention period in minute.
// The reorg cache repopulates local mempool in a reorg scenario.
#[serde(default = "default_reorg_cache_period")]
pub reorg_cache_period: i64,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets make this u64 as it does not make sense to support -ve values in the config.

Copy link
Contributor Author

@deevope deevope Nov 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't do u64, as minutes() requires i64. Either we set it to u32 and turn it into i64 in minutes() or we just let i64 but no error will be thrown if users put a negative value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

u32 works then, we're not going to need huge values in here.

Copy link
Member

@antiochp antiochp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this looks good.
We should investigate the various re-broadcast strategies that have been discussed recently. But exposing this config now makes a lot of sense.

Minor comment on i64 vs u64.

👍

Copy link
Member

@antiochp antiochp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good.

@antiochp
Copy link
Member

Going to merge this for 5.0.0 beta.

@antiochp antiochp merged commit fd5dfaa into mimblewimble:master Nov 25, 2020
@antiochp antiochp mentioned this pull request Nov 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants