Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: set recent reject db log file limit to 10 #3681

Conversation

driftluo
Copy link
Collaborator

@driftluo driftluo commented Oct 31, 2022

What problem does this PR solve?

In this PR, we introduce the recent reject DB to record recently rejected transactions, which is implemented as rocksdb with TTL, using the default DB parameter configuration.

Since then, this db has been very heavily used on my machine until now:

dust -n 50 -d 1                                                                                                                      
4.0K   ┌── logs
8.0K   ├── ancient
212K   ├── tmp
2.8M   ├── network
164M   ├── indexer
1.5G   ├── tx_pool
 24G   ├── db
 26G ┌─┴ .        

However, the data inside is actually all LOG.old.xxx logs.

As a TTL database, its data storage is inherently time-limited, and it is obviously not reasonable to keep so many DB logs by default.

By default, rocksdb keeps 1000 log files, which is too heavy for this DB, so this PR will change the default value to 10

Check List

Tests

  • Unit test
  • Integration test

Release note

Title Only: Include only the PR title in the release note.

@driftluo driftluo requested a review from a team as a code owner October 31, 2022 08:59
@driftluo driftluo requested review from doitian and removed request for a team October 31, 2022 08:59
Copy link
Member

@doitian doitian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me.

@zhangsoledad please look whether this is a reasonable change.

@zhangsoledad
Copy link
Member

76dc5b9 is ok to me.

3f049d3
The original option file we provided was trade-off for memory usage and speed of ckb synchronization, and there was no optimization for indexer, also I don't think these options are necessary for secondary_db, some options are not available for secondary_db. Given that there is no feedback from https://github.com/nervosnetwork/ckb-indexer about storage performance, I suggest to leave it unchanged and do special optimization if we find bottlenecks here in profiling later.

@driftluo driftluo force-pushed the set-recent-reject-db-log-file-limit-to-ten branch from 3f049d3 to 76dc5b9 Compare November 1, 2022 10:30
@zhangsoledad
Copy link
Member

bors r+

@bors bors bot merged commit 631f887 into nervosnetwork:develop Nov 2, 2022
@driftluo driftluo deleted the set-recent-reject-db-log-file-limit-to-ten branch November 2, 2022 02:19
@doitian doitian mentioned this pull request Nov 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants