Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DB compaction after the deletion of a DB prefix #2003

Merged

Conversation

michaelsutton
Copy link
Collaborator

@michaelsutton michaelsutton commented Apr 3, 2022

Apparently LevelDB does not necessarily compact data following a large delete operation, but rather marks items as deleted, which at times might increase DB size until a compaction is triggered.

Since DB size effects performance even if disk has sufficient space, it makes sense to explicitly call Compact whenever we delete a whole DB prefix.

Specifically, this will help nodes using migrate to reduce DB size following the migration.

@codecov
Copy link

codecov bot commented Apr 3, 2022

Codecov Report

Merging #2003 (f64771d) into dev (ab73def) will decrease coverage by 0.07%.
The diff coverage is 81.81%.

❗ Current head f64771d differs from pull request most recent head 46e7759. Consider uploading reports for the commit 46e7759 to get more accurate results

@@            Coverage Diff             @@
##              dev    #2003      +/-   ##
==========================================
- Coverage   59.70%   59.63%   -0.08%     
==========================================
  Files         663      663              
  Lines       31514    31521       +7     
==========================================
- Hits        18817    18798      -19     
- Misses      10028    10047      +19     
- Partials     2669     2676       +7     
Impacted Files Coverage Δ
...nsensus/processes/pruningmanager/pruningmanager.go 53.47% <ø> (ø)
domain/prefixmanager/prefix.go 60.71% <75.00%> (-0.83%) ⬇️
infrastructure/db/database/ldb/leveldb.go 66.66% <100.00%> (+2.38%) ⬆️
app/appmessage/error.go 0.00% <0.00%> (-27.28%) ⬇️
...etadapter/server/grpcserver/protowire/rpc_error.go 75.00% <0.00%> (-25.00%) ⬇️
app/rpc/rpchandlers/get_mempool_entry.go 41.17% <0.00%> (-23.53%) ⬇️
infrastructure/network/rpcclient/rpc_get_info.go 45.45% <0.00%> (-18.19%) ⬇️
...ructure/network/rpcclient/rpc_get_mempool_entry.go 45.45% <0.00%> (-18.19%) ⬇️
app/protocol/flows/v5/blockrelay/block_locator.go 52.94% <0.00%> (-11.77%) ⬇️
...ain/consensus/database/binaryserialization/hash.go 69.56% <0.00%> (-8.70%) ⬇️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab73def...46e7759. Read the comment docs.

@michaelsutton michaelsutton merged commit ca5c854 into kaspanet:dev Apr 3, 2022
@michaelsutton michaelsutton deleted the compact-inactive-db-prefix branch April 3, 2022 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants