Skip to content
This repository was archived by the owner on Jan 22, 2025. It is now read-only.

Add RocksSlotTtl storage type: A slot-based FIFO compatible with Level#27567

Closed
yhchiang-sol wants to merge 1 commit intosolana-labs:masterfrom
yhchiang-sol:slot-ttl
Closed

Add RocksSlotTtl storage type: A slot-based FIFO compatible with Level#27567
yhchiang-sol wants to merge 1 commit intosolana-labs:masterfrom
yhchiang-sol:slot-ttl

Conversation

@yhchiang-sol
Copy link
Copy Markdown
Contributor

Summary of Changes

This PR introduces ShredStorageType::RocksSlotTtl. The shred storage type
that works similarly to FIFO except it uses a slot-based TTL to purge files and
it's compatible with level compaction.

Under the hood, RocksSlotTtl has no background compaction, and it relies on
the recently introduced #26651 that enables LedgerCleanupService to directly
purge any sst files that are older than both --limit-ledger-size and root.

I am still baking the PR and will soon start running experiments. If the experimental
results sound, then RocksSlotTtl should have similar performance as FIFO but
have a stronger consistency guarantee than FIFO and is compatible with the existing
level compaction.

@yhchiang-sol
Copy link
Copy Markdown
Contributor Author

yhchiang-sol commented Sep 2, 2022

Crap. This doesn't work because DeleteFilesInRange does not cover L0. Such a pity :'(.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant