This repository was archived by the owner on Jan 22, 2025. It is now read-only.
Support opening an in-use rocksdb as secondary#10209
Merged
solana-grimes merged 12 commits intosolana-labs:masterfrom Jun 3, 2020
Merged
Support opening an in-use rocksdb as secondary#10209solana-grimes merged 12 commits intosolana-labs:masterfrom
solana-grimes merged 12 commits intosolana-labs:masterfrom
Conversation
ryoqun
commented
May 24, 2020
ryoqun
commented
May 24, 2020
| let allow_dead_slots = arg_matches.is_present("allow_dead_slots"); | ||
| output_ledger( | ||
| open_blockstore(&ledger_path), | ||
| open_blockstore(&ledger_path, AccessType::AllowSecondary), |
Contributor
Author
There was a problem hiding this comment.
TODO: carefully audit for the appropriate AccessType for all of subcommands.
Contributor
Author
There was a problem hiding this comment.
Done, I've tested all the subcommands and they worked as intended.
@mvines FYI, I found graph to be very useful to create a graph right from a still running validator's ledger. :)
Contributor
There was a problem hiding this comment.
totally, this is gonna be yuuuuge!
ryoqun
commented
May 24, 2020
Contributor
Author
|
Contributor
Author
|
CI is broken... ;) |
ryoqun
commented
May 24, 2020
ryoqun
commented
May 24, 2020
Contributor
Author
|
TODO: maybe don't fallback by default; instead explicit |
This was referenced May 24, 2020
Contributor
|
This looks very promising, thanks @ryoqun |
ryoqun
force-pushed
the
rocksdb-secondary-open
branch
from
3be691a to
e88a3c9
Compare
Codecov Report
@@ Coverage Diff @@
## master #10209 +/- ##
=========================================
- Coverage 81.3% 81.3% -0.1%
=========================================
Files 288 288
Lines 67000 67135 +135
=========================================
+ Hits 54510 54603 +93
- Misses 12490 12532 +42 |
mvines
reviewed
May 25, 2020
Contributor
Author
I've tested enough. Be confident to start using the secondary fallback by default. |
ryoqun
changed the title
ryoqun
force-pushed
the
rocksdb-secondary-open
branch
from
1042882 to
890e823
Compare
Contributor
Author
|
@mvines This is ready for review!! :) |
mvines
reviewed
Jun 1, 2020
Contributor
mvines
left a comment
mvines
left a comment
There was a problem hiding this comment.
It looks like solana-validator opens with AccessType::OnlyPrimary (Blockstore::open_with_signal)?
| Self::do_open_with_signal(ledger_path, AccessType::OnlyPrimary) | ||
| } | ||
|
|
||
| pub fn open_with_signal_with_access_type( |
Contributor
There was a problem hiding this comment.
Suggested change
| pub fn open_with_signal_with_access_type( | |
| pub fn open_with_signal_and_access_type( |
Contributor
Author
@mvines Yeah, I deliberately do so because...:
|
Co-authored-by: Michael Vines <mvines@gmail.com>
mvines
reviewed
Jun 1, 2020
mvines
previously approved these changes
Jun 2, 2020
mergify Bot
pushed a commit
that referenced
this pull request
Jun 3, 2020
automerge (cherry picked from commit caa7f7a) # Conflicts: # ledger-tool/Cargo.toml # ledger-tool/src/main.rs
mergify Bot
pushed a commit
that referenced
this pull request
Jun 3, 2020
automerge (cherry picked from commit caa7f7a)
solana-grimes
pushed a commit
that referenced
this pull request
Jun 3, 2020
solana-grimes
pushed a commit
that referenced
this pull request
Jun 3, 2020
This was referenced Jun 3, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
it seems that the rocksdb's notorious limitation is now lifted: facebook/rocksdb#4899 ㊗️
Maybe our bitter experience caused blindness for the rocksdb; it's certainly improving. ;)
Summary of Changes
Fallback to secondary access when primary failed from
ledget-tool's read-only subcommands.solana-validatorand other subcommands behavior hasn't been changed.This realizes to run
create-snapshotwhile a validator is running and other various debugging subcommands.PS: I know my half-backed
ledger-toolpr #9490 is laying around... Sorry for our longer pr queue. ;) I thought this worth to interrupt. :)