blockstore: send duplicate proofs for chained merkle root conflicts#835
Conversation
| let candidate_erasure_set = ErasureSetId::new(slot, candidate_fec_set_index); | ||
| let candidate_erasure_meta: ErasureMeta = deserialize(candidate_erasure_meta.as_ref())?; | ||
|
|
||
| // Add this candidate to erasure metas to avoid blockstore lookup in future |
There was a problem hiding this comment.
no reason to do this as it is only called at the end of do_insert_shreds which does not need to perform any lookups after.
AshwinSekar
force-pushed
the
chain-dup-proof-altern
branch
from
dff346d to
19a153d
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #835 +/- ##
========================================
Coverage 81.9% 81.9%
========================================
Files 855 855
Lines 232134 232385 +251
========================================
+ Hits 190134 190544 +410
+ Misses 42000 41841 -159 |
behzadnouri
left a comment
behzadnouri
left a comment
There was a problem hiding this comment.
overall this looks less convoluted that the previous code.
| /// Offset to the index of the first received coding shred | ||
| first_received_coding_shred_offset: usize, |
There was a problem hiding this comment.
Why do we need an "offset" here instead of just first_received_coding_shred_index?
There was a problem hiding this comment.
I used an offset to avoid having to convert the index from u32 to usize, but I think it should be fine to just store the index directly.
There was a problem hiding this comment.
For easier review, can you please move all ErasureMeta changes to a separate PR?
| ShredType::Code, | ||
| ); | ||
| let shred = Shred::new_from_serialized_shred( | ||
| self.get_shred_from_just_inserted_or_db(&just_inserted_shreds, shred_id) |
There was a problem hiding this comment.
just_inserted_shreds should always include this shred because this is a new ErasureMeta, no?
| merkle_root_meta.first_received_shred_type(), | ||
| ); | ||
| let shred = Shred::new_from_serialized_shred( | ||
| self.get_shred_from_just_inserted_or_db(&just_inserted_shreds, shred_id) |
There was a problem hiding this comment.
similarly here, shouldn't we expect shred to be already in just_inserted_shreds?
| next fec set shred {next_erasure_set:?} type {:?} chains to merkle root {chained_merkle_root:?}. | ||
| Reporting as duplicate", | ||
| shred.shred_type(), | ||
| next_shred_type, |
There was a problem hiding this comment.
AshwinSekar
force-pushed
the
chain-dup-proof-altern
branch
2 times, most recently
from
9364871 to
271cca6
Compare
AshwinSekar
force-pushed
the
chain-dup-proof-altern
branch
from
271cca6 to
e17bd8b
Compare
|
Backports to the beta branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. Exceptions include CI/metrics changes, CLI improvements and documentation updates on a case by case basis. |
| let (next_shred_index, next_shred_type) = if let Some(merkle_root_meta_entry) = | ||
| merkle_root_metas.get(&next_erasure_set) | ||
| { | ||
| ( | ||
| merkle_root_meta_entry.as_ref().first_received_shred_index(), | ||
| merkle_root_meta_entry.as_ref().first_received_shred_type(), | ||
| ) | ||
| } else if let Some(merkle_root_meta) = self.merkle_root_meta(next_erasure_set).unwrap() { | ||
| ( | ||
| merkle_root_meta.first_received_shred_index(), | ||
| merkle_root_meta.first_received_shred_type(), | ||
| ) | ||
| } else { | ||
| // No shred from the next fec set has been received | ||
| return true; |
There was a problem hiding this comment.
Might be more readable to define next_merkle_root_meta as below and work with that.
let Some(next_merkle_root_meta) = merkle_root_metas
.get(&next_erasure_set)
.map(WorkingEntry::as_ref)
.map(Cow::Borrowed)
.or_else(|| {
self.merkle_root_meta(next_erasure_set)
.unwrap()
.map(Cow::Owned)
})
else {
return true; // No shred from the next fec set has been received
};
let next_shred_id = ShredId::new(
slot,
next_merkle_root_meta.first_received_shred_index(),
next_merkle_root_meta.first_received_shred_type(),
);| let (prev_shred_index, prev_shred_type) = | ||
| if let Some(prev_merkle_root_meta_entry) = merkle_root_metas.get(&prev_erasure_set) { | ||
| ( | ||
| prev_merkle_root_meta_entry | ||
| .as_ref() | ||
| .first_received_shred_index(), | ||
| prev_merkle_root_meta_entry | ||
| .as_ref() | ||
| .first_received_shred_type(), | ||
| ) | ||
| } else { | ||
| let merkle_root_meta = self | ||
| .merkle_root_meta(prev_erasure_set) | ||
| .unwrap() | ||
| .expect("merkle root meta must exist for erasure meta"); | ||
| ( | ||
| merkle_root_meta.first_received_shred_index(), | ||
| merkle_root_meta.first_received_shred_type(), | ||
| ) | ||
| }; | ||
| let prev_shred_id = ShredId::new(slot, prev_shred_index, prev_shred_type); |
There was a problem hiding this comment.
Similarly here:
let prev_merkle_root_meta = merkle_root_metas
.get(&prev_erasure_set)
.map(WorkingEntry::as_ref)
.map(Cow::Borrowed)
.or_else(|| {
self.merkle_root_meta(prev_erasure_set)
.unwrap()
.map(Cow::Owned)
})
.expect("merkle root meta must exist for erasure meta");
let prev_shred_id = ShredId::new(
slot,
prev_merkle_root_meta.first_received_shred_index(),
prev_merkle_root_meta.first_received_shred_type(),
);| } | ||
| let (slot, _) = erasure_set.store_key(); | ||
| // First coding shred from this erasure batch, check the forward merkle root chaining | ||
| if !self.has_duplicate_shreds_in_slot(slot) { |
There was a problem hiding this comment.
Personally rather avoid excessively indented code. This could be:
if self.has_duplicate_shreds_in_slot(slot) {
continue;
}to avoid extra indentation.
| } | ||
| let (slot, _) = erasure_set.store_key(); | ||
| // First shred from this erasure batch, check the backwards merkle root chaining | ||
| if !self.has_duplicate_shreds_in_slot(slot) { |
There was a problem hiding this comment.
similarly here:
if self.has_duplicate_shreds_in_slot(slot) {
continue;
}
AshwinSekar
force-pushed
the
chain-dup-proof-altern
branch
from
02ac1d1 to
4c79fca
Compare
AshwinSekar
added
the
automerge
…licts (backport of #835) (#1009) blockstore: send duplicate proofs for chained merkle root conflicts (#835) * blockstore: send duplicate proofs for chained merkle root conflicts * pr feedback: avoid deserialization, inline warn logs * remove allow[dead_code] after rebase * pr feedback: cow to simplify, avoid extra indentation (cherry picked from commit 175e152) Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
…licts (backport of anza-xyz#835) (anza-xyz#1009) blockstore: send duplicate proofs for chained merkle root conflicts (anza-xyz#835) * blockstore: send duplicate proofs for chained merkle root conflicts * pr feedback: avoid deserialization, inline warn logs * remove allow[dead_code] after rebase * pr feedback: cow to simplify, avoid extra indentation (cherry picked from commit 175e152) Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
3 participants
alternative implementation of #102
Contributes to solana-labs#34897