Skip to content
This repository was archived by the owner on Jan 16, 2026. It is now read-only.

feat(derive): Channel Flushing#576

Closed
refcell wants to merge 2 commits intomainfrom
rf/sb-prefix-check
Closed

feat(derive): Channel Flushing#576
refcell wants to merge 2 commits intomainfrom
rf/sb-prefix-check

Conversation

@refcell
Copy link
Contributor

@refcell refcell commented Sep 26, 2024

Description

Channel flushing on invalid span batch in the BatchStream.

Adds validation checks to the SpanBatch for holocene.

@refcell refcell changed the title feat(derive): channel flushing feat(derive): Channel Flushing Sep 26, 2024
Copy link
Contributor Author

refcell commented Sep 26, 2024

@refcell refcell requested a review from clabby September 26, 2024 15:34
@refcell refcell added A-proof Area: proof crates W-holocene Workstream: Holocene K-feature Kind: feature labels Sep 26, 2024 — with Graphite App
@refcell refcell marked this pull request as ready for review September 26, 2024 15:35
@refcell refcell added the M-do-not-merge Meta: Do not merge label Sep 26, 2024 — with Graphite App
@refcell refcell self-assigned this Sep 26, 2024
@codecov
Copy link

codecov bot commented Sep 26, 2024

Codecov Report

Attention: Patch coverage is 14.11765% with 146 lines in your changes missing coverage. Please review.

Project coverage is 78.3%. Comparing base (1147740) to head (7bfdf5a).
Report is 3 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
crates/derive/src/batch/span_batch/batch.rs 0.0% 74 Missing ⚠️
crates/derive/src/stages/batch_stream.rs 18.2% 67 Missing ⚠️
crates/derive/src/stages/channel_reader.rs 72.7% 3 Missing ⚠️
crates/derive/src/pipeline/builder.rs 0.0% 2 Missing ⚠️
Additional details and impacted files

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@refcell refcell changed the base branch from rf/bs-flush to main September 30, 2024 18:28

/// Returns the ending timestamp for the span batch.
pub fn end_timestamp(&self) -> u64 {
self.batches.last().map_or(0, |b| b.timestamp)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should follow the same behavior as timestamp and panic if batches is empty. Getting 0 could cause some weird UB.

l2_safe_head: L2BlockInfo,
block_time: u64,
fetcher: &mut BF,
) -> bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use BatchValidity? There are circumstances within these checks where we're undecided, not Drop.

Ok(block) => block,
Err(e) => {
warn!("failed to fetch L2 block number {parent_num}: {e}");
return false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RPC fetch failing should be temporary - should not influence batch validity

Comment on lines +76 to +91
// If the span batch L1 origin check is not part of
// the canonical L1 chain, the span batch is invalid.
let starting_epoch_num = self.starting_epoch_num();
if starting_epoch_num > parent_block.l1_origin.number + 1 {
warn!(
"batch is for future epoch too far ahead, while it has the next timestamp, so it must be invalid. starting epoch: {} | next epoch: {}",
starting_epoch_num,
parent_block.l1_origin.number + 1
);
return false;
}
// Check if the batch is too old.
if starting_epoch_num < parent_block.l1_origin.number {
warn!("dropped batch, epoch is too old, minimum: {:?}", parent_block.block_info.id());
return false;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +43 to +49
/// A consecutive, time-centric window of L1 Blocks.
/// Every L1 origin of unsafe L2 Blocks must be included in this list.
/// If every L2 Block corresponding to a single L1 Block becomes safe,
/// the block is popped from this list.
/// If new L2 Block's L1 origin is not included in this list, fetch and
/// push it to the list.
l1_blocks: Vec<BlockInfo>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can re-use the l1_blocks in the BatchQueue rather than doubling it up? The batch queue already holds onto these - would it work to extend the BatchQueueProvider trait's next_batch signature to include the l1_blocks?

They get prepared directly before the call to next_batch, think that would work fine: https://github.com/anton-rs/kona/blob/main/crates/derive/src/stages/batch_queue.rs#L306-L339

@refcell refcell closed this Sep 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

A-proof Area: proof crates K-feature Kind: feature M-do-not-merge Meta: Do not merge W-holocene Workstream: Holocene

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants