Skip to content

feat(conductor): skip outdated block metadata#1120

Merged
SuperFluffy merged 2 commits intomainfrom
superfluffy/skip-known-heights
Jun 4, 2024
Merged

feat(conductor): skip outdated block metadata#1120
SuperFluffy merged 2 commits intomainfrom
superfluffy/skip-known-heights

Conversation

@SuperFluffy
Copy link
Contributor

@SuperFluffy SuperFluffy commented May 29, 2024

Summary

Conductor drops sequencer metadata items retrieved from Celestia if their height has already been executed/committed.

Background

Conductor would try and unnecessarily verify Sequencer metadata retrieved from Celestia, putting a lot of pressure on the Sequencer node. Because Conductor is stateless, it fetches all Celestia blobs for the sequencer and rollup namespaces starting from the celestia_base_block_height configured the rollup. It would then verify all sequencer metadata items retrieved in this way against the Sequencer, even if the metadata was for old blocks (meaning blocks with a height below the next firm height expected by the rollup).

Changes

  • Check the Sequencer height of each sequencer metadata item received from Celestia against the next expected firm height; drop the item without verification if its less than the expected height

Testing

  • Cannot yet be tested programatically, but the blackbox tests have been updated to contain several sequencer blocks per Celestia blob. Running the following one can observe a specific line indicating that the logic works:
TEST_LOG=1 RUST_LOG=debug cargo test firm_only::skips_already_executed_heights
...
2024-05-29T13:00:48.704019Z  INFO execute:verify_metadata: astria_conductor::celestia::verify: dropping Sequencer metadata item without verifying against Sequencer because its height is below the next expected firm height next_expected_firm_sequencer_height=7 sequencer_height_in_metadata=6 celestia_height=1 rollup_namespace=AAAAAAAAAAAAAAAAAAAAAAAAACoqKioqKioqKio= sequencer_namespace=AAAAAAAAAAAAAAAAAAAAAAAAAKDcd7eO77LbDQs=

@github-actions github-actions bot added the conductor pertaining to the astria-conductor crate label May 29, 2024
@SuperFluffy SuperFluffy marked this pull request as ready for review May 29, 2024 13:20
@SuperFluffy SuperFluffy requested a review from a team as a code owner May 29, 2024 13:20
@SuperFluffy SuperFluffy requested review from Fraser999 and noot and removed request for noot May 29, 2024 13:20
@SuperFluffy SuperFluffy changed the title feat(conductor): drop outdated block metadata feat(conductor): skip outdated block metadata May 29, 2024

#[must_use]
pub fn make_sequencer_block(height: u32) -> astria_core::sequencerblock::v1alpha1::SequencerBlock {
fn repeat_bytes_of_u32_as_array(val: u32) -> [u8; 32] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a unique block hash for each sequencer height.

@SuperFluffy SuperFluffy requested a review from joroshiba June 3, 2024 11:34
@SuperFluffy SuperFluffy added this pull request to the merge queue Jun 3, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 3, 2024
Comment on lines +112 to +118
if blob.height().value() < next_expected_firm_sequencer_height {
info!(
next_expected_firm_sequencer_height,
sequencer_height_in_metadata = blob.height().value(),
"dropping Sequencer metadata item without verifying against Sequencer because its \
height is below the next expected firm height"
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this guaranteed safe? ie are these blobs already guaranteed to be executed/verified as soft blocks already and we're sure that they are the same as the ones already verified?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is checking against the next expected firm height - this is independent of the soft blocks.

@SuperFluffy SuperFluffy force-pushed the superfluffy/skip-known-heights branch from a2fcaab to c6cb7a6 Compare June 4, 2024 11:29
@SuperFluffy SuperFluffy added this pull request to the merge queue Jun 4, 2024
Merged via the queue into main with commit 9925122 Jun 4, 2024
@SuperFluffy SuperFluffy deleted the superfluffy/skip-known-heights branch June 4, 2024 11:50
steezeburger added a commit that referenced this pull request Jun 5, 2024
* main:
  fix(charts): conductor configmap fix (#1146)
  feat(sequencer): add `allowed_fee_asset_ids` abci query and `sequencer_client` support (#1127)
  chore(conductor): release conductor 0.17 (#1139)
  feat: use macro to declare metric constants (#1129)
  refactor(merkle): remove source of panics in audit API (#1137)
  feat(conductor): skip outdated block metadata (#1120)
  refactor(sequencer): remove mint module (#1134)
  feat(bridge-withdrawer): add justfile (#1135)
  chore(chart): change evm back to latest on dev (#1132)
  feat(conductor, proto)!: celestia base heights in commitment state (#1121)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conductor pertaining to the astria-conductor crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments