Skip to content

Handle unknown block root attestations - #5327

Merged
twoeths merged 10 commits into
tuyen/new_gossip_queuesfrom
tuyen/unknown_block_root_attestations
Apr 7, 2023
Merged

Handle unknown block root attestations#5327
twoeths merged 10 commits into
tuyen/new_gossip_queuesfrom
tuyen/unknown_block_root_attestations

Conversation

@twoeths

@twoeths twoeths commented Mar 31, 2023

Copy link
Copy Markdown
Member

Motivation

Right now if a gossip attestation is validated and has unknown block root, it'll be reprocessed once the block comes. This has some drawback:

  • Some redundancy to validate unknown block root attestations because we can extract it from the serialized and do the check first
  • The number of attestations to reprocess could be up to 16_000, it's bad for the event loop to reprocess all of them at once

Description

  • Extract the slot and root from Attestation first and put to a map of root/slot if the block has not come yet. Ignore the message if there are too many items in the waiting queue
  • Process them once block come
  • Remove waiting gossip messages if block hasn't come in same slot

@twoeths twoeths changed the title Tuyen/unknown block root attestations Handle unknown block root attestations Mar 31, 2023
@twoeths
twoeths force-pushed the tuyen/unknown_block_root_attestations branch from fe12837 to afec1c3 Compare April 4, 2023 07:27
@twoeths

twoeths commented Apr 4, 2023

Copy link
Copy Markdown
Member Author

Some metrics captured

Screenshot 2023-04-04 at 21 30 43

Screenshot 2023-04-04 at 21 32 22

Screenshot 2023-04-04 at 21 33 33

==> it'll be improved if we trigger block search once we receive an unknown block root attestation, that's to be done in a separate work once we merge new_gossip_queues branch to unstable

@twoeths
twoeths marked this pull request as ready for review April 4, 2023 14:35
@twoeths
twoeths requested a review from a team as a code owner April 4, 2023 14:35
Comment thread packages/beacon-node/src/network/processor/extractSlotRootFns.ts
Comment thread packages/beacon-node/src/network/processor/index.ts Outdated
Comment thread packages/beacon-node/src/network/processor/index.ts Outdated
Comment thread packages/beacon-node/src/util/sszBytes.ts Outdated
@twoeths
twoeths requested a review from dapplion April 5, 2023 06:24
Comment thread packages/beacon-node/src/network/processor/extractSlotRootFns.ts Outdated
Comment thread packages/beacon-node/src/util/sszBytes.ts Outdated
Comment thread packages/beacon-node/src/util/wrapError.ts Outdated

@dapplion dapplion left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@twoeths
twoeths merged commit 46ae0a3 into tuyen/new_gossip_queues Apr 7, 2023
@twoeths
twoeths deleted the tuyen/unknown_block_root_attestations branch April 7, 2023 07:19
twoeths added a commit that referenced this pull request Apr 7, 2023
* Add sszBytes utils

* Unknown Block Root queues

* Use Uint8Array.subarray()

* Merge tuyen/new_gossip_queues

* Avoid forEach pattern

* Process unknown attestations: yield to event loop

* Fix metrics

* Name variable and function as *attDataBase64 everywhere

* Handle invalid serialized data

* Check ssz data length of gossip messages
wemeetagain pushed a commit that referenced this pull request Apr 7, 2023
* Add sszBytes utils

* Unknown Block Root queues

* Use Uint8Array.subarray()

* Merge tuyen/new_gossip_queues

* Avoid forEach pattern

* Process unknown attestations: yield to event loop

* Fix metrics

* Name variable and function as *attDataBase64 everywhere

* Handle invalid serialized data

* Check ssz data length of gossip messages
@twoeths
twoeths restored the tuyen/unknown_block_root_attestations branch April 8, 2023 23:40
@wemeetagain

Copy link
Copy Markdown
Member

馃帀 This PR is included in v1.8.0 馃帀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants