Repository navigation
test: data column sidecar gossip validation spec tests #9430
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: nflaig/deneb-gossip-spec-tests
Are you sure you want to change the base?
Changes from 3 commits
2f877d3
3435d9b
589caee
1dfcc2c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,7 +10,7 @@ import { | |
| getBlockHeaderProposerSignatureSetByHeaderSlot, | ||
| getBlockHeaderProposerSignatureSetByParentStateSlot, | ||
| } from "@lodestar/state-transition"; | ||
| import {DataColumnSidecar, Root, Slot, SubnetID, fulu, gloas, ssz} from "@lodestar/types"; | ||
| import {DataColumnSidecar, Root, RootHex, Slot, SubnetID, fulu, gloas, ssz} from "@lodestar/types"; | ||
| import {byteArrayEquals, toRootHex, verifyMerkleBranch} from "@lodestar/utils"; | ||
| import {BeaconMetrics} from "../../metrics/metrics/beacon.js"; | ||
| import {Metrics} from "../../metrics/metrics.js"; | ||
|
|
@@ -35,6 +35,24 @@ export async function validateGossipFuluDataColumnSidecar( | |
| metrics: Metrics | null | ||
| ): Promise<void> { | ||
| const blockHeader = dataColumnSidecar.signedBlockHeader.message; | ||
|
|
||
| // [IGNORE] The sidecar is the first sidecar for the tuple (block_header.slot, block_header.proposer_index, | ||
| // sidecar.index). Spec orders this check first, before any expensive verification. | ||
| if ( | ||
| chain.seenBlockInputCache.isSeenDataColumnSidecar( | ||
| blockHeader.slot, | ||
| blockHeader.proposerIndex, | ||
| dataColumnSidecar.index | ||
| ) | ||
| ) { | ||
| throw new DataColumnSidecarGossipError(GossipAction.IGNORE, { | ||
| code: DataColumnSidecarErrorCode.ALREADY_SEEN_TUPLE, | ||
| slot: blockHeader.slot, | ||
| proposerIndex: blockHeader.proposerIndex, | ||
| columnIndex: dataColumnSidecar.index, | ||
| }); | ||
| } | ||
|
|
||
| const blockRootHex = toRootHex(ssz.phase0.BeaconBlockHeader.hashTreeRoot(blockHeader)); | ||
|
|
||
| // 1) [REJECT] The sidecar is valid as verified by verify_data_column_sidecar | ||
|
|
@@ -104,9 +122,24 @@ export async function validateGossipFuluDataColumnSidecar( | |
| }); | ||
| } | ||
|
|
||
| // getBlockSlotState also checks for whether the current finalized checkpoint is an ancestor of the block. | ||
| // As a result, we throw an IGNORE (whereas the spec says we should REJECT for this scenario). | ||
| // this is something we should change this in the future to make the code airtight to the spec. | ||
| // 9) [REJECT] The current finalized_checkpoint is an ancestor of the sidecar's block | ||
| // -- i.e. get_checkpoint_block(store, block_header.parent_root, store.finalized_checkpoint.epoch) | ||
| // == store.finalized_checkpoint.root | ||
| const finalizedAncestorSlot = computeStartSlotAtEpoch(finalizedCheckpoint.epoch); | ||
| let ancestorRoot: RootHex | undefined; | ||
| try { | ||
| ancestorRoot = chain.forkChoice.getAncestor(parentRoot, finalizedAncestorSlot).blockRoot; | ||
| } catch { | ||
| // parent not in fork-choice or finalized slot before parent's chain start | ||
| } | ||
|
Comment on lines
+130
to
+135
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If According to the consensus spec's let ancestorRoot: RootHex | undefined;
if (parentBlock.slot <= finalizedAncestorSlot) {
ancestorRoot = parentRoot;
} else {
try {
ancestorRoot = chain.forkChoice.getAncestor(parentRoot, finalizedAncestorSlot).blockRoot;
} catch {
// parent not in fork-choice or finalized slot before parent's chain start
}
}
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the careful read, but I think the premise here is incorrect.
if (block.slot <= ancestorSlot) {
return block;
}It only throws in two cases:
For
This pattern directly mirrors Spec reference: https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/fork-choice.md#get_ancestor No change needed here. |
||
| if (ancestorRoot !== finalizedCheckpoint.rootHex) { | ||
| throw new DataColumnSidecarGossipError(GossipAction.REJECT, { | ||
| code: DataColumnSidecarErrorCode.FINALIZED_NOT_ANCESTOR, | ||
| parentRoot, | ||
| finalizedRoot: finalizedCheckpoint.rootHex, | ||
| }); | ||
| } | ||
|
|
||
| // 7) [REJECT] The sidecar's block's parent passes validation. | ||
| const blockState = await chain.regen | ||
| .getBlockSlotState(parentBlock, blockHeader.slot, {dontTransferCache: true}, RegenCaller.validateGossipDataColumn) | ||
|
|
@@ -160,11 +193,6 @@ export async function validateGossipFuluDataColumnSidecar( | |
| chain.seenBlockInputCache.markVerifiedProposerSignature(blockHeader.slot, blockRootHex, signature); | ||
| } | ||
|
|
||
| // 9) [REJECT] The current finalized_checkpoint is an ancestor of the sidecar's block | ||
| // -- i.e. get_checkpoint_block(store, block_header.parent_root, store.finalized_checkpoint.epoch) | ||
| // == store.finalized_checkpoint.root | ||
| // Handled by 7) | ||
|
|
||
| // 10) [REJECT] The sidecar's kzg_commitments field inclusion proof is valid as verified by | ||
| // verify_data_column_sidecar_inclusion_proof | ||
| // TODO: Can cache result on (commitments, proof, header) in the future | ||
|
|
@@ -200,9 +228,7 @@ export async function validateGossipFuluDataColumnSidecar( | |
| kzgProofTimer?.(); | ||
| } | ||
|
|
||
| // 12) [IGNORE] The sidecar is the first sidecar for the tuple (block_header.slot, block_header.proposer_index, | ||
| // sidecar.index) with valid header signature, sidecar inclusion proof, and kzg proof | ||
| // -- Handled in seenGossipBlockInput | ||
| chain.seenBlockInputCache.markSeenDataColumnSidecar(blockHeader.slot, proposerIndex, dataColumnSidecar.index); | ||
| } | ||
|
|
||
| // SPEC FUNCTION | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,6 +36,7 @@ import {validateGossipAttesterSlashing} from "../../../src/chain/validation/atte | |
| import {validateGossipBlobSidecar} from "../../../src/chain/validation/blobSidecar.js"; | ||
| import {validateGossipBlock} from "../../../src/chain/validation/block.js"; | ||
| import {validateGossipBlsToExecutionChange} from "../../../src/chain/validation/blsToExecutionChange.js"; | ||
| import {validateGossipFuluDataColumnSidecar} from "../../../src/chain/validation/dataColumnSidecar.js"; | ||
| import {validateGossipProposerSlashing} from "../../../src/chain/validation/proposerSlashing.js"; | ||
| import {validateGossipSyncCommittee} from "../../../src/chain/validation/syncCommittee.js"; | ||
| import {validateSyncCommitteeGossipContributionAndProof} from "../../../src/chain/validation/syncCommitteeContributionAndProof.js"; | ||
|
|
@@ -162,6 +163,7 @@ const gossipTopicByHandler = { | |
| gossip_sync_committee_contribution_and_proof: GossipType.sync_committee_contribution_and_proof, | ||
| gossip_bls_to_execution_change: GossipType.bls_to_execution_change, | ||
| gossip_blob_sidecar: GossipType.blob_sidecar, | ||
| gossip_data_column_sidecar: GossipType.data_column_sidecar, | ||
| } as const satisfies Record<string, GossipType>; | ||
|
|
||
| export function isGossipValidationHandler(topicHandler: string): topicHandler is keyof typeof gossipTopicByHandler { | ||
|
|
@@ -741,6 +743,18 @@ async function validateMessageForTopic( | |
| break; | ||
| } | ||
|
|
||
| case GossipType.data_column_sidecar: { | ||
| const dataColumnSidecar = rejectOnInvalidSerializedBytes(() => ssz.fulu.DataColumnSidecar.deserialize(bytes)); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When these networking tests run for Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right that the Gloas Scope-deferring this for two reasons:
Tracking as a follow-up — I'll wire the per-fork dispatch when Gloas DCS gossip fixtures land in consensus-specs. |
||
| const parentRootHex = toRootHex(dataColumnSidecar.signedBlockHeader.message.parentRoot); | ||
|
|
||
| if (rejectedFailedBlockRoots.has(parentRootHex)) { | ||
| throw new GossipActionError(GossipAction.REJECT, {code: "SPEC_PARENT_BLOCK_FAILED"}); | ||
| } | ||
|
|
||
| await validateGossipFuluDataColumnSidecar(chain, dataColumnSidecar, Number(message.subnet_id ?? 0), null); | ||
| break; | ||
| } | ||
|
|
||
| default: | ||
| throw new Error(`Unknown gossip topic: ${topic}`); | ||
| } | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need this, doesn't the
BlockInputhandle seen columns already?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These two checks have different keys and the spec mandates the tuple check.
BlockInput.hasColumn(gossipHandlers.ts:354-366) is keyed by(blockRoot, columnIndex)— it short-circuits re-gossip of the same column for the same block (or columns we already fetched via reqresp /getBlobsV2).seenBlockInputCache.isSeenDataColumnSidecaris keyed by(slot, proposerIndex, columnIndex)and is the spec's first check. It catches equivocation: a proposer signing two different blocks at the same slot produces sidecars with the same(slot, proposer, index)but differentblockRoot, so the BlockInput-keyed check would miss the second one and we'd re-run KZG + propagate. The class comment atseenGossipBlockInput.ts:440-442calls this out.Same shape as the blob path (
isSeenBlobSidecar,blobSidecar.ts:182).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lodekeeper are you sure about this, if that's the case this seems like a bug in our current implementation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch — my "spec orders this first" framing was wrong. Re-reading the Deneb spec and Fulu spec, the IGNORE tuple check is listed after KZG and qualified by "with valid header signature, sidecar inclusion proof, and kzg proof" — i.e. "first validated tuple", not "first tuple seen".
So neither file has a literal spec-correctness bug:
markSeen*runs only after all checks pass (data column line 231, blob line 209). Good.verify_is_unknown_sidecar) for perf, then the late write under the spec-listed position. Seeconsensus-specs#3261for the related blob-spam discussion.This PR places the data column read check early;
blobSidecar.ts(on this branch) places it after KZG. Both are correctness-equivalent given the post-validation mark; the early position saves a BLS sig + inclusion proof + KZG verify on duplicate gossip.The two-key argument still holds:
BlockInput.hasColumncatches re-gossip of(blockRoot, columnIndex), seen-tuple catches equivocations on(slot, proposerIndex, columnIndex). So we still need the seen-tuple check somewhere. Happy to align both files to the early position in a follow-up if you prefer the perf path.