From f7361d8125f6842d597c16e5c8696c072dc0d4b6 Mon Sep 17 00:00:00 2001 From: Andronik Date: Thu, 17 Nov 2022 17:12:26 +0100 Subject: [PATCH 1/4] guide: remove refences to outdated secondary checkers --- node/primitives/src/lib.rs | 2 +- roadmap/implementers-guide/src/glossary.md | 3 +-- roadmap/implementers-guide/src/runtime/README.md | 14 ++++++++------ roadmap/implementers-guide/src/types/approval.md | 2 +- roadmap/implementers-guide/src/types/candidate.md | 4 ++-- .../src/types/overseer-protocol.md | 2 +- runtime/parachains/src/paras/mod.rs | 2 +- 7 files changed, 15 insertions(+), 14 deletions(-) diff --git a/node/primitives/src/lib.rs b/node/primitives/src/lib.rs index e75181b900e9..f9403ea6c186 100644 --- a/node/primitives/src/lib.rs +++ b/node/primitives/src/lib.rs @@ -383,7 +383,7 @@ impl std::fmt::Debug for CollationGenerationConfig { pub struct AvailableData { /// The Proof-of-Validation of the candidate. pub pov: std::sync::Arc, - /// The persisted validation data needed for secondary checks. + /// The persisted validation data needed for approval checks. pub validation_data: PersistedValidationData, } diff --git a/roadmap/implementers-guide/src/glossary.md b/roadmap/implementers-guide/src/glossary.md index 8612d8834cb8..b6cc449e1819 100644 --- a/roadmap/implementers-guide/src/glossary.md +++ b/roadmap/implementers-guide/src/glossary.md @@ -2,6 +2,7 @@ Here you can find definitions of a bunch of jargon, usually specific to the Polkadot project. +- **Approval Checker:** A validator who has been randomly selected to perform validity checks on a parablock which is pending approval. - **BABE:** (Blind Assignment for Blockchain Extension). The algorithm validators use to safely extend the Relay Chain. See [the Polkadot wiki][0] for more information. - **Backable Candidate:** A Parachain Candidate which is backed by a majority of validators assigned to a given parachain. - **Backed Candidate:** A Backable Candidate noted in a relay-chain block @@ -31,11 +32,9 @@ exactly one downward message queue. - **PVF Prechecking:** This is the process of initially checking the PVF when it is first added. We attempt preparation of the PVF and make sure it succeeds within a given timeout. - **PVF Preparation:** This is the process of preparing the WASM blob and includes both prevalidation and compilation. As prevalidation is pretty minimal right now, preparation mostly consists of compilation. - **Relay Parent:** A block in the relay chain, referred to in a context where work is being done in the context of the state at this block. -- **Router:** The router module is a meta module that consists of three runtime modules responsible for routing messages between paras and the relay chain. The three separate runtime modules are: Dmp, Ump, Hrmp, each responsible for the respective part of message routing. - **Runtime:** The relay-chain state machine. - **Runtime Module:** See Module. - **Runtime API:** A means for the node-side behavior to access structured information based on the state of a fork of the blockchain. -- **Secondary Checker:** A validator who has been randomly selected to perform secondary approval checks on a parablock which is pending approval. - **Subsystem:** A long-running task which is responsible for carrying out a particular category of work. - **UMP:** (Upward Message Passing) A vertical message passing mechanism from a parachain to the relay chain. - **Validator:** Specially-selected node in the network who is responsible for validating parachain blocks and issuing attestations about their validity. diff --git a/roadmap/implementers-guide/src/runtime/README.md b/roadmap/implementers-guide/src/runtime/README.md index 178346e184f5..f1f9d6c950e2 100644 --- a/roadmap/implementers-guide/src/runtime/README.md +++ b/roadmap/implementers-guide/src/runtime/README.md @@ -14,16 +14,18 @@ There is some functionality of the relay chain relating to parachains that we al We will split the logic of the runtime up into these modules: -* Initializer: manage initialization order of the other modules. +* Initializer: manages initialization order of the other modules. * Shared: manages shared storage and configurations for other modules. -* Configuration: manage configuration and configuration updates in a non-racy manner. -* Paras: manage chain-head and validation code for parachains and parathreads. +* Configuration: manages configuration and configuration updates in a non-racy manner. +* Paras: manages chain-head and validation code for parachains and parathreads. * Scheduler: manages parachain and parathread scheduling as well as validator assignments. * Inclusion: handles the inclusion and availability of scheduled parachains and parathreads. -* Validity: handles secondary checks and dispute resolution for included, available parablocks. +* SessionInfo: manages various session keys of validators and other params stored per session. +* Disputes: handles dispute resolution for included, available parablocks. +* Slashing: handles slashing logic for concluded disputes. * HRMP: handles horizontal messages between paras. -* UMP: Handles upward messages from a para to the relay chain. -* DMP: Handles downward messages from the relay chain to the para. +* UMP: handles upward messages from a para to the relay chain. +* DMP: handles downward messages from the relay chain to the para. The [Initializer module](initializer.md) is special - it's responsible for handling the initialization logic of the other modules to ensure that the correct initialization order and related invariants are maintained. The other modules won't specify a on-initialize logic, but will instead expose a special semi-private routine that the initialization module will call. The other modules are relatively straightforward and perform the roles described above. diff --git a/roadmap/implementers-guide/src/types/approval.md b/roadmap/implementers-guide/src/types/approval.md index e85a625b0710..b58e0a8187e1 100644 --- a/roadmap/implementers-guide/src/types/approval.md +++ b/roadmap/implementers-guide/src/types/approval.md @@ -6,7 +6,7 @@ The public key of a keypair used by a validator for determining assignments to a ## `AssignmentCert` -An `AssignmentCert`, short for Assignment Certificate, is a piece of data provided by a validator to prove that they have been selected to perform secondary approval checks on an included candidate. +An `AssignmentCert`, short for Assignment Certificate, is a piece of data provided by a validator to prove that they have been selected to perform approval checks on an included candidate. These certificates can be checked in the context of a specific block, candidate, and validator assignment VRF key. The block state will also provide further context about the availability core states at that block. diff --git a/roadmap/implementers-guide/src/types/candidate.md b/roadmap/implementers-guide/src/types/candidate.md index b9d5900b7f17..43f72d13becc 100644 --- a/roadmap/implementers-guide/src/types/candidate.md +++ b/roadmap/implementers-guide/src/types/candidate.md @@ -76,7 +76,7 @@ struct CandidateDescriptor { collator: CollatorId, /// The blake2-256 hash of the persisted validation data. These are extra parameters /// derived from relay-chain state that influence the validity of the block which - /// must also be kept available for secondary checkers. + /// must also be kept available for approval checkers. persisted_validation_data_hash: Hash, /// The blake2-256 hash of the `pov-block`. pov_hash: Hash, @@ -100,7 +100,7 @@ Since this data is used to form inputs to the validation function, it needs to b Furthermore, the validation data acts as a way to authorize the additional data the collator needs to pass to the validation function. For example, the validation function can check whether the incoming messages (e.g. downward messages) were actually sent by using the data provided in the validation data using so called MQC heads. -Since the commitments of the validation function are checked by the relay-chain, secondary checkers can rely on the invariant that the relay-chain only includes para-blocks for which these checks have already been done. As such, there is no need for the validation data used to inform validators and collators about the checks the relay-chain will perform to be persisted by the availability system. +Since the commitments of the validation function are checked by the relay-chain, approval checkers can rely on the invariant that the relay-chain only includes para-blocks for which these checks have already been done. As such, there is no need for the validation data used to inform validators and collators about the checks the relay-chain will perform to be persisted by the availability system. The `PersistedValidationData` should be relatively lightweight primarily because it is constructed during inclusion for each candidate and therefore lies on the critical path of inclusion. diff --git a/roadmap/implementers-guide/src/types/overseer-protocol.md b/roadmap/implementers-guide/src/types/overseer-protocol.md index 4b9dc97c27e2..1541bca160ae 100644 --- a/roadmap/implementers-guide/src/types/overseer-protocol.md +++ b/roadmap/implementers-guide/src/types/overseer-protocol.md @@ -827,7 +827,7 @@ pub enum CandidateValidationMessage { /// /// This request doesn't involve acceptance criteria checking, therefore only useful for the /// cases where the validity of the candidate is established. This is the case for the typical - /// use-case: secondary checkers would use this request relying on the full prior checks + /// use-case: approval checkers would use this request relying on the full prior checks /// performed by the relay-chain. ValidateFromExhaustive( PersistedValidationData, diff --git a/runtime/parachains/src/paras/mod.rs b/runtime/parachains/src/paras/mod.rs index 3c5744b96546..c4fadcf5642b 100644 --- a/runtime/parachains/src/paras/mod.rs +++ b/runtime/parachains/src/paras/mod.rs @@ -659,7 +659,7 @@ pub mod pallet { /// Past code of parachains. The parachains themselves may not be registered anymore, /// but we also keep their code on-chain for the same amount of time as outdated code - /// to keep it available for secondary checkers. + /// to keep it available for approval checkers. #[pallet::storage] #[pallet::getter(fn past_code_meta)] pub(super) type PastCodeMeta = From a0cbf4cbe38bbcb8faf4855172b586e5bba620d8 Mon Sep 17 00:00:00 2001 From: Andronik Date: Fri, 18 Nov 2022 09:29:18 +0100 Subject: [PATCH 2/4] Update roadmap/implementers-guide/src/glossary.md --- roadmap/implementers-guide/src/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roadmap/implementers-guide/src/glossary.md b/roadmap/implementers-guide/src/glossary.md index b6cc449e1819..8cf279aadc9f 100644 --- a/roadmap/implementers-guide/src/glossary.md +++ b/roadmap/implementers-guide/src/glossary.md @@ -2,7 +2,7 @@ Here you can find definitions of a bunch of jargon, usually specific to the Polkadot project. -- **Approval Checker:** A validator who has been randomly selected to perform validity checks on a parablock which is pending approval. +- **Approval Checker:** A validator who randomly self-selects so to perform validity checks on a parablock which is pending approval. - **BABE:** (Blind Assignment for Blockchain Extension). The algorithm validators use to safely extend the Relay Chain. See [the Polkadot wiki][0] for more information. - **Backable Candidate:** A Parachain Candidate which is backed by a majority of validators assigned to a given parachain. - **Backed Candidate:** A Backable Candidate noted in a relay-chain block From 2716b2abe352985d3314eeb44ace6b573ba8f9ad Mon Sep 17 00:00:00 2001 From: Andronik Date: Fri, 18 Nov 2022 09:41:49 +0100 Subject: [PATCH 3/4] guide: remove refences to Fisherman --- .../implementers-guide/src/node/availability/README.md | 2 +- roadmap/implementers-guide/src/protocol-overview.md | 1 - roadmap/parachains.md | 10 +++++----- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/roadmap/implementers-guide/src/node/availability/README.md b/roadmap/implementers-guide/src/node/availability/README.md index 46ee4b204982..76bd6467e178 100644 --- a/roadmap/implementers-guide/src/node/availability/README.md +++ b/roadmap/implementers-guide/src/node/availability/README.md @@ -1,3 +1,3 @@ # Availability Subsystems -The availability subsystems are responsible for ensuring that Proofs of Validity of backed candidates are widely available within the validator set, without requiring every node to retain a full copy. They accomplish this by broadly distributing erasure-coded chunks of the PoV, keeping track of which validator has which chunk by means of signed bitfields. They are also responsible for reassembling a complete PoV when required, e.g. when a fisherman reports a potentially invalid block. +The availability subsystems are responsible for ensuring that Proofs of Validity of backed candidates are widely available within the validator set, without requiring every node to retain a full copy. They accomplish this by broadly distributing erasure-coded chunks of the PoV, keeping track of which validator has which chunk by means of signed bitfields. They are also responsible for reassembling a complete PoV when required, e.g. when an approval checker needs to validate a parachain block. diff --git a/roadmap/implementers-guide/src/protocol-overview.md b/roadmap/implementers-guide/src/protocol-overview.md index 77b3a7448c44..fa5a866e6121 100644 --- a/roadmap/implementers-guide/src/protocol-overview.md +++ b/roadmap/implementers-guide/src/protocol-overview.md @@ -8,7 +8,6 @@ First, it's important to go over the main actors we have involved in this protoc 1. Validators. These nodes are responsible for validating proposed parachain blocks. They do so by checking a Proof-of-Validity (PoV) of the block and ensuring that the PoV remains available. They put financial capital down as "skin in the game" which can be slashed (destroyed) if they are proven to have misvalidated. 1. Collators. These nodes are responsible for creating the Proofs-of-Validity that validators know how to check. Creating a PoV typically requires familiarity with the transaction format and block authoring rules of the parachain, as well as having access to the full state of the parachain. -1. Fishermen. These are user-operated, permissionless nodes whose goal is to catch misbehaving validators in exchange for a bounty. Collators and validators can behave as Fishermen too. Fishermen aren't necessary for security, and aren't covered in-depth by this document. This implies a simple pipeline where collators send validators parachain blocks and their requisite PoV to check. Then, validators validate the block using the PoV, signing statements which describe either the positive or negative outcome, and with enough positive statements, the block can be noted on the relay-chain. Negative statements are not a veto but will lead to a dispute, with those on the wrong side being slashed. If another validator later detects that a validator or group of validators incorrectly signed a statement claiming a block was valid, then those validators will be _slashed_, with the checker receiving a bounty. diff --git a/roadmap/parachains.md b/roadmap/parachains.md index 89e8fdaf3892..e724c37ae99a 100644 --- a/roadmap/parachains.md +++ b/roadmap/parachains.md @@ -20,11 +20,11 @@ This section contains various sub-projects and the features that make them up. Category: Networking -Validators assigned to a parachain need a way to discover and connect to collators in order to get fresh parachain blocks to validate. +Validators assigned to a parachain (backing group) need a way to discover and connect to collators in order to get fresh parachain blocks to validate. Collators need to discover and connect to validators in order to submit parachain blocks. -Fishermen need to talk to validators and collators to fetch available data and circulate reports. +Approval checkers need to talk to validators and collators to fetch available data and circulate reports. Some connections are long-lived, some are just for a single request. @@ -74,7 +74,7 @@ For each potential, considered parachain block, perform an erasure-coding of the Category: Networking -A black-box networking component for validators or fishermen on a parachain to obtain the PoV block referenced by hash in an attestation, for the purpose of validating. When fetching "current" PoV blocks (close to the head of the chain, or relating to the block currently being built), this should be fast. When fetching "old" PoV blocks, it should be possible and fall back on recovering from the availability erasure-coding. +A black-box networking component for validators on a parachain to obtain the PoV block referenced by hash in an attestation, for the purpose of validating. When fetching "current" PoV blocks (close to the head of the chain, or relating to the block currently being built), this should be fast. When fetching "old" PoV blocks, it should be possible and fall back on recovering from the availability erasure-coding. #### *Parathread Auction Voting* @@ -155,7 +155,7 @@ Runtime logic for paras to open and close channels by putting down a deposit. Th Category: Runtime -In Polkadot, a bad parachain group can force inclusion of an invalid or unavailable parachain block. It is the job of fishermen to detect those blocks and report them to the runtime. This item is about the report handler +In Polkadot, a bad parachain group can force inclusion of an invalid or unavailable parachain block. It is the job of approval checkers to detect those blocks and report them to the runtime. This item is about the report handler The W3F-research writeup on availability/validity provides a high-level view of the dispute resolution process: [Availability and Validity — Research at W3F](https://research.web3.foundation/en/latest/polkadot/Availability_and_Validity.html) @@ -173,7 +173,7 @@ Runtime handlers that take two conflicting votes as arguments and slash the offe Category: Node -This code-path is also taken by validators who self-select based on VRF [Availability and Validity — Research at W3F](https://research.web3.foundation/en/latest/polkadot/Availability_and_Validity.html). Validators and fishermen will select parachain blocks to re-validate. In these steps: +This code-path is also taken by validators who self-select based on VRF [Availability and Validity — Research at W3F](https://research.web3.foundation/en/latest/polkadot/Availability_and_Validity.html). Validators will select parachain blocks to re-validate. In these steps: * Attempt to recover the PoV block, falling back on the erasure-coding. If not available, issue report. * Attempt to validate the PoV block. If invalid, issue report. From 1b6be205f24bb3b03038126191aea9bd7952c583 Mon Sep 17 00:00:00 2001 From: Andronik Date: Fri, 18 Nov 2022 09:57:48 +0100 Subject: [PATCH 4/4] revert changes to roadmap/parachains.md --- roadmap/parachains.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/roadmap/parachains.md b/roadmap/parachains.md index e724c37ae99a..89e8fdaf3892 100644 --- a/roadmap/parachains.md +++ b/roadmap/parachains.md @@ -20,11 +20,11 @@ This section contains various sub-projects and the features that make them up. Category: Networking -Validators assigned to a parachain (backing group) need a way to discover and connect to collators in order to get fresh parachain blocks to validate. +Validators assigned to a parachain need a way to discover and connect to collators in order to get fresh parachain blocks to validate. Collators need to discover and connect to validators in order to submit parachain blocks. -Approval checkers need to talk to validators and collators to fetch available data and circulate reports. +Fishermen need to talk to validators and collators to fetch available data and circulate reports. Some connections are long-lived, some are just for a single request. @@ -74,7 +74,7 @@ For each potential, considered parachain block, perform an erasure-coding of the Category: Networking -A black-box networking component for validators on a parachain to obtain the PoV block referenced by hash in an attestation, for the purpose of validating. When fetching "current" PoV blocks (close to the head of the chain, or relating to the block currently being built), this should be fast. When fetching "old" PoV blocks, it should be possible and fall back on recovering from the availability erasure-coding. +A black-box networking component for validators or fishermen on a parachain to obtain the PoV block referenced by hash in an attestation, for the purpose of validating. When fetching "current" PoV blocks (close to the head of the chain, or relating to the block currently being built), this should be fast. When fetching "old" PoV blocks, it should be possible and fall back on recovering from the availability erasure-coding. #### *Parathread Auction Voting* @@ -155,7 +155,7 @@ Runtime logic for paras to open and close channels by putting down a deposit. Th Category: Runtime -In Polkadot, a bad parachain group can force inclusion of an invalid or unavailable parachain block. It is the job of approval checkers to detect those blocks and report them to the runtime. This item is about the report handler +In Polkadot, a bad parachain group can force inclusion of an invalid or unavailable parachain block. It is the job of fishermen to detect those blocks and report them to the runtime. This item is about the report handler The W3F-research writeup on availability/validity provides a high-level view of the dispute resolution process: [Availability and Validity — Research at W3F](https://research.web3.foundation/en/latest/polkadot/Availability_and_Validity.html) @@ -173,7 +173,7 @@ Runtime handlers that take two conflicting votes as arguments and slash the offe Category: Node -This code-path is also taken by validators who self-select based on VRF [Availability and Validity — Research at W3F](https://research.web3.foundation/en/latest/polkadot/Availability_and_Validity.html). Validators will select parachain blocks to re-validate. In these steps: +This code-path is also taken by validators who self-select based on VRF [Availability and Validity — Research at W3F](https://research.web3.foundation/en/latest/polkadot/Availability_and_Validity.html). Validators and fishermen will select parachain blocks to re-validate. In these steps: * Attempt to recover the PoV block, falling back on the erasure-coding. If not available, issue report. * Attempt to validate the PoV block. If invalid, issue report.