v0.6: merge master branch

Cargo.toml

@@ -4,7 +4,7 @@ path = "src/main.rs"
 
 [package]
 name = "polkadot"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 build = "build.rs"
 edition = "2018"

README.adoc

@@ -12,9 +12,9 @@ In 2017 we split our implementation of "Polkadot" from its platform-level compon
 
 We are actively building both Substrate and Polkadot, but things will be a little odd for a while.  If you see "substrate" and are wondering why you need it for Polkadot, now you know.
 
-To connect on the "Kusama" canary network, you will want the `v0.5` code which is in this **Polkadot** repo. To play on the ("Alexander") testnet, you'll want the PoC-4 code instead.  Note that PoC-3 uses the Alexander testnet, but will not be able to sync to the latest block.
+To connect on the "Kusama" canary network, you will want the `v0.6` code, which is in this **Polkadot** repo. To play on the ("Alexander") testnet, you'll want the PoC-4 code instead.  Note that PoC-3 uses the Alexander testnet, but will not be able to sync to the latest block.
 
-* **Kusama** is in this link:https://github.com/paritytech/polkadot/tree/v0.5[**Polkadot**] repo branch `v0.5`.
+* **Kusama** is in this link:https://github.com/paritytech/polkadot/tree/v0.6[**Polkadot**] repo branch `v0.6`.
 
 * **Polkadot PoC-4 "Alexander"** is in this link:https://github.com/paritytech/polkadot/tree/v0.4[**Polkadot**] repo branch `v0.4`.
 
@@ -48,7 +48,7 @@ rustup update
 Build Kusama by cloning this repository and running the following commands from the root directory of the repo:
 
 ```bash
-git checkout v0.5
+git checkout v0.6
 ./scripts/init.sh
 cargo build --release
 ```
@@ -59,6 +59,8 @@ Connect to the global Kusama canary network by default by running:
 ./target/release/polkadot
 ```
 
+You can see your node on link:https://telemetry.polkadot.io/#list/Kusama%20CC2[telemetry].
+
 === Install PoC-4 on "Alexander" Testnet
 
 Build Polkadot PoC-4 by cloning this repository and running the following commands from the root directory of the repo:

SECURITY.md

@@ -0,0 +1,101 @@
+# Security Policy
+
+Parity Technologies is committed to resolving security vulnerabilities in our software quickly and carefully. We take the necessary steps to minimize risk, provide timely information, and deliver vulnerability fixes and mitigations required to address security issues.
+
+## Reporting a Vulnerability
+
+Security vulnerabilities in Parity software should be reported by email to [email protected]. If you think your report might be eligible for the Parity Bug Bounty Program, your email should be sent to [email protected]. 
+
+Your report should include the following:
+
+- your name
+- description of the vulnerability
+- attack scenario (if any)
+- components
+- reproduction
+- other details
+
+Try to include as much information in your report as you can, including a description of the vulnerability, its potential impact, and steps for reproducing it. Be sure to use a descriptive subject line.
+
+You'll receive a response to your email within two business days indicating the next steps in handling your report. We encourage finders to use encrypted communication channels to protect the confidentiality of vulnerability reports. You can encrypt your report using our public key. This key is [on MIT's key server](https://pgp.mit.edu/pks/lookup?op=get&search=0x5D0F03018D07DE73) server and reproduced below.
+
+After the initial reply to your report, our team will endeavor to keep you informed of the progress being made towards a fix. These updates will be sent at least every five business days. 
+
+Thank you for taking the time to responsibly disclose any vulnerabilities you find.
+
+## Responsible Investigation and Reporting
+
+Responsible investigation and reporting includes, but isn't limited to, the following:
+
+- Don't violate the privacy of other users, destroy data, etc.
+- Don’t defraud or harm Parity Technologies Ltd or its users during your research; you should make a good faith effort to not interrupt or degrade our services.
+- Don't target our physical security measures, or attempt to use social engineering, spam, distributed denial of service (DDOS) attacks, etc.
+- Initially report the bug only to us and not to anyone else.
+- Give us a reasonable amount of time to fix the bug before disclosing it to anyone else, and give us adequate written warning before disclosing it to anyone else.
+- In general, please investigate and report bugs in a way that makes a reasonable, good faith effort not to be disruptive or harmful to us or our users. Otherwise your actions might be interpreted as an attack rather than an effort to be helpful.
+
+## Bug Bounty Program
+
+Our Bug Bounty Program allows us to recognise and reward members of the Parity community for helping us find and address significant bugs, in accordance with the terms of the Parity Bug Bounty Program. A detailed description on eligibility, rewards, legal information and terms & conditions for contributors can be found on [our website](https://paritytech.io/bug-bounty.html). 
+
+
+
+
+
+
+## Plaintext PGP Key
+
+```
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+
+mQINBF0vHwQBEADKui4qAo4bzdzRhMm+uhUpYGf8jjjmET3zJ8kKQIpp6JTsV+HJ
+6m1We0QYeMRXoOYH1xVHBf2zNCuHS0nSQdUCQA7SHWsPB05STa2hvlR7fSdQnCCp
+gnLOJWXvvedlRDIAhvqI6cwLdUlXgVSKEwrwmrpiBhh4NxI3qX+LyIa+Ovkchu2S
+d/YCnE4GqojSGRfJYiGwe2N+sF7OfaoKhQuTrtdDExHrMU4cWnTXW2wyxTr4xkj9
+jS2WeLVZWflvkDHT8JD9N6jNxBVEF/Qvjk83zI0kCOzkhek8x+YUgfLq3/rHOYbX
+3pW21ccHYPacHjHWvKE+xRebjeEhJ4KxKHfCVjQcxybwDBqDka1AniZt4CQ7UORf
+MU/ue2oSZ9nNg0uMdb/0AbQPZ04OlMcYPAPWzFL08nVPox9wT9uqlL6JtcOeC90h
+oOeDmfgwmjMmdwWTRgt9qQjcbgXzVvuAzIGbzj1X3MdLspWdHs/d2+US4nji1TkN
+oYIW7vE+xkd3aB+NZunIlm9Rwd/0mSgDg+DaNa5KceOLhq0/qKgcXC/RRU29I8II
+tusRoR/oesGJGYTjh4k6PJkG+nvDPsoQrwYT44bhnniS1xYkxWYXF99JFI7LgMdD
+e1SgKeIDVpvm873k82E6arp5655Wod1XOjaXBggCwFp84eKcEZEN+1qEWwARAQAB
+tClQYXJpdHkgU2VjdXJpdHkgVGVhbSA8c2VjdXJpdHlAcGFyaXR5LmlvPokCVAQT
+AQoAPhYhBJ1LK264+XFW0ZZpqf8IEtSRuWeYBQJdLx8EAhsDBQkDwmcABQsJCAcC
+BhUKCQgLAgQWAgMBAh4BAheAAAoJEP8IEtSRuWeYL84QAI6NwnwS561DWYYRAd4y
+ocGPr3CnwFSt1GjkSkRy3B+tMhzexBg1y7EbLRUefIrO4LwOlywtRk8tTRGgEI4i
+5xRLHbOkeolfgCFSpOj5d8cMKCt5HEIv18hsv6dkrzlSYA5NLX/GRBEh3F/0sGny
+vCXapfxa1cx72sU7631JBK7t2Tf+MfwxdfyFZ9TI9WdtP5AfVjgTkIVkEDFcZPTc
+n3CYXqTYFIBCNUD8LP4iTi3xUt7pTGJQQoFT8l15nJCgzRYQ+tXpoTRlf+/LtXmw
+6iidPV87E06jHdK9666rBouIabAtx7i0/4kwo+bSZ8DiSKRUaehiHGd212HSEmdF
+jxquWE4pEzoUowYznhSIfR+WWIqRBHxEYarP4m98Hi+VXZ7Fw1ytzO8+BAKnLXnj
+2W2+T9qJks5gqVEoaWNnqpvya6JA11QZvZ0w7Om2carDc2ILNm2Xx9J0mRUye8P0
+KxcgqJuKNGFtugebQAsXagkxOKsdKna1PlDlxEfTf6AgI3ST8qSiMAwaaIMB/REF
+VKUapGoslQX4tOCjibI2pzEgE//D8NAaSVu2A9+BUcFERdZRxsI7fydIXNeZ2R46
+N2qfW+DP3YR/14QgdRxDItEavUoE1vByRXwIufKAkVemOZzIoFXKFsDeXwqTVW5i
+6CXu6OddZ3QHDiT9TEbRny4QuQINBF0vKCwBEACnP5J7LEGbpxNBrPvGdxZUo0YA
+U8RgeKDRPxJTvMo27V1IPZGaKRCRq8LBfg/eHhqZhQ7SLJBjBljd8kuT5dHDBTRe
+jE1UIOhmnlSlrEJjAmpVO08irlGpq1o+8mGcvkBsR0poCVjeNeSnwYfRnR+c3GK5
+Er6/JRqfN4mJvnEC9/Pbm6C7ql6YLKxC3yqzF97JL5brbbuozrW7nixY/yAI8619
+VlBIMP7PAUbGcnSQyuV5b/Wr2Sgr6NJclnNSLjh2U9/Du6w/0tDGlMBts8HjRnWJ
+BXbkTdQKCTaqgK68kTKSiN1/x+lynxHC2AavMpH/08Kopg2ZCzJowMKIgcB+4Z/I
+DJKZWHWKumhaZMGXcWgzgcByog9IpamuROEZFJNEUAFf7YIncEckPSif4looiOdS
+VurKZGvYXXaGSsZbGgHxI5CWu7ZxMdLBLvtOcCYmRQrG+g/h+PGU5BT0bNAfNTkm
+V3/n1B/TWbpWRmB3AwT2emQivXHkaubGI0VivhaO43AuI9JWoqiMqFtxbuTeoxwD
+xlu2Dzcp0v+AR4T5cIG9D5/+yiPc25aIY7cIKxuNFHIDL4td5fwSGC7vU6998PIG
+2Y48TGBnw7zpEfDfMayqAeBjX0YU6PTNsvS5O6bP3j4ojTOUYD7Z8QdCvgISDID3
+WMGAdmSwmCRvsQ/OJwARAQABiQI8BBgBCgAmFiEEnUsrbrj5cVbRlmmp/wgS1JG5
+Z5gFAl0vKCwCGwwFCQB2pwAACgkQ/wgS1JG5Z5hdbw//ZqR+JcWm59NUIHjauETJ
+sYDYhcAfa3txTacRn5uPz/TQiTd7wZ82+G8Et0ZnpEHy6eWyBqHpG0hiPhFBzxjY
+nhjHl8jJeyo2mQIVJhzkL58BHBZk8WM2TlaU7VxZ6TYOmP2y3qf6FD6mCcrQ4Fml
+E9f0lyVUoI/5Zs9oF0izRk8vkwaY3UvLM7XEY6nM8GnFG8kaiZMYmx26Zo7Uz31G
+7EGGZFsrVDXfNhSJyz79Gyn+Lx9jOTdoR0sH/THYIIosE83awMGE6jKeuDYTbVWu
++ZtHQef+pRteki3wvNLJK+kC1y3BtHqDJS9Lqx0s8SCiVozlC+fZfC9hCtU7bXJK
+0UJZ4qjSvj6whzfaNgOZAqJpmwgOnd8W/3YJk1DwUeX98FcU38MR23SOkx2EDdDE
+77Kdu62vTs/tLmOTuyKBvYPaHaYulYjQTxurG+o8vhHtaL87ARvuq+83dj+nO5z3
+5O9vkcVJYWjOEnJe7ZvCTxeLJehpCmHIbyUuDx5P24MWVbyXOxIlxNxTqlub5GlW
+rQF6Qsa/0k9TRk7Htbct6fAA0/VahJS0g096MrTH8AxBXDNE8lIoNeGikVlaxK9Z
+S+aannlWYIJymZ4FygIPPaRlzhAoXBuJd8OaR5giC7dS1xquxKOiQEXTGsLeGFaI
+BZYiIhW7GG4ozvKDqyNm4eg=
+=yKcB
+-----END PGP PUBLIC KEY BLOCK-----
+```

availability-store/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "polkadot-availability-store"
 description = "Persistent database for parachain data"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 edition = "2018"
 

cli/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "polkadot-cli"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 description = "Polkadot node implementation in Rust."
 edition = "2018"

collator/Cargo.toml

@@ -1,13 +1,13 @@
 [package]
 name = "polkadot-collator"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 description = "Collator node implementation"
 edition = "2018"
 
 [dependencies]
 futures = "0.1.17"
-futures03 = { package = "futures-preview", version = "0.3.0-alpha.19", features = ["compat"] }
+futures03 = { package = "futures-preview", version = "0.3.0-alpha.18", features = ["compat"] }
 client = { package = "substrate-client", git = "https://github.com/paritytech/substrate", branch = "polkadot-master" }
 primitives = { package = "substrate-primitives", git = "https://github.com/paritytech/substrate", branch = "polkadot-master" }
 substrate-network = { git = "https://github.com/paritytech/substrate", branch = "polkadot-master" }

collator/src/lib.rs

@@ -51,25 +51,25 @@ use std::time::Duration;
 
 use futures::{future, Stream, Future, IntoFuture};
 use futures03::{TryStreamExt as _, StreamExt as _};
-use log::{info, warn};
+use log::{warn, error};
 use client::BlockchainEvents;
-use primitives::Pair;
+use primitives::{Pair, Blake2Hasher};
 use polkadot_primitives::{
 	BlockId, Hash, Block,
 	parachain::{
-		self, BlockData, DutyRoster, HeadData, ConsolidatedIngress, Message, Id as ParaId, OutgoingMessages,
-		PoVBlock, Status as ParachainStatus, ValidatorId, CollatorPair,
+		self, BlockData, DutyRoster, HeadData, ConsolidatedIngress, Message, Id as ParaId,
+		OutgoingMessages, PoVBlock, Status as ParachainStatus, ValidatorId, CollatorPair,
 	}
 };
 use polkadot_cli::{
-	Worker, IntoExit, ProvideRuntimeApi, TaskExecutor, AbstractService,
-	CustomConfiguration, ParachainHost,
+	Worker, IntoExit, ProvideRuntimeApi, AbstractService, CustomConfiguration, ParachainHost,
 };
 use polkadot_network::validation::{LeafWorkParams, ValidationNetwork};
 use polkadot_network::{PolkadotNetworkService, PolkadotProtocol};
+use polkadot_runtime::RuntimeApi;
 use tokio::timer::Timeout;
 
-pub use polkadot_cli::VersionInfo;
+pub use polkadot_cli::{VersionInfo, TaskExecutor};
 pub use polkadot_network::validation::Incoming;
 pub use polkadot_validation::SignedStatement;
 pub use polkadot_primitives::parachain::CollatorId;
@@ -128,13 +128,24 @@ impl<R: fmt::Display> fmt::Display for Error<R> {
 	}
 }
 
+/// The Polkadot client type.
+pub type PolkadotClient<B, E> = client::Client<B, E, Block, RuntimeApi>;
+
 /// Something that can build a `ParachainContext`.
 pub trait BuildParachainContext {
 	/// The parachain context produced by the `build` function.
 	type ParachainContext: self::ParachainContext;
 
 	/// Build the `ParachainContext`.
-	fn build(self, network: Arc<dyn Network>) -> Result<Self::ParachainContext, ()>;
+	fn build<B, E>(
+		self,
+		client: Arc<PolkadotClient<B, E>>,
+		task_executor: TaskExecutor,
+		network: Arc<dyn Network>,
+	) -> Result<Self::ParachainContext, ()>
+		where
+			B: client::backend::Backend<Block, Blake2Hasher> + 'static,
+			E: client::CallExecutor<Block, Blake2Hasher> + Clone + Send + Sync + 'static;
 }
 
 /// Parachain context needed for collation.
@@ -147,7 +158,7 @@ pub trait ParachainContext: Clone {
 	/// Produce a candidate, given the relay parent hash, the latest ingress queue information
 	/// and the last parachain head.
 	fn produce_candidate<I: IntoIterator<Item=(ParaId, Message)>>(
-		&self,
+		&mut self,
 		relay_parent: Hash,
 		status: ParachainStatus,
 		ingress: I,
@@ -174,7 +185,7 @@ pub fn collate<'a, R, P>(
 	local_id: ParaId,
 	parachain_status: ParachainStatus,
 	relay_context: R,
-	para_context: P,
+	mut para_context: P,
 	key: Arc<CollatorPair>,
 )
 	-> impl Future<Item=(parachain::Collation, OutgoingMessages), Error=Error<R::Error>> + 'a
@@ -289,10 +300,18 @@ impl<P, E> Worker for CollationNode<P, E> where
 	}
 
 	fn work<S, SC, B, CE>(self, service: &S, task_executor: TaskExecutor) -> Self::Work
-	where S: AbstractService<Block = polkadot_service::Block, RuntimeApi = polkadot_service::RuntimeApi, Backend = B, SelectChain = SC, NetworkSpecialization = PolkadotProtocol, CallExecutor = CE>,
-		SC: polkadot_service::SelectChain<polkadot_service::Block> + 'static,
-		B: polkadot_service::Backend<polkadot_service::Block, polkadot_service::Blake2Hasher> + 'static,
-		CE: polkadot_service::CallExecutor<polkadot_service::Block, polkadot_service::Blake2Hasher> + Clone + Send + Sync + 'static
+	where
+		S: AbstractService<
+			Block = Block,
+			RuntimeApi = RuntimeApi,
+			Backend = B,
+			SelectChain = SC,
+			NetworkSpecialization = PolkadotProtocol,
+			CallExecutor = CE,
+		>,
+		SC: polkadot_service::SelectChain<Block> + 'static,
+		B: client::backend::Backend<Block, Blake2Hasher> + 'static,
+		CE: client::CallExecutor<Block, Blake2Hasher> + Clone + Send + Sync + 'static
 	{
 		let CollationNode { build_parachain_context, exit, para_id, key } = self;
 		let client = service.client();
@@ -301,7 +320,7 @@ impl<P, E> Worker for CollationNode<P, E> where
 		let select_chain = if let Some(select_chain) = service.select_chain() {
 			select_chain
 		} else {
-			info!("The node cannot work because it can't select chain.");
+			error!("The node cannot work because it can't select chain.");
 			return Box::new(future::err(()));
 		};
 
@@ -334,13 +353,25 @@ impl<P, E> Worker for CollationNode<P, E> where
 			exit.clone(),
 			message_validator,
 			client.clone(),
-			task_executor,
+			task_executor.clone(),
 		));
 
-		let parachain_context = build_parachain_context.build(validation_network.clone()).unwrap();
+		let parachain_context = match build_parachain_context.build(
+			client.clone(),
+			task_executor,
+			validation_network.clone(),
+		) {
+			Ok(ctx) => ctx,
+			Err(()) => {
+				error!("Could not build the parachain context!");
+				return Box::new(future::err(()))
+			}
+		};
+
 		let inner_exit = exit.clone();
 		let work = client.import_notification_stream()
-			.map(|v| Ok::<_, ()>(v)).compat()
+			.map(|v| Ok::<_, ()>(v))
+			.compat()
 			.for_each(move |notification| {
 				macro_rules! try_fr {
 					($e:expr) => {
@@ -489,7 +520,7 @@ mod tests {
 		type ProduceCandidate = Result<(BlockData, HeadData, OutgoingMessages), InvalidHead>;
 
 		fn produce_candidate<I: IntoIterator<Item=(ParaId, Message)>>(
-			&self,
+			&mut self,
 			_relay_parent: Hash,
 			_status: ParachainStatus,
 			ingress: I,

docker/Dockerfile

@@ -1,4 +1,4 @@
-FROM phusion/baseimage:0.10.2 as builder
+FROM phusion/baseimage:0.11 as builder
 LABEL maintainer "[email protected]"
 LABEL description="This is the build stage for Polkadot. Here we create the binary."
 
@@ -17,7 +17,7 @@ RUN curl https://sh.rustup.rs -sSf | sh -s -- -y && \
 
 # ===== SECOND STAGE ======
 
-FROM phusion/baseimage:0.10.2
+FROM phusion/baseimage:0.11
 LABEL maintainer "[email protected]"
 LABEL description="This is the 2nd stage: a very small image where we copy the Polkadot binary."
 ARG PROFILE=release

erasure-coding/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "polkadot-erasure-coding"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 edition = "2018"
 

executor/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "polkadot-executor"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 description = "Polkadot node implementation in Rust."
 edition = "2018"

network/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "polkadot-network"
-version = "0.6.2"
+version = "0.6.3"
 authors = ["Parity Technologies <[email protected]>"]
 description = "Polkadot-specific networking protocol"
 edition = "2018"

network/src/gossip.rs

@@ -265,7 +265,8 @@ impl<F, P> ChainContext for (F, P) where
 		let leaf_id = BlockId::Hash(leaf);
 		let active_parachains = api.active_parachains(&leaf_id)?;
 
-		for para_id in active_parachains {
+		// TODO: https://github.com/paritytech/polkadot/issues/467
+		for (para_id, _) in active_parachains {
 			if let Some(ingress) = api.ingress(&leaf_id, para_id, None)? {
 				for (_height, _from, queue_root) in ingress.iter() {
 					with_queue_root(queue_root);

network/src/tests/validation.rs

@@ -30,7 +30,7 @@ use polkadot_primitives::{Block, BlockNumber, Hash, Header, BlockId};
 use polkadot_primitives::parachain::{
 	Id as ParaId, Chain, DutyRoster, ParachainHost, TargetedMessage,
 	ValidatorId, StructuredUnroutedIngress, BlockIngressRoots, Status,
-	FeeSchedule, HeadData,
+	FeeSchedule, HeadData, Retriable, CollatorId
 };
 use parking_lot::Mutex;
 use substrate_client::error::Result as ClientResult;
@@ -177,7 +177,7 @@ impl NetworkService for TestNetwork {
 struct ApiData {
 	validators: Vec<ValidatorId>,
 	duties: Vec<Chain>,
-	active_parachains: Vec<ParaId>,
+	active_parachains: Vec<(ParaId, Option<(CollatorId, Retriable)>)>,
 	ingress: HashMap<ParaId, StructuredUnroutedIngress>,
 }
 
@@ -279,7 +279,7 @@ impl ParachainHost<Block> for RuntimeApi {
 		_: ExecutionContext,
 		_: Option<()>,
 		_: Vec<u8>,
-	) -> ClientResult<NativeOrEncoded<Vec<ParaId>>> {
+	) -> ClientResult<NativeOrEncoded<Vec<(ParaId, Option<(CollatorId, Retriable)>)>>> {
 		Ok(NativeOrEncoded::Native(self.data.lock().active_parachains.clone()))
 	}