Genesis Boot core logic and CLI #31

jack-kearney · 2022-06-09T02:06:47Z

Also includes

Switch to Borsh serialization, removing all direct reliance on CBOR
Initial Standard Boot logic
Add envelope encryption using RSA and AES 256 CBC
Add Protocol Phase
Gate routes depending on the protocol phase

Primary follow up work

Add attestation doc verification to boot_genesis CLI command
CLI command for decrypting personal genesis outputs, and then writing them to disk as password protected values.
Comment out the entire genesis boot flow
Standard boot CLI
Better documentation for the envelope encryption and add a justification for why the particular encryption strategy was used.

emostov · 2022-06-10T05:28:16Z

qos-core/src/protocol/mod.rs

+	}
+
+	fn routes(&self) -> Vec<Box<ProtocolHandler>> {
+		match self.state.phase {


We gate the available routes based on phase

emostov · 2022-06-10T05:28:49Z

qos-core/src/protocol/mod.rs

+			provisioner,
+			pivot_file,
+			ephemeral_key_file,
+			phase: ProtocolPhase::WaitingForBootInstruction,


We can view/update the protocol phase from state

emostov · 2022-06-10T05:29:06Z

qos-core/src/protocol/mod.rs

+			pivot_file,
+			ephemeral_key_file,
+			phase: ProtocolPhase::WaitingForBootInstruction,
+			manifest: None,


We will always have access to the manifest from state

emostov · 2022-06-10T05:30:14Z

qos-core/src/protocol/mod.rs

+	IOError,
+	CryptoError,
+	InvalidManifestApproval(Approval),
+	NoMatchingRoute(ProtocolPhase),


We add some error types that we can propagate back to qos clients in protocol messages - hopefully we can get rid of all generic errors in protocol message responses

emostov · 2022-06-10T05:30:29Z

qos-core/src/protocol/genesis.rs

+	// to reconstruct the original Quorum Key?
+	//
+	// TODO: Disaster recovery logic!
+	// Maybe we can just accept 2 set configs, and one is the recovery set?``


emostov · 2022-06-10T05:30:56Z

qos-core/src/protocol/genesis.rs

+			member_outputs,
+			quorum_key: quorum_pair.public_key_to_der()?,
+			// TODO: generate N choose K recovery permutations
+			recovery_permutations: vec![],


Note I put this place holder in for the recovery permutations

qos-core/src/protocol/mod.rs

emostov · 2022-06-10T05:33:23Z

qos-core/src/protocol/mod.rs

+	fn routes(&self) -> Vec<Box<ProtocolHandler>> {
+		match self.state.phase {
+			ProtocolPhase::UnrecoverableError => {
+				vec![Box::new(handlers::status)]


Always include the status handler so we can at least get some insight.

In order for this status endpoint to always be useful, (and not blocked by some other ongoing actions,) we should consider making the enclave server async. I actually don't think it would be too hard. Main thing is we would need to make ProtocolState thread safe by adding a Mutex

Yeah, this is a really interesting topic. On the one hand, before pivoting I don't think the operations of the enclave will ever be complicated enough to warrant introducing this complexity. On the other hand, after pivoting we will almost certainly need the secure application to be async.

Let's discuss today

qos-core/src/protocol/mod.rs

emostov · 2022-06-10T05:38:18Z

qos-crypto/Cargo.toml

+openssl = "0.10.40"
+serde = { version = "1.0", default-features = false, features = ["derive", "std"] }
+serde_cbor = { version = "0.11", default-features = false, features = ["std"] }


We needed both of these to have std enabled. I think there was something in the other crate implicitly requiring serde with std enabled farther down the dep tree, so it was building fine. But when we built this crate in isolation we got issues because we didn't implicitly have the std feature

qos-crypto/src/lib.rs

emostov · 2022-06-10T05:39:27Z

qos-crypto/src/lib.rs

 }

 impl RsaPair {
+	pub fn generate() -> Result<Self, CryptoError> {
+		Rsa::generate(RSA_KEY_LEN)?.try_into()


Wrapped this up and replaced it everywhere so we always get the same key len. But need to look into how to add entropy

emostov · 2022-06-10T05:41:42Z

qos-crypto/src/lib.rs

+impl Deref for RsaPair {
+	type Target = Rsa<Private>;
+
+	fn deref(&self) -> &Self::Target {


This is good to know about and understand. Basically if you go like *rsa_pair, or long form rsa_pair.deref(), you will get a Rsa<Private>. Kind of magic, but super convenient since we get access to the underlying openssl methods defined on Rsa<Private>

emostov · 2022-06-10T05:42:44Z

qos-crypto/src/lib.rs

+	pub fn encrypt(&self, data: &[u8]) -> Result<Vec<u8>, CryptoError> {
+		let public_key_size = self.public_key.size() as usize;
+		// TODO: WTF?
+		if data.len() > public_key_size - 42 {


yeah - no idea why i found the data len had to be 42 less bytes then key size. I just found this from writing tests

Interesting! Definitely worth investigating I think :)

emostov · 2022-06-14T05:14:54Z

qos-client/src/attest/nitro/mod.rs

@@ -20,20 +20,21 @@ static AWS_NITRO_CERT_SIG_ALG: &[&webpki::SignatureAlgorithm] =

 /// Corresponds to `MockNsm` attestation document response. This time is
 /// valid for the mock and should only be used for testing.
-pub(crate) const MOCK_SECONDS_SINCE_EPOCH: u64 = 1652756400;
+#[cfg(any(feature = "mock", test))]
+pub const MOCK_SECONDS_SINCE_EPOCH: u64 = 1652756400;


Feature gated so it doesn't slip into production

emostov · 2022-06-14T05:15:13Z

qos-client/src/attest/nitro/mod.rs


 /// AWS Nitro root CA certificate.
 ///
 /// This should be validated against the checksum:
 /// `8cf60e2b2efca96c6a9e71e851d00c1b6991cc09eadbe64a6a1d1b1eb9faff7c`. This
 /// checksum and the certificate should be manually verified against
 /// https://docs.aws.amazon.com/enclaves/latest/user/verify-root.html.
-pub(crate) const AWS_ROOT_CERT: &'static [u8] =
+pub const AWS_ROOT_CERT_PEM: &'static [u8] =


This is public now, but we should at least check the checksum via CLI

qos-core/src/client.rs

emostov · 2022-06-14T22:15:57Z

qos-core/src/protocol/attestor/types.rs

@@ -0,0 +1,318 @@
+//! Types specific to AWS nitro enclave protocol implementation.
+use std::collections::BTreeSet;
+


All these types map 1 to 1 with aws_nitro_enclaves_nsm_api::api types. We do this so we can derive borsh methods

qos-core/src/protocol/boot.rs

…nto bootstrap-and-genesis

emostov · 2022-06-14T23:27:57Z

qos-client/src/cli.rs

+
+	// TODO: verify AWS_ROOT_CERT_PEM against a checksum
+	// TODO: verify PCRs
+	pub(super) fn boot_genesis(options: ClientOptions) {


Note that boot genesis implicitly creates the genesis set. This is opposed to the initial design of creating the genesis set and writing it to file in one command, and then reading+broadcasting the genesis set in another command.

Now the genesis set is an implementation detail the CLI user doesn't need to know about. Instead they just need to have a folder with the appropriate keys ready.

emostov · 2022-06-15T01:21:11Z

qos-test/tests/genesis.rs

+
+	// -- CLIENT Create 3 setup keys
+	// Make sure the directory keys are getting written to already exist.
+	let _ = std::fs::create_dir(key_dir);


This test is a good description of flow for genesis ceremony

jack-kearney and others added 14 commits June 8, 2022 22:06

Lots of changes, lots of broken things

28d49aa

Envelope encryption working

627a26f

Updates but things are broken

fd00488

Get envelope encrypt/decrypt working with cbor

80b7e86

Run fmt

5506754

Remove expects in qos-crypto

214d78a

Only use RsaPair::generate

a943be1

Test encrypt

82db50e

State gated routes

cefe64e

Write pivot after checking manifest

f56264a

Initial pass at creating genesis outputs

c6a7469

Factor out Genesis Output generation

c0a7ebe

fmt

a3c1df5

Addres some clippy warnings

f602f98

emostov reviewed Jun 10, 2022

View reviewed changes

qos-core/src/protocol/mod.rs Outdated Show resolved Hide resolved

emostov reviewed Jun 10, 2022

View reviewed changes

qos-core/src/protocol/mod.rs Outdated Show resolved Hide resolved

emostov reviewed Jun 10, 2022

View reviewed changes

qos-core/src/protocol/mod.rs Outdated Show resolved Hide resolved

emostov reviewed Jun 10, 2022

View reviewed changes

qos-core/src/protocol/mod.rs Outdated Show resolved Hide resolved

emostov reviewed Jun 10, 2022

View reviewed changes

qos-crypto/src/lib.rs Outdated Show resolved Hide resolved

emostov reviewed Jun 10, 2022

View reviewed changes