Skip to content

support ics23 no-std #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 12, 2021
Merged

support ics23 no-std #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a70c877
support ics23 no-std
Jun 11, 2021
0ecf7f4
update Cargo.toml
Jun 11, 2021
e5ecaf1
update CI
Jun 11, 2021
dac5c3d
Use sp-std support no_std and update CI
Jun 28, 2021
99c8eda
Update CI
Jun 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,23 @@ jobs:
keys:
- v4-cargo-cache-{{ arch }}-{{ checksum "Cargo.lock" }}
- run:
name: Build all targets
name: Check all targets in std
command: cargo check --all
- run:
name: Check all target in no_std
command: cargo check --no-default-features --all
- run:
name: Build all targets in std
command: cargo build --all --all-targets
- run:
name: Build all targets in no_std
command: cargo build --all --no-default-features
- run:
name: Run all tests with no-std
command: cargo test --all --no-default-features
- run:
name: Run all tests with std
command: cargo test --all
- save_cache:
paths:
- /usr/local/cargo/registry
Expand Down
7 changes: 7 additions & 0 deletions rust/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 21 additions & 7 deletions rust/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,28 @@ members = ["codegen"]

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
prost = "0.7.0"
bytes = "1.0.1"
hex = "0.4.3"
sha2 = "0.9.3"
sha3 = "0.9.1"
ripemd160 = "0.9.1"
anyhow = "1.0.40"
prost = { version = "0.7.0", default-features = false, features = ["prost-derive"] }
bytes = { version = "1.0.1", default-features = false }
hex = { version = "0.4.3", default-features = false, features = [ "alloc" ] }
sha2 = { version = "0.9.3", default-features = false }
sha3 = { version = "0.9.1", default-features = false }
ripemd160 = { version = "0.9.1", default-features = false }
anyhow = { version = "1.0.40", default-features = false }
sp-std = {version = "3.0.0", default-features = false }

[dev-dependencies]
serde = { version = "1.0.125", features = ["derive"] }
serde_json = "1.0.64"

[features]
default = ["std"]
std = [
"prost/std",
"bytes/std",
"hex/std",
"sha2/std",
"sha3/std",
"ripemd160/std",
"anyhow/std",
"sp-std/std",
]
39 changes: 33 additions & 6 deletions rust/src/api.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
use std::collections::HashMap;
use std::hash::BuildHasher;
use std::collections::btree_map::BTreeMap as HashMap;

#[cfg(not(feature = "std"))]
use std::prelude::*;

use crate::compress::{decompress, is_compressed};
use crate::ics23;
Expand Down Expand Up @@ -64,11 +66,11 @@ pub fn verify_non_membership(
}

#[allow(clippy::ptr_arg)]
pub fn verify_batch_membership<G: BuildHasher>(
pub fn verify_batch_membership(
proof: &ics23::CommitmentProof,
spec: &ics23::ProofSpec,
root: &CommitmentRoot,
items: HashMap<&[u8], &[u8], G>,
items: HashMap<&[u8], &[u8]>,
) -> bool {
// ugly attempt to conditionally decompress...
let mut proof = proof;
Expand Down Expand Up @@ -204,14 +206,18 @@ pub fn tendermint_spec() -> ics23::ProofSpec {

#[cfg(test)]
mod tests {
extern crate std as _std;
use super::*;

use anyhow::{bail, ensure};
use prost::Message;
use serde::Deserialize;
use std::fs::File;
use std::io::prelude::*;
#[cfg(feature = "std")]
use _std::fs::File;
#[cfg(feature = "std")]
use _std::io::prelude::*;
use std::vec::Vec;
use alloc::string::String;

use crate::compress::compress;
use crate::helpers::Result;
Expand All @@ -230,6 +236,7 @@ mod tests {
pub value: Option<Vec<u8>>,
}

#[cfg(feature = "std")]
Copy link
Contributor

@ethanfrey ethanfrey Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so these tests are not run in no_std mode.

It gets the code to pass (and I guess I can merge it), but it would be nice to run the whole test suite with no-std

Copy link
Author

@DaviRain-Su DaviRain-Su Jul 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is necessary to test here in the std environment, for File ,io and some other libraries are not available in core and alloc, but only in the standard library.
Relate tourtial: https://docs.rust-embedded.org/book/intro/no-std.html

Copy link
Contributor

@ethanfrey ethanfrey Jul 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah... you are very right.

I guess none of these work in no-std but it would be nice to have equivalent tests.
Maybe you can use include_bytes, and then get rid of all the load_file stuff as well (and have tests work in both environments)

fn load_file(filename: &str) -> Result<(ics23::CommitmentProof, RefData)> {
let mut file = File::open(filename)?;
let mut contents = String::new();
Expand All @@ -252,6 +259,7 @@ mod tests {
Ok((parsed, data))
}

#[cfg(feature = "std")]
fn verify_test_vector(filename: &str, spec: &ics23::ProofSpec) -> Result<()> {
let (proof, data) = load_file(filename)?;

Expand All @@ -267,77 +275,90 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_left() -> Result<()> {
let spec = iavl_spec();
verify_test_vector("../testdata/iavl/exist_left.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_right() -> Result<()> {
let spec = iavl_spec();
verify_test_vector("../testdata/iavl/exist_right.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_middle() -> Result<()> {
let spec = iavl_spec();
verify_test_vector("../testdata/iavl/exist_middle.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_left_non() -> Result<()> {
let spec = iavl_spec();
verify_test_vector("../testdata/iavl/nonexist_left.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_right_non() -> Result<()> {
let spec = iavl_spec();
verify_test_vector("../testdata/iavl/nonexist_right.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_middle_non() -> Result<()> {
let spec = iavl_spec();
verify_test_vector("../testdata/iavl/nonexist_middle.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_left() -> Result<()> {
let spec = tendermint_spec();
verify_test_vector("../testdata/tendermint/exist_left.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_right() -> Result<()> {
let spec = tendermint_spec();
verify_test_vector("../testdata/tendermint/exist_right.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_middle() -> Result<()> {
let spec = tendermint_spec();
verify_test_vector("../testdata/tendermint/exist_middle.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_left_non() -> Result<()> {
let spec = tendermint_spec();
verify_test_vector("../testdata/tendermint/nonexist_left.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_right_non() -> Result<()> {
let spec = tendermint_spec();
verify_test_vector("../testdata/tendermint/nonexist_right.json", &spec)
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_middle_non() -> Result<()> {
let spec = tendermint_spec();
verify_test_vector("../testdata/tendermint/nonexist_middle.json", &spec)
}

#[cfg(feature = "std")]
fn load_batch(files: &[&str]) -> Result<(ics23::CommitmentProof, Vec<RefData>)> {
let mut entries = Vec::new();
let mut data = Vec::new();
Expand Down Expand Up @@ -393,6 +414,7 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_batch_exist() -> Result<()> {
let spec = iavl_spec();
let (proof, data) = load_batch(&[
Expand All @@ -407,6 +429,7 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn compressed_iavl_batch_exist() -> Result<()> {
let spec = iavl_spec();
let (proof, data) = load_batch(&[
Expand All @@ -422,6 +445,7 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn test_vector_iavl_batch_nonexist() -> Result<()> {
let spec = iavl_spec();
let (proof, data) = load_batch(&[
Expand All @@ -436,6 +460,7 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn compressed_iavl_batch_nonexist() -> Result<()> {
let spec = iavl_spec();
let (proof, data) = load_batch(&[
Expand All @@ -451,6 +476,7 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_batch_exist() -> Result<()> {
let spec = tendermint_spec();
let (proof, data) = load_batch(&[
Expand All @@ -465,6 +491,7 @@ mod tests {
}

#[test]
#[cfg(feature = "std")]
fn test_vector_tendermint_batch_nonexist() -> Result<()> {
let spec = tendermint_spec();
let (proof, data) = load_batch(&[
Expand Down
7 changes: 5 additions & 2 deletions rust/src/compress.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use prost::Message;
use std::collections::HashMap;
use std::collections::btree_map::BTreeMap as HashMap;
use std::vec::Vec;
use std::borrow::ToOwned;

use crate::helpers::Result;
use crate::ics23;
Expand Down Expand Up @@ -86,7 +87,9 @@ pub fn compress_exist(
.iter()
.map(|x| {
let mut buf = Vec::new();
x.encode(&mut buf)?;
x.encode(&mut buf).map_err(|e : prost::EncodeError | {
anyhow::anyhow!(e)
})?;

if let Some(&idx) = registry.get(buf.as_slice()) {
return Ok(idx);
Expand Down
Loading