Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Key deserializer improvements #467

Merged
merged 7 commits into from
Oct 4, 2021
Merged

Key deserializer improvements #467

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5fab3c2
As #[inline] to as_slice
maurolacy Oct 4, 2021
e641796
Use inline(always)
maurolacy Oct 4, 2021
be1f414
Use from_vec to save a clone in case of raw keys
maurolacy Oct 4, 2021
ff351b8
Inline always from_vec too
maurolacy Oct 4, 2021
97fcb6c
Move impl to from_vec
maurolacy Oct 4, 2021
c4c8af2
Use Self::Output in forwarding impls
maurolacy Oct 4, 2021
2629245
Use split_off instead of split_at
maurolacy Oct 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 67 additions & 6 deletions packages/storage-plus/src/de.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ pub trait KeyDeserialize {
type Output: Sized;

fn from_slice(value: &[u8]) -> StdResult<Self::Output>;
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output>;
}

impl KeyDeserialize for () {
Expand All @@ -18,6 +19,11 @@ impl KeyDeserialize for () {
fn from_slice(_value: &[u8]) -> StdResult<Self::Output> {
Ok(())
}

#[inline(always)]
fn from_vec(_value: Vec<u8>) -> StdResult<Self::Output> {
Ok(())
}
}

impl KeyDeserialize for Vec<u8> {
Expand All @@ -27,6 +33,11 @@ impl KeyDeserialize for Vec<u8> {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
Ok(value.to_vec())
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
Ok(value)
}
}

impl KeyDeserialize for &Vec<u8> {
Expand All @@ -36,6 +47,11 @@ impl KeyDeserialize for &Vec<u8> {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
<Vec<u8>>::from_slice(value)
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
Ok(value)
}
}

impl KeyDeserialize for &[u8] {
Expand All @@ -45,6 +61,11 @@ impl KeyDeserialize for &[u8] {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
<Vec<u8>>::from_slice(value)
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
Ok(value)
}
}

impl KeyDeserialize for String {
Expand All @@ -56,6 +77,11 @@ impl KeyDeserialize for String {
// FIXME: Add and use StdError utf-8 error From helper
.map_err(|err| StdError::generic_err(err.to_string()))
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
maurolacy marked this conversation as resolved.
Show resolved Hide resolved
String::from_slice(value.as_slice())
}
maurolacy marked this conversation as resolved.
Show resolved Hide resolved
}

impl KeyDeserialize for &String {
Expand All @@ -65,6 +91,11 @@ impl KeyDeserialize for &String {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
String::from_slice(value)
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
String::from_slice(value.as_slice())
maurolacy marked this conversation as resolved.
Show resolved Hide resolved
}
}

impl KeyDeserialize for &str {
Expand All @@ -74,6 +105,11 @@ impl KeyDeserialize for &str {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
String::from_slice(value)
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
String::from_slice(value.as_slice())
maurolacy marked this conversation as resolved.
Show resolved Hide resolved
}
}

impl KeyDeserialize for Addr {
Expand All @@ -83,6 +119,11 @@ impl KeyDeserialize for Addr {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
Ok(Addr::unchecked(String::from_slice(value)?))
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
Addr::from_slice(value.as_slice())
maurolacy marked this conversation as resolved.
Show resolved Hide resolved
}
}

impl KeyDeserialize for &Addr {
Expand All @@ -92,6 +133,11 @@ impl KeyDeserialize for &Addr {
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
Addr::from_slice(value)
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
Addr::from_slice(value.as_slice())
}
}

macro_rules! integer_de {
Expand All @@ -105,6 +151,11 @@ macro_rules! integer_de {
// FIXME: Add and use StdError try-from error From helper
.map_err(|err: TryFromSliceError| StdError::generic_err(err.to_string()))?))
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
<IntKey<$t>>::from_slice(value.as_slice())
}
})*
}
}
Expand All @@ -116,12 +167,12 @@ impl KeyDeserialize for TimestampKey {

#[inline(always)]
fn from_slice(value: &[u8]) -> StdResult<Self::Output> {
Ok(<u64>::from_be_bytes(
value
.try_into()
// FIXME: Add and use StdError try-from error From helper
.map_err(|err: TryFromSliceError| StdError::generic_err(err.to_string()))?,
))
<IntKey<u64>>::from_slice(value)
}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
<IntKey<u64>>::from_slice(value.as_slice())
}
}

Expand All @@ -140,6 +191,11 @@ impl<T: KeyDeserialize, U: KeyDeserialize> KeyDeserialize for (T, U) {

Ok((T::from_slice(t)?, U::from_slice(u)?))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm.. could you use from_vec here as well?

I guess you would need value.split_off rather than split_at. But should be more consistent/efficient

}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
<(T, U)>::from_slice(value.as_slice())
}
}

impl<T: KeyDeserialize, U: KeyDeserialize, V: KeyDeserialize> KeyDeserialize for (T, U, V) {
Expand All @@ -165,6 +221,11 @@ impl<T: KeyDeserialize, U: KeyDeserialize, V: KeyDeserialize> KeyDeserialize for

Ok((T::from_slice(t)?, U::from_slice(u)?, V::from_slice(v)?))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the (T, U) tuples (from_vec)

Copy link
Contributor Author

@maurolacy maurolacy Oct 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did it. It's a little unclear with the var names. And, I doubt it's more efficient.

If we wanted a self-contained version (that is, one that doesn't depend on from_slice), we could always use from_vec(t.to_vec()) and still work with slices / references in the mean time.

Copy link
Contributor Author

@maurolacy maurolacy Oct 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, for triples, the version with split_off() has four vec allocations instead of three; because of the small vec for the length of v. Plus, it requieres a conversion to to_slice() for calling try_into(). Plus mut vecs.

I really like the previous version better.

}

#[inline(always)]
fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
<(T, U, V)>::from_slice(value.as_slice())
}
}

#[cfg(test)]
Expand Down
2 changes: 1 addition & 1 deletion packages/storage-plus/src/iter_helpers.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ pub(crate) fn deserialize_kv<K: KeyDeserialize, T: DeserializeOwned>(
kv: Pair,
) -> StdResult<(K::Output, T)> {
let (k, v) = kv;
let kt = K::from_slice(&k)?;
let kt = K::from_vec(k)?;
let vt = from_slice::<T>(&v)?;
Ok((kt, vt))
}
Expand Down