[WIP] Slate Serialization

[j01tz]

Rendered link to RFC document

This is a proposal for standardizing slate-serialization for Grin.

[j01tz]

The proposed SlatePack serialization method adds a nontrivial amount of complexity. An alternative serialization option with some tradeoffs to reduce this complexity that may be worth considering is simply using MessagePack directly on slate objects:

• Reduces complexity while retaining efficiency benefits
• Does not support encrypted slates without breaking compatibility
• Can't ensure integrity of slate data in transit
• Some of the complexity to support extendable features could be added at the armoring layer instead of serialization layer
  • In this case we lose some efficiency in ability to encode for QR

I wanted to propose the SlatePack option to show what is possible for future extensions to support encrypted slates, more robust payment proofs and more. However we may decide that the added complexity is not worth the feature benefits and refocus toward a bare minimum serialization scheme.

[j01tz]

Based on the keybase chat, here are some updated considerations based on my current understanding of serialization options. After more discussion I'll take another pass at the RFC to update it with current thinking.

grin/../ser.rs:

pros

• most compact option by a few bytes
• no new dependencies
• already used in serialization in other areas

cons

• non libwallet implementations must do a custom implementation of grin-serialization rather than using existing library
• non libwallet implementations can't infer the types and so must know the spec well to implement
• is not easily extendable at this layer

MessagePack:

pros

• reasonably compact
• well-defined, clear and commonly used serialization standard
• libraries available in many languages
• easy to implement
• provides types

cons

• requires additional library to use
• does not provide versioning
• is not easily extendable at this layer

SlatePack:

pros

• reasonably compact
• not much more complex than msgpack to implement
• extendable to support future slate features: encryption, robust payment proofs
• provides robustness to a feature that will be commonly used by humans

cons

• the most complex option from the perspective of implementing in libwallet
• the least compact option by a few bytes

[j01tz]

The RFC has been significantly reworked based on feedback over the last weeks. It still needs proper examples and links but otherwise nearly ready for review. It may still need an iteration or two depending on feedback. Please let me know if you see anything that should be clarified, added or removed.

Depending on the outcome of mimblewimble/grin-wallet#385 some details will need to be added to the Binary Serialization section.

[j01tz]

Closing in favor of #55