Slatepack

[j01tz]

Rendered link to RFC document

Replaces #48, #53, #50

[lehnberg]

Great work on this @j01tz! Took a more detailed pass, will think about this some more over weekend.

[yeastplume]

Great work on this so far! I've done some exploration on embedding age encryption now via mimblewimble/grin-wallet#411 and I have a few thoughts, particularly on the encryption. For reference the current state of that PR is that encryption works, but all of the required age header data is contained within the payload as per the age spec.

Firstly, what is the benefit of splitting out and manually managing of the age header data into the slatepack struct as opposed to simply embedding standard age binary data (which already has the header included ready to write to file)? There are a few major disadvantages to doing this that I can see:

• It makes the slatepack format and updates to it more complicated to manage and extend. If the encryption format's header data is contained within the encrypted payload, then the slatepack format becomes bin(version|fields|payload) which is straightforward to add fields to (by encoding the payload position, reading known fields then skipping to the payload position). In the case of having to support bin(version|fields|encryption-specific-header|payload) this becomes much more difficult to manage given the content of the headers will differ depending on encryption used. I'm not sure what we gain by having to explicitly worry about these fields.
• It strongly ties us to a particular encryption format. age itself is very new and it may become desirable to support other encryption formats (like mode 0=plaintext, 1=age encrypted 2=next big thing) and so on. Without having to manually maintain header info, adding new encryption formats should have minimal effect of the slatepack spec.
• The current rust library currently only exposes an API that assumes the result will be written to a file (or armored and written to a file). If we need to break this out into an API that breaks up and exposes the header data, we'll likely need to fork, develop that API and maintain it, which is a serious undertaking. (or we could just parse the result from the existing API and break out headers that way, but again, why?)
• If we just embed headers in the payload, we don't need to worry about cut-pasting parts of the AGE spec into our slatepack RFC and keeping up with changes, we can basically just say 'we use age v1.x' and point to the spec
• If slatepacks are binary output and armored anyhow, I don't think it makes a material difference whether the header data is encoded within the slatepack struct or the payload itself. The only people ever likely to want to see it or have to care about it anyhow would be developers.

So unless there's a very compelling reason why we need to explicitly manage all of this header data, I'd very much be in favor of letting the payload worry about any headers it needs, freeing us up to think about key management and any fields that are more relevant to us in particular.

Second, on the subject of key generation, I don't have particular answers here, but I can just describe how it currently works. Each wallet account under a master seed has a blinding factor derivation path, and a second derivation path used to automatically generate ed25519 keys for use in TOR addresses (or rather they derive 32 bytes that can be used for secp256k1 or dalek seckeys). . At the moment, there is a hardcoded assumption that the only element being used on the TOR derivation path is index 0. This makes some sense in the TOR world, since it this key corresponds to a hidden service endpoint, having a separate address for each transation wouldn't work. You could cycle endpoints frequently, choosing to use index 1, then index 2. Your identity will change each time you do this, but funds sent via TOR to your previous endpoints won't end up being received since you're only ever running the current listener (which is the only one currently defined in the DHT used by TOR). (NB This address cycling is something we know we want to add at some point when we've decided how we want to manage it)

TOR downgrading to slatepack is fairly simple in the case where we're using index 0 only; try to send via TOR, then if that doesn't work, age encrypt with the slatepack address and cut/paste/QR it there somehow, and the recipient will use the key at index 0 to decrypt.

Difference here obviously is that wallets receiving slatepacks are in a better position to decrypt slatepacks encoded for older addresses (though they'd need to grind if there isn't some path metadata somewhere as well, and I hate key grinding). But the async/sync stories here are incompatible. In the case of slatepack exchanges you'd think that you'd want to encourage using a new address every time, but this is far more difficult to manage in the TOR world. So the notion of 'just use tor, if you can't then use slatepack' is more nuanced due to possibly differing key management expectations of each format. Of course, it's made much more simple if we assume very few (or infrequent) address changes, and assume grin wallet addresses are more akin to GPG/age public keys in all cases.

Only other minor thing that really jumps out ATM is that I'm not sure we need to bech32 encode the address, if TOR is our first order send mechanism it seems to make a bit more sense to me to just keep addresses in that format. But I also see the sense in having a human-readable prefix as well, so perhaps 'prefix|onionv3address'? (I take the points about advantages bech32 addresses above as well, so don't have hugely strong feelings about this)

[j01tz]

what is the benefit of splitting out and manually managing of the age header data into the slatepack struct

After thinking about this some I think you raise an important point: right now it may be cleaner to abandon the header field in the slatepack encrypted payloads. My reasoning for splitting this out initially was that age headers support more complexity than we need and they don't support ed25519 recipient stanzas so we would need to use x25519 keys here (which is fine in our basic construction technically speaking- it just could be confusing since we are using ed25519 as the primary keys for Tor as opposed to age's x25519- this just changes when the key mapping happens). As you mentioned, the ability to just say 'encryption follows age v1.x' and have the further details be self-evident would be very nice.

One important consideration if we go this route: to support multiparty transactions in the future, we will need to bring some of this data out of the payload anyway in one form or another (otherwise for Alice, Bob and Charlie to build a tx we have to do weird stuff for Bob to derive Charlies Tor address to forward the slate after he receives it from Alice- it is much more desirable to have Charlies ed25519 key here than his x25519 key for our purposes of finishing building the tx over Tor).

re: key generation

For privacy reasons I think users will want the ability to use a fresh address for each tx. This also means that we would need to support listening for multiple addresses at the same time- this shouldn't be a problem for average users if we allow a configuration with a default maximum (e.g. 5-10) listeners open at once. Adding another live hidden service endpoint should be as easy as adding an entry to torrc:

HiddenServiceDir /var/lib/tor/www2_service/
HiddenServicePort 80 127.0.0.1:8080

HiddenServiceDir /var/lib/tor/www3_service/
HiddenServicePort 80 127.0.0.1:8081

Maybe we can step indexes with wallet and Tor keys at the same rate by default to prevent reuse? Index 0 could be the default listener for 'public listening' which would be an optional static, known, publicly shared address for a user. Beyond that the keys could be for 'private listening' which would have changing availabilities based on current set of 'ongoing' txs.

Preventing address reuse here has different privacy implications than preventing address reuse on chain- fortunately they are not as dire. However, there are still some new privacy issues that will appear as a result of address reuse over time that we might be able to prevent if we can find a sane way to not reuse addresses without breaking our ability to easily listen for txs.

As you indicated this would require encoding derivation path somewhere to avoid grinding. I realize this starts getting us into some ugly complexity to squeeze out some more privacy but just briefly advocating for the perspective here so it is on the table. There is definitely important nuance here that we don't want to bulldoze through but also don't want to get hung up on for too long.

I'm not sure we need to bech32 encode the address, if TOR is our first order send mechanism it seems to make a bit more sense to me to just keep addresses in that format

Others have raised this feedback as well. For me, this is less of a technical decision and more of a human decision:

• Reduced cognitive load for users pasting a bech32 address into a wallet for a spend as opposed to an unfamiliar format
• Humans sometimes have negative gut reactions to onion addresses and less so to bech32 addresses- we have the opportunity to convert users that would normally not touch anything that uses onion addresses with our 'up-and-up' bech32 addresses
• Improved interoperability with other ecosystems/tools that use bech32 allows for more support for key distribution and encryption- age, keys.pub

It feels like the bech32 encoding is an important detail for this scheme working in a more human friendly way but I obviously don't have any data saying this is a technically superior decision.

[yeastplume]

what is the benefit of splitting out and manually managing of the age header data into the slatepack struct

After thinking about this some I think you raise an important point: right now it may be cleaner to abandon the header field in the slatepack encrypted payloads. My reasoning for splitting this out initially was that age headers support more complexity than we need and they don't support ed25519 recipient stanzas so we would need to use x25519 keys here (which is fine in our basic construction technically speaking- it just could be confusing since we are using ed25519 as the primary keys for Tor as opposed to age's x25519- this just changes when the key mapping happens). As you mentioned, the ability to just say 'encryption follows age v1.x' and have the further details be self-evident would be very nice.

I think doing the key mapping at this stage is fine, given the added complexity the alternative entails. This will all be taken care of in the wallet libraries, where we already have the same keys being used for for secp256k blinding factors, ed25519 keys and now x25519 keys.

One important consideration if we go this route: to support multiparty transactions in the future, we will need to bring some of this data out of the payload anyway in one form or another (otherwise for Alice, Bob and Charlie to build a tx we have to do weird stuff for Bob to derive Charlies Tor address to forward the slate after he receives it from Alice- it is much more desirable to have Charlies ed25519 key here than his x25519 key for our purposes of finishing building the tx over Tor).

That makes sense, I'll see how simple it is to add an optional 'participants' struct into the existing format for future use.

re: key generation

For privacy reasons I think users will want the ability to use a fresh address for each tx. This also means that we would need to support listening for multiple addresses at the same time- this shouldn't be a problem for average users if we allow a configuration with a default maximum (e.g. 5-10) listeners open at once. Adding another live hidden service endpoint should be as easy as adding an entry to torrc:

HiddenServiceDir /var/lib/tor/www2_service/
HiddenServicePort 80 127.0.0.1:8080

HiddenServiceDir /var/lib/tor/www3_service/
HiddenServicePort 80 127.0.0.1:8081

Maybe we can step indexes with wallet and Tor keys at the same rate by default to prevent reuse? Index 0 could be the default listener for 'public listening' which would be an optional static, known, publicly shared address for a user. Beyond that the keys could be for 'private listening' which would have changing availabilities based on current set of 'ongoing' txs.

Preventing address reuse here has different privacy implications than preventing address reuse on chain- fortunately they are not as dire. However, there are still some new privacy issues that will appear as a result of address reuse over time that we might be able to prevent if we can find a sane way to not reuse addresses without breaking our ability to easily listen for txs.

As you indicated this would require encoding derivation path somewhere to avoid grinding. I realize this starts getting us into some ugly complexity to squeeze out some more privacy but just briefly advocating for the perspective here so it is on the table. There is definitely important nuance here that we don't want to bulldoze through but also don't want to get hung up on for too long.

Agreed with the analysis here, and that our ultimate solution to this is going to have to get the balance right. We may have to defer the decision on how we handle this to post-4.0.0, but introducing a solution a bit later should be fine as it'll be backwards compatible.

I'm not sure we need to bech32 encode the address, if TOR is our first order send mechanism it seems to make a bit more sense to me to just keep addresses in that format

Others have raised this feedback as well. For me, this is less of a technical decision and more of a human decision:

• Reduced cognitive load for users pasting a bech32 address into a wallet for a spend as opposed to an unfamiliar format
• Humans sometimes have negative gut reactions to onion addresses and less so to bech32 addresses- we have the opportunity to convert users that would normally not touch anything that uses onion addresses with our 'up-and-up' bech32 addresses
• Improved interoperability with other ecosystems/tools that use bech32 allows for more support for key distribution and encryption- age, keys.pub

It feels like the bech32 encoding is an important detail for this scheme working in a more human friendly way but I obviously don't have any data saying this is a technically superior decision.

I can take these points, let me think about this it a bit further.

[lehnberg]

Where are we with open questions here @j01tz, what is the stuff that still needs figuring out? Are we far from an FCP @quentinlesceller?

[j01tz]

The remaining details needed before the RFC can be completed are regarding the key generation. I feel confident that @yeastplume and I will converge on something sensible there this week to update the RFC. Beyond that we want to add some notes about number of active Tor listeners but do not necessarily need these details finalized before moving forward with the RFC. Overall, we should be close to FCP by the end of this week.

[j01tz]

The most recent update contains some changes that were a result of discussing the implementation with @yeastplume as well as a few changes as a result of thinking more about the sender field.

Changelog:

• Change armor edge case support from multi-part messages to files, limiting strings to 1MB
• Remove header fields in favor of using age's header
• Use full default age encrypted payload including header for Slatepack encrypted payload
• Soul searching about sender, MACs, privacy..
  - ..we should probably encrypt sender for encrypted Slatepacks in next release
  - Privacy win by encrypting sender: it should be possible to complete a Slatepack transaction on an adversarial platform without the sender revealing any identifying information to anyone but the recipient
• Add encrypted metadata field to support above point

TODO:
[ ] Add key derivation details as implemented and discussed

The most notable change is the addition of the notion of encrypted metadata as a result of some more thought regarding sender. I know it is a bit out of the way considering our current schedule but there is no reason why it couldn't be done in the next version (besides the need to maintain backwards compatible code in the future I guess..) and I even wrote that in to the RFC (if we manage to get this change in for the first release I can update the language there).

I'm not set on this change but think it is worth considering, especially now before there is significantly more friction to changing anything. The primary use I'm imagining is manual exchange of slatepack messages in private messages on an untrustworthy forum or other platform: if users are seeking extra privacy by using encryption we can offer even better privacy by encrypting the sender address too. Otherwise a nosy moderator could try to use the senders address to potentially extract metadata, ddos to disrupt service, insert a different address to attempt to steal etc. The cost for us is not much, just some plumbing and for the user is not noticable beyond the improved privacy.

Curious for other thoughts here- I realize this is coming late but there is a lot here and we have been moving pretty fast :)

Once feedback on the above is integrated and TODOs checked off, I feel good about moving to FCP.

[quentinlesceller]

@j01tz ping me when you think it's ready for FCP.

[j01tz]

Thanks @quentinlesceller just gave it another quick review with some trivial changes. It should now be good for FCP, with any potentially lacking information being implementation details that can be added/updated in the future.

[lehnberg]

@j01tz @yeastplume @quentinlesceller @DavidBurkett and others - forgive me for the bike shed, I had a shower thought yesterday, not sure it makes sense at all... so feel free to dismiss but...

Calling the bech32 grin1... string an address irks me a bit from a usability standpoint. It's more than an address, and sometimes it's not even used as an address at all, and even when it is used as an address, it's different than any address in other blockchain projects. So really, why is it called an address?

Some different trains of thoughts I had for alternatives:

1. It's true that the "address", when provided, at times will act and work as an address (when Tor is available and both users are online), but it will always be an encryption key. So wouldn't that already be a more descriptive name for it? Your grin key. It holds the keys to the Tor destination too.
2. Or, maybe there's some other naming we could have that more accurately describes that it's not your typical "address" like in all other blockchain projects, and that it instead has a unique role in grin where it has powerful dual properties (Tor sending + encryption). What about transport key?
3. Or, maybe call it something a bit left-field and quirky that we could make part of our own brand like... envelope. In a way, that's what it is right? You stick your slate in an envelope and seal it (encrypt it) and write the address on it for where it should be sent (derive the Tor address). If you don't provide an envelope, well then the slate will neither be encrypted, nor sent the automatic way. You'll have to deliver it manually, but you can still do so in a sealed envelope (encrypted) if you like. It also fits with the idea of users having many different envelopes, not necessarily wanting to reuse envelopes, etc. But yeah you share your personal envelope to someone, and they use it for their slate communications with you.

Dunno. Does any of this make sense?

[quentinlesceller]

In line with our governance process, this RFC can be considered being in Final Comment Period, with a disposition to merge in two weeks time, on June 30 .

Please ensure any comments are made before then!

[ramheat]

@j01tz @yeastplume @quentinlesceller @DavidBurkett and others - forgive me for the bike shed, I had a shower thought yesterday, not sure it makes sense at all... so feel free to dismiss but...

Calling the bech32 grin1... string an address irks me a bit from a usability standpoint. It's more than an address, and sometimes it's not even used as an address at all, and even when it is used as an address, it's different than any address in other blockchain projects. So really, why is it called an address?

Some different trains of thoughts I had for alternatives:

1. It's true that the "address", when provided, at times will act and work as an address (when Tor is available and both users are online), but it will always be an encryption key. So wouldn't that already be a more descriptive name for it? Your grin key. It holds the keys to the Tor destination too.
2. Or, maybe there's some other naming we could have that more accurately describes that it's not your typical "address" like in all other blockchain projects, and that it instead has a unique role in grin where it has powerful dual properties (Tor sending + encryption). What about transport key?
3. Or, maybe call it something a bit left-field and quirky that we could make part of our own brand like... envelope. In a way, that's what it is right? You stick your slate in an envelope and seal it (encrypt it) and write the address on it for where it should be sent (derive the Tor address). If you don't provide an envelope, well then the slate will neither be encrypted, nor sent the automatic way. You'll have to deliver it manually, but you can still do so in a sealed envelope (encrypted) if you like. It also fits with the idea of users having many different envelopes, not necessarily wanting to reuse envelopes, etc. But yeah you share your personal envelope to someone, and they use it for their slate communications with you.

Dunno. Does any of this make sense?

Really it makes sense.Average users(non tech savy) will perceive it''look Grin has adresses''
grin1 not good imo.
grinkey,envelope,mwkey,mwslate..envelope defines the tx very well..

Envelope gives the message of encrypted so well.Makes a light in people's mind.###

[Paouky]

A follow-up proposal to @lehnberg's delightful shower thought.

While envelope is technically the best workflow analogy, it has two issues:

1. It's long and messy to pronounce. Users might fall back to using the familiar, two-syllable word 'address' instead.
2. Users will most commonly use phrases like "what's your address?" and "send the grins to this address". Try to replace the word address with envelope and what you get is unintuitive and quite odd.

For SlatepackAddress I think the term route would be better for users:

1. It's much more suggestive that if the user isn't available on the other side, the tx won't go through. A tx tries to go through the route. This aspect is probably the most important for user experience.
2. Has a similar connotation as address, so somewhat familiar.
3. Bonus: One-syllable.

A drawback is that if users already agree beforehand to transact only by SlatepackMessage then route is not suggestive of it also serving as an encryption tool.

Also, The way I see it, having the eyes of a non-technical user, we should name SlatepackMessage and SlatepackAddress separately.
We could call the message, quite simply, a pack. When I read through the RFC this is the term I naturally used to describe SlatepackMessage to myself, so I suspect it's incredibly intuitive.

1. The squared block of text reminds you of a pack. It even bears the word in its content.
2. Its intuitive to the workflow. SlatepackMessage feels like a pack of text to the user. The other side needs to receive, open, and send it back. Same as envelope in that regard.
3. Again, one-syllable.

What do you think, is this the right approach?

[DavidBurkett]

When did we become such a PR coin? It's an address. If users aren't able to learn the difference between an on-chain address and an off-chain one, then there are a number of other things that will already turn them away from Grin (like "unlimited" emission). It seems like using anything other than address will result in non-uniform naming, since some places will call it an address regardless.

[ramheat]

When did we become such a PR coin? It's an address. If users aren't able to learn the difference between an on-chain address and an off-chain one, then there are a number of other things that will already turn them away from Grin (like "unlimited" emission). It seems like using anything other than address will result in non-uniform naming, since some places will call it an address regardless.

Since you need adoption..Coca Cola,Pepsi monopoly,they still make Pr..
Fed makes PR us dollars,BTC maximalist make PR..ECB makes PR of euro..why dont we?

[Paouky]

When did we become such a PR coin? It's an address. If users aren't able to learn the difference between an on-chain address and an off-chain one, then there are a number of other things that will already turn them away from Grin (like "unlimited" emission). It seems like using anything other than address will result in non-uniform naming, since some places will call it an address regardless.

I don't see how this is about PR at all. It's about user experience, which you are a big advocate for. Sending a transaction to something called an address and not realizing why the grins aren't being sent (like bitcoin would) is simply a horrible user experience. If we could avoid that confusion by careful naming, we should. It has zero connection to marketing.

If you think the default word address would be used regardless then that's a different argument, which I somewhat agree with.

[DavidBurkett]

@Paouky Sorry, the comment wasn't directed to you. The reasoning I saw in keybase was that we don't want to use "address" because we've been telling people some of Grin's privacy comes from not using addresses.

[lehnberg]

Good to see you join the conversation @jasotimur and @Paouky. I agree, this has more to do with user experience than anything else.

As I outlined in my comment above there's a logic to why address is not a great descriptor for the type of functionality that's enabled by the use of this optional bech32 string.

On the other hand, as you and others point out, users will likely be asking "hey so what's your address?", and developers and UX designers alike will do good by not trying to force a new term into users mouths and minds. It might end up being even more confusing to see a field for entering the "grin envelope key" or whatever. Whereas address is already a well known concept in these circumstances and expected to be this bech32 style string.

So I came around to using address in the meeting last week, thanks to those who pointed out why it made sense. 👍

That said, I still believe there's room for simplifying terminology in both the RFC, the code, and the user facing parts. @DavidBurkett / @antiochp's BEGRIN... ...BEGONE suggestion for replacing the current armor headers/footers is an interesting example. I'm not sure we need slatepack visible as it's not immediately obvious what that is, or even what a slate is. We'd do well to converge on something that feels simple, minimal, and intuitive before this goes to production, as it will be much harder to change later.

[lehnberg]

Hmm... actually, drawing inspiration from Tari, how about simply going with ID?

• A wallet has a (non-unique) ID, and you can try to send to that ID.
• Sometimes that ID might not be reachable, and then you will need to send it manually.
• If you specify which ID is the intended recipient, you make sure only that ID can read the message (i.e. encryption).
• I can definitely imagine users asking each other "Hey what's your ID?"
• And UIs would be intuitive if it was asking for "Destination ID" or something like that.

Isn't that a more simple way to describe what this is? Feels like it might do a better job than address, as it also encompasses the encryption part of the functionality. Plus it's shorter. :)

[DavidBurkett]

ID isn't bad, though there will still be many who refer to it as an address, like here: https://t.me/tarilab/31317

[ramheat]

You are highly tech persons,when you talk about İD,adresss,slatepack..you know it is meaning technically,encrypted data ,slatepack..you get it in your tech mind..the difference..

But for a non tech person,if you name it adresss they wont get it,understand as you..and İD is the worst of all..so you ask a average person what is your İD in crypto tx??
They never gonna believe it is privacy..İD is worst label of all..Sending alarm bells for an average user..
What is your İD=what is your passport number,social security number,in the mind of average joe..
Please think like a little like a non-tech people.Understand their mindset.
it should give the message''encrypted'' .Always..medium is the message.

Thats all my perspective from an average user,who hasnt got skills like you..

[ramheat]

it can have a name..similar payment,cash ''prepayment''..making cash payment between 2 party..Since it is about fungibility..cash payment,precash,prepayment,grinpay,mwpay..

[quentinlesceller]

All good. Two links comments and also please rename the file to 0015-slatepack.md.

After that we'll be good to go :).

[kaidiren]

cool, just go ahead

[quentinlesceller]

🎉 Wohooo! This RFC has now been merged! 🤸‍♀️
Tracking issue: mimblewimble/grin-wallet#406