From c2d869a46fffd5d00bf54f14870d012568327dec Mon Sep 17 00:00:00 2001 From: Daira Hopwood Date: Thu, 6 Jun 2019 23:30:24 +0100 Subject: [PATCH 001/231] Add ZIPs 68 and 112 (OP_CHECKSEQUENCEVERIFY). Signed-off-by: Daira Hopwood --- zip-0068.rst | 315 +++++++++++++++++++++++++++++++++++++++++ zip-0112.rst | 386 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 701 insertions(+) create mode 100644 zip-0068.rst create mode 100644 zip-0112.rst diff --git a/zip-0068.rst b/zip-0068.rst new file mode 100644 index 000000000..0765c4e31 --- /dev/null +++ b/zip-0068.rst @@ -0,0 +1,315 @@ +.. + + ZIP: 68 + Title: Relative lock-time using consensus-enforced sequence numbers + Credits: Mark Friedenbach , BtcDrak , Nicolas Dorier , kinoshitajona + Category: Consensus + Status: Draft + Created: 2016-06-06 + + +Terminology +=========== + +The key words "MUST" and "MAY" in this document are to be interpreted as described in +RFC 2119. [#RFC2119]_ + +The "Median Time Past" of a block in this document refers to the median time of the prior +PoWMedianBlockSpan blocks, where the constant PoWMedianBlockSpan is defined in [#protocol]_ +section 5.3, and the median function is defined as in [#protocol]_ section 7.6.3. + + +Abstract +======== + +This ZIP introduces relative lock-time (RLT) consensus-enforced semantics of the sequence +number field, to enable a signed transaction input to remain invalid for a defined period +of time after confirmation of its corresponding outpoint. + + +Motivation +========== + +Zcash transactions have a sequence number field for each input, inherited from Bitcoin. +The original idea in Bitcoin appears to have been that a transaction in the mempool would +be replaced by using the same input with a higher sequence value. Although this was not +properly implemented, it assumes miners would prefer higher sequence numbers even if the +lower ones were more profitable to mine. However, a miner acting on profit motives alone +would break that assumption completely. The change described by this ZIP repurposes the +sequence number for new use cases without breaking existing functionality. It also leaves +room for future expansion and other use cases. + +The transaction ``nLockTime`` is used to prevent the mining of a transaction until a +certain date. ``nSequence`` will be repurposed to prevent mining of a transaction until +a certain age of the spent output in blocks or timespan. This, among other uses, allows +bi-directional payment channels as used in [#HTLCs]_ and [#zip-0112]_. + + +Specification +============= + +This specification defines the meaning of sequence numbers for transactions in blocks +after this proposal has activated. + +If bit (1 << 31) of the sequence number is set, then no consensus meaning is applied to +the sequence number and can be included in any block under all currently possible +circumstances. + +If bit (1 << 31) of the sequence number is not set, then the sequence number is +interpreted as an encoded relative lock-time. + +The sequence number encoding MUST be interpreted as follows: + +Bit (1 << 22) determines if the relative lock-time is time-based or block based: +If the bit is set, the relative lock-time specifies a timespan in units of 512 seconds +granularity. The timespan starts from the median-time-past of the output’s previous block, +and ends at the Median Time Past of the previous block. If the bit is not set, the +relative lock-time specifies a number of blocks. + +The flag (1 << 22) is the highest order bit in a 3-byte signed integer for use in +Zcash scripts as a 3-byte ``PUSHDATA`` with ``OP_CHECKSEQUENCEVERIFY`` [#zip-0112]_. + +This specification only interprets 16 bits of the sequence number as relative lock-time, +so a mask of ``0x0000FFFF`` MUST be applied to the sequence field to extract the relative +lock-time. The 16-bit specification allows for a year of relative lock-time and the +remaining bits allow for future expansion. + +:raw-html-m2r:`` + +For time based relative lock-time, 512-second granularity was chosen because Bitcoin +blocks are generated every 600 seconds. [TODO: should this be changed for Zcash?] +So when using block-based or time-based, the same amount of time can be encoded with +the available number of bits. Converting from a sequence number to seconds is performed +by multiplying by 512 = 2^9, or equivalently shifting up by 9 bits. + +When the relative lock-time is time-based, it is interpreted as a minimum block-time +constraint over the input's age. A relative time-based lock-time of zero indicates an +input which can be included in any block. More generally, a relative time-based +lock-time n can be included into any block produced 512 \* n seconds after the mining +date of the output it is spending, or any block thereafter. +The mining date of the output is equal to the Median Time Past of the previous block +which mined it. + +The block produced time is equal to the Median Time Past of its previous block. + +When the relative lock-time is block-based, it is interpreted as a minimum block-height +constraint over the input's age. A relative block-based lock-time of zero indicates an +input which can be included in any block. More generally, a relative block lock-time n +MAY be included n blocks after the mining date of the output it is spending, or any +block thereafter. + +The new rules are not applied to the ``nSequence`` field of the input of the coinbase +transaction. + + +Reference Implementation +======================== + +.. highlight::c++ + + enum { + /* Interpret sequence numbers as relative lock-time constraints. */ + LOCKTIME_VERIFY_SEQUENCE = (1 << 0), + }; + + /* Setting nSequence to this value for every input in a transaction + * disables nLockTime. */ + static const uint32_t SEQUENCE_FINAL = 0xffffffff; + + /* Below flags apply in the context of ZIP 68*/ + /* If this flag set, CTxIn::nSequence is NOT interpreted as a + * relative lock-time. */ + static const uint32_t SEQUENCE_LOCKTIME_DISABLE_FLAG = (1 << 31); + + /* If CTxIn::nSequence encodes a relative lock-time and this flag + * is set, the relative lock-time has units of 512 seconds, + * otherwise it specifies blocks with a granularity of 1. */ + static const uint32_t SEQUENCE_LOCKTIME_TYPE_FLAG = (1 << 22); + + /* If CTxIn::nSequence encodes a relative lock-time, this mask is + * applied to extract that lock-time from the sequence field. */ + static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; + + /* In order to use the same number of bits to encode roughly the + * same wall-clock duration, and because blocks are naturally + * limited to occur every 600s on average, the minimum granularity + * for time-based relative lock-time is fixed at 512 seconds. + * Converting from CTxIn::nSequence to seconds is performed by + * multiplying by 512 = 2^9, or equivalently shifting up by + * 9 bits. */ + static const int SEQUENCE_LOCKTIME_GRANULARITY = 9; + + /** + * Calculates the block height and previous block's Median Time Past at + * which the transaction will be considered final in the context of ZIP 68. + * Also removes from the vector of input heights any entries which did not + * correspond to sequence locked inputs as they do not affect the calculation. + */ + static std::pair CalculateSequenceLocks(const CTransaction &tx, int flags, std::vector* prevHeights, const CBlockIndex& block) + { + assert(prevHeights->size() == tx.vin.size()); + + // Will be set to the equivalent height- and time-based nLockTime + // values that would be necessary to satisfy all relative lock- + // time constraints given our view of block chain history. + // The semantics of nLockTime are the last invalid height/time, so + // use -1 to have the effect of any height or time being valid. + int nMinHeight = -1; + int64_t nMinTime = -1; + + // tx.nVersion is signed integer so requires cast to unsigned otherwise + // we would be doing a signed comparison and half the range of nVersion + // wouldn't support ZIP 68. + bool fEnforceZIP68 = static_cast(tx.nVersion) >= 2 + && flags & LOCKTIME_VERIFY_SEQUENCE; + + // Do not enforce sequence numbers as a relative lock time + // unless we have been instructed to + if (!fEnforceZIP68) { + return std::make_pair(nMinHeight, nMinTime); + } + + for (size_t txinIndex = 0; txinIndex < tx.vin.size(); txinIndex++) { + const CTxIn& txin = tx.vin[txinIndex]; + + // Sequence numbers with the most significant bit set are not + // treated as relative lock-times, nor are they given any + // consensus-enforced meaning at this point. + if (txin.nSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) { + // The height of this input is not relevant for sequence locks + (*prevHeights)[txinIndex] = 0; + continue; + } + + int nCoinHeight = (*prevHeights)[txinIndex]; + + if (txin.nSequence & CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG) { + int64_t nCoinTime = block.GetAncestor(std::max(nCoinHeight-1, 0))->GetMedianTimePast(); + // NOTE: Subtract 1 to maintain nLockTime semantics + // ZIP 68 relative lock times have the semantics of calculating + // the first block or time at which the transaction would be + // valid. When calculating the effective block time or height + // for the entire transaction, we switch to using the + // semantics of nLockTime which is the last invalid block + // time or height. Thus we subtract 1 from the calculated + // time or height. + + // Time-based relative lock-times are measured from the + // smallest allowed timestamp of the block containing the + // txout being spent, which is the Median Time Past of the + // block prior. + nMinTime = std::max(nMinTime, nCoinTime + (int64_t)((txin.nSequence & CTxIn::SEQUENCE_LOCKTIME_MASK) << CTxIn::SEQUENCE_LOCKTIME_GRANULARITY) - 1); + } else { + nMinHeight = std::max(nMinHeight, nCoinHeight + (int)(txin.nSequence & CTxIn::SEQUENCE_LOCKTIME_MASK) - 1); + } + } + + return std::make_pair(nMinHeight, nMinTime); + } + + static bool EvaluateSequenceLocks(const CBlockIndex& block, std::pair lockPair) + { + assert(block.pprev); + int64_t nBlockTime = block.pprev->GetMedianTimePast(); + if (lockPair.first >= block.nHeight || lockPair.second >= nBlockTime) + return false; + + return true; + } + + bool SequenceLocks(const CTransaction &tx, int flags, std::vector* prevHeights, const CBlockIndex& block) + { + return EvaluateSequenceLocks(block, CalculateSequenceLocks(tx, flags, prevHeights, block)); + } + + bool CheckSequenceLocks(const CTransaction &tx, int flags) + { + AssertLockHeld(cs_main); + AssertLockHeld(mempool.cs); + + CBlockIndex* tip = chainActive.Tip(); + CBlockIndex index; + index.pprev = tip; + // CheckSequenceLocks() uses chainActive.Height()+1 to evaluate + // height based locks because when SequenceLocks() is called within + // ConnectBlock(), the height of the block *being* + // evaluated is what is used. + // Thus if we want to know if a transaction can be part of the + // *next* block, we need to use one more than chainActive.Height() + index.nHeight = tip->nHeight + 1; + + // pcoinsTip contains the UTXO set for chainActive.Tip() + CCoinsViewMemPool viewMemPool(pcoinsTip, mempool); + std::vector prevheights; + prevheights.resize(tx.vin.size()); + for (size_t txinIndex = 0; txinIndex < tx.vin.size(); txinIndex++) { + const CTxIn& txin = tx.vin[txinIndex]; + CCoins coins; + if (!viewMemPool.GetCoins(txin.prevout.hash, coins)) { + return error("%s: Missing input", __func__); + } + if (coins.nHeight == MEMPOOL_HEIGHT) { + // Assume all mempool transaction confirm in the next block + prevheights[txinIndex] = tip->nHeight + 1; + } else { + prevheights[txinIndex] = coins.nHeight; + } + } + + std::pair lockPair = CalculateSequenceLocks(tx, flags, &prevheights, index); + return EvaluateSequenceLocks(index, lockPair); + } + + +Acknowledgments +=============== + +This ZIP is closely based on BIP 68, authored by Mark Friedenbach, BtcDrak, Nicolas Dorier, +and kinoshitajona. + +Credit goes to Gregory Maxwell for providing a succinct and clear description of the behavior of +this change, which became the basis of the BIP text. + + +Deployment +========== + +This ZIP must be deployed simultaneously with [#zip-0112]_. + + +Compatibility +============= + +The only use of sequence numbers by the zcashd reference client software is to disable checking +the ``nLockTime`` constraints in a transaction. The semantics of that application are preserved +by this ZIP. + +As can be seen from the specification section, a number of bits are undefined by this ZIP to allow for other use cases by setting bit (1 << 31) as the remaining 31 bits have no meaning under this ZIP. Additionally, bits (1 << 23) through (1 << 30) inclusive have no meaning at all when bit (1 << 31) is unset. + +Additionally, this ZIP specifies only 16 bits to actually encode relative lock-time meaning a further 6 are unused (1 << 16 through 1 << 21 inclusive). This allows the possibility to increase granularity by soft-fork, or for increasing the maximum possible relative lock-time in the future. + +The most efficient way to calculate sequence number from relative lock-time is with bit masks and shifts: + +.. hightlight::c++ + + // 0 <= nHeight < 65,535 blocks (1.25 years) + nSequence = nHeight; + nHeight = nSequence & 0x0000ffff; + + // 0 <= nTime < 33,554,431 seconds (1.06 years) + nSequence = (1 << 22) | (nTime >> 9); + nTime = (nSequence & 0x0000ffff) << 9; + + +References +========== + +Bitcoin mailing list discussion: https://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg07864.html + +BIP9: https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki + +ZIP112: https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki + +BIP113: https://github.com/bitcoin/bips/blob/master/bip-0113.mediawiki + +Hashed Timelock Contracts (HTLCs): https://github.com/ElementsProject/lightning/raw/master/doc/deployable-lightning.pdf diff --git a/zip-0112.rst b/zip-0112.rst new file mode 100644 index 000000000..d13146a52 --- /dev/null +++ b/zip-0112.rst @@ -0,0 +1,386 @@ +.. + + ZIP: 112 + Title: CHECKSEQUENCEVERIFY + Author: Daira Hopwood + Credits: BtcDrak , Mark Friedenbach , Eric Lombrozo + Category: Consensus + Status: Draft + Created: 2019-06-06 + License: MIT + + +Terminology +=========== + +The key word "MUST" in this document is to be interpreted as described in RFC 2119. [#RFC2119]_ + + +Abstract +======== + +This ZIP describes a new opcode (``CHECKSEQUENCEVERIFY``) for the Zcash scripting system, that +in combination with ZIP 68 allows execution pathways of a script to be restricted based on the +age of the output being spent. + + +Motivation +========== + +ZIP 68 repurposes the transaction ``nSequence`` field meaning by giving sequence numbers new +consensus-enforced semantics as a relative lock-time. However, there is no way to build Zcash +scripts to make decisions based on this field. + +By making the ``nSequence`` field accessible to script, it becomes possible to construct code +pathways that only become accessible some minimum time after proof-of-publication. This enables +a wide variety of applications in phased protocols such as escrow, payment channels, or +bidirectional pegs. + + +Specification +============= + +``CHECKSEQUENCEVERIFY`` redefines the existing ``NOP3`` opcode. When executed, if any of the +following conditions are true, the script interpreter MUST terminate with an error: + +* the stack is empty; or +* the top item on the stack is less than 0; or +* the top item on the stack has the disable flag (1 << 31) unset; and any of the following hold: + + * the transaction version is less than 2; or + * the transaction input sequence number disable flag (1 << 31) is set; or + * the relative lock-time type is not the same; or + * the top stack item is greater than the transaction input sequence (when masked according to + [#zip-0068]_; + +Otherwise, script execution MUST continue as if a ``NOP`` had been executed. + +ZIP 68 prevents a non-final transaction from being selected for inclusion in a block until the +corresponding input has reached the specified age, as measured in block-height or block-time. By +comparing the argument to ``CHECKSEQUENCEVERIFY`` against the ``nSequence`` field, we indirectly +verify a desired minimum age of the the output being spent; until that relative age has been +reached any script execution pathway including the ``CHECKSEQUENCEVERIFY`` will fail to validate, +causing the transaction not to be selected for inclusion in a block. + + +Use cases +========= + +Contracts With Expiration Deadlines +----------------------------------- + +Escrow with Timeout +''''''''''''''''''' + +An escrow that times out automatically 30 days after being funded can be established in the +following way. Alice, Bob and Escrow create a 2-of-3 address with the following redeem script:: + + IF + 2 3 CHECKMULTISIG + ELSE + "30d" CHECKSEQUENCEVERIFY DROP + CHECKSIG + ENDIF + +At any time funds can be spent using signatures from any two of Alice, Bob or the Escrow. + +After 30 days Alice can sign alone. + +The clock does not start ticking until the payment to the escrow address confirms. + + +Retroactive Invalidation +------------------------ + +In many instances, we would like to create contracts that can be revoked in case of some future +event. However, given the immutable nature of the block chain, it is practically impossible to +retroactively invalidate a previous commitment that has already confirmed. The only mechanism we +really have for retroactive invalidation is block chain reorganization which, for fundamental +security reasons, is designed to be very hard and very expensive to do. + +Despite this limitation, we do have a way to provide something functionally similar to retroactive +invalidation while preserving irreversibility of past commitments using ``CHECKSEQUENCEVERIFY``. +By constructing scripts with multiple branches of execution where one or more of the branches are +delayed we provide a time window in which someone can supply an invalidation condition that allows +the output to be spent, effectively invalidating the would-be delayed branch and potentially +discouraging another party from broadcasting the transaction in the first place. If the +invalidation condition does not occur before the timeout, the delayed branch becomes spendable, +honoring the original contract. + +Some more specific applications of this idea: + +Hash Time-Locked Contracts +'''''''''''''''''''''''''' + +Hash Time-Locked Contracts (HTLCs) provide a general mechanism for off-chain contract negotiation. +An execution pathway can be made to require knowledge of a secret (a hash preimage) that can be +presented within an invalidation time window. By sharing the secret it is possible to guarantee +to the counterparty that the transaction will never be broadcast since this would allow the +counterparty to claim the output immediately while one would have to wait for the time window +to pass. If the secret has not been shared, the counterparty will be unable to use the instant +pathway and the delayed pathway must be used instead. + +Bidirectional Payment Channels +'''''''''''''''''''''''''''''' + +Scriptable relative locktime provides a predictable amount of time to respond in the event a +counterparty broadcasts a revoked transaction: Absolute locktime necessitates closing the channel +and reopening it when getting close to the timeout, whereas with relative locktime, the clock +starts ticking the moment the transaction confirms in a block. It also provides a means to know +exactly how long to wait (in number of blocks) before funds can be pulled out of the channel in +the event of a noncooperative counterparty. + +Lightning Network +''''''''''''''''' + +The lightning network protocol [#lightning]_ extends the bidirectional payment channel idea to +allow for payments to be routed over multiple bidirectional payment channel hops. + +These channels are based on an anchor transaction that requires a 2-of-2 multisig from Alice and +Bob, and a series of revocable commitment transactions that spend the anchor transaction. The +commitment transaction splits the funds from the anchor between Alice and Bob and the latest +commitment transaction may be published by either party at any time, finalising the channel. + +Ideally then, a revoked commitment transaction would never be able to be successfully spent; and the +latest commitment transaction would be able to be spent very quickly. + +To allow a commitment transaction to be effectively revoked, Alice and Bob have slightly different +versions of the latest commitment transaction. In Alice's version, any outputs in the commitment +transaction that pay Alice also include a forced delay, and an alternative branch that allows Bob +to spend the output if he knows that transaction's revocation code. In Bob's version, payments +to Bob are similarly encumbered. When Alice and Bob negotiate new balances and new commitment +transactions, they also reveal the old revocation code, thus committing to not relaying the old +transaction. + +A simple output, paying to Alice might then look like:: + + HASH160 EQUAL + IF + + ELSE + "24h" CHECKSEQUENCEVERIFY DROP + + ENDIF + CHECKSIG + +This allows Alice to publish the latest commitment transaction at any time and spend the funds +after 24 hours, but also ensures that if Alice relays a revoked transaction, that Bob has 24 hours +to claim the funds. + +With ``CHECKLOCKTIMEVERIFY``, this would look like:: + + HASH160 EQUAL + IF + + ELSE + "2015/12/15" CHECKLOCKTIMEVERIFY DROP + + ENDIF + CHECKSIG + +This form of transaction would mean that if the anchor is unspent on 2015/12/16, Alice can use this +commitment even if it has been revoked, simply by spending it immediately, giving no time for Bob +to claim it. + +This means that the channel has a deadline that cannot be pushed back without hitting the blockchain; +and also that funds may not be available until the deadline is hit. ``CHECKSEQUENCEVERIFY`` allows +you to avoid making such a tradeoff. + +Hashed Time-Lock Contracts (HTLCs) make this slightly more complicated, since in principle they may +pay either Alice or Bob, depending on whether Alice discovers a secret R, or a timeout is reached, +but the same principle applies -- the branch paying Alice in Alice's commitment transaction gets a +delay, and the entire output can be claimed by the other party if the revocation secret is known. +With ``CHECKSEQUENCEVERIFY``, a HTLC payable to Alice might look like the following in Alice's +commitment transaction:: + + HASH160 DUP EQUAL + IF + "24h" CHECKSEQUENCEVERIFY + 2DROP + + ELSE + EQUAL + NOTIF + "2015/10/20 10:33" CHECKLOCKTIMEVERIFY DROP + ENDIF + + ENDIF + CHECKSIG + +and correspondingly in Bob's commitment transaction:: + + HASH160 DUP EQUAL + SWAP EQUAL ADD + IF + + ELSE + "2015/10/20 10:33" CHECKLOCKTIMEVERIFY + "24h" CHECKSEQUENCEVERIFY + 2DROP + + ENDIF + CHECKSIG + +Note that both ``CHECKSEQUENCEVERIFY`` and ``CHECKLOCKTIMEVERIFY`` are used in the final branch +above to ensure Bob cannot spend the output until after both the timeout is complete and Alice +has had time to reveal the revocation secret. + +See also the 'Deployable Lightning' paper [#deployable-lightning]_. + + +2-Way Pegged Sidechains +----------------------- + +The 2-way pegged sidechain requires a new ``REORGPROOFVERIFY`` opcode, the semantics of which +are outside the scope of this ZIP. ``CHECKSEQUENCEVERIFY`` is used to make sure that sufficient +time has passed since the return peg was posted to publish a reorg proof:: + + IF + lockTxHeight nlocktxOut [] reorgBounty Hash160(<...>) REORGPROOFVERIFY + ELSE + withdrawLockTime CHECKSEQUENCEVERIFY DROP HASH160 p2shWithdrawDest EQUAL + ENDIF + + + +Reference Implementation +======================== + +.. highlight::c++ + + /* Below flags apply in the context of ZIP 68 */ + /* If this flag set, CTxIn::nSequence is NOT interpreted as a + * relative lock-time. */ + static const uint32_t SEQUENCE_LOCKTIME_DISABLE_FLAG = (1 << 31); + + /* If CTxIn::nSequence encodes a relative lock-time and this flag + * is set, the relative lock-time has units of 512 seconds, + * otherwise it specifies blocks with a granularity of 1. */ + static const uint32_t SEQUENCE_LOCKTIME_TYPE_FLAG = (1 << 22); + + /* If CTxIn::nSequence encodes a relative lock-time, this mask is + * applied to extract that lock-time from the sequence field. */ + static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; + + case OP_NOP3: + { + if (!(flags & SCRIPT_VERIFY_CHECKSEQUENCEVERIFY)) { + // not enabled; treat as a NOP3 + if (flags & SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS) { + return set_error(serror, SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS); + } + break; + } + + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + + // Note that elsewhere numeric opcodes are limited to + // operands in the range -2**31+1 to 2**31-1, however it is + // legal for opcodes to produce results exceeding that + // range. This limitation is implemented by CScriptNum's + // default 4-byte limit. + // + // Thus as a special case we tell CScriptNum to accept up + // to 5-byte bignums, which are good until 2**39-1, well + // beyond the 2**32-1 limit of the nSequence field itself. + const CScriptNum nSequence(stacktop(-1), fRequireMinimal, 5); + + // In the rare event that the argument may be < 0 due to + // some arithmetic being done first, you can always use + // 0 MAX CHECKSEQUENCEVERIFY. + if (nSequence < 0) + return set_error(serror, SCRIPT_ERR_NEGATIVE_LOCKTIME); + + // To provide for future soft-fork extensibility, if the + // operand has the disabled lock-time flag set, + // CHECKSEQUENCEVERIFY behaves as a NOP. + if ((nSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) != 0) + break; + + // Compare the specified sequence number with the input. + if (!checker.CheckSequence(nSequence)) + return set_error(serror, SCRIPT_ERR_UNSATISFIED_LOCKTIME); + + break; + } + + bool TransactionSignatureChecker::CheckSequence(const CScriptNum& nSequence) const + { + // Relative lock times are supported by comparing the passed + // in operand to the sequence number of the input. + const int64_t txToSequence = (int64_t)txTo->vin[nIn].nSequence; + + // Fail if the transaction's version number is not set high + // enough to trigger ZIP 68 rules. + if (static_cast(txTo->nVersion) < 2) + return false; + + // Sequence numbers with their most significant bit set are not + // consensus constrained. Testing that the transaction's sequence + // number do not have this bit set prevents using this property + // to get around a CHECKSEQUENCEVERIFY check. + if (txToSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) + return false; + + // Mask off any bits that do not have consensus-enforced meaning + // before doing the integer comparisons + const uint32_t nLockTimeMask = CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG | CTxIn::SEQUENCE_LOCKTIME_MASK; + const int64_t txToSequenceMasked = txToSequence & nLockTimeMask; + const CScriptNum nSequenceMasked = nSequence & nLockTimeMask; + + // There are two kinds of nSequence: lock-by-blockheight + // and lock-by-blocktime, distinguished by whether + // nSequenceMasked < CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG. + // + // We want to compare apples to apples, so fail the script + // unless the type of nSequenceMasked being tested is the same as + // the nSequenceMasked in the transaction. + if (!( + (txToSequenceMasked < CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG && nSequenceMasked < CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG) || + (txToSequenceMasked >= CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG && nSequenceMasked >= CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG) + )) + return false; + + // Now that we know we're comparing apples-to-apples, the + // comparison is a simple numeric one. + if (nSequenceMasked > txToSequenceMasked) + return false; + + return true; + } + + +Deployment +========== + +At the time of writing it has not been decided which network upgrade (if any) will implement this +proposal. + +This ZIP must be deployed simultaneously with ZIP 68 [#zip-0068]_. + + +Acknowledgements +================ + +This ZIP is closely based on BIP 112, authored by BtcDrak. + +Mark Friedenbach invented the application of sequence numbers to achieve relative lock-time, and +wrote the reference implementation of ``CHECKSEQUENCEVERIFY`` for Bitcoin. + +The Bitcoin reference implementation and BIP 112 was based heavily on work done by Peter Todd for +the closely related BIP 65. Eric Lombrozo and Anthony Towns contributed example use cases. + + +References +========== + +.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels `_ +.. [#protocol] `Zcash Protocol Specification, Version 2019.0.1 or later [Overwinter+Sapling] `_ +.. [#bip-0065] `BIP 65: OP_CHECKLOCKTIMEVERIFY `_ +.. [#zip-0068] `ZIP 68: Relative lock-time through consensus-enforced sequence numbers `_ +.. [#deployable-lightning] `Deployable Lightning `_ +.. [#lightning] `Lightning Network paper `_ +.. [#HTLCs] `HTLCs using OP_CHECKSEQUENCEVERIFY/OP_LOCKTIMEVERIFY and revocation hashes `_ +.. [#scaling] `Scaling Bitcoin to Billions of Transactions Per Day `_ +.. [#micropayments] `Jeremy Spilman, Micropayment Channels `_ From d8b98a708b5ca58fd53ff9f5fd45dd4a8007b447 Mon Sep 17 00:00:00 2001 From: Daira Hopwood Date: Tue, 18 Jun 2019 20:04:23 +0100 Subject: [PATCH 002/231] Update ZIPs 68 and 112, and add ZIP 113. Signed-off-by: Daira Hopwood --- zip-0068.rst | 110 +++++++++++++++++++++++++--------------------- zip-0112.rst | 6 ++- zip-0113.rst | 122 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+), 52 deletions(-) create mode 100644 zip-0113.rst diff --git a/zip-0068.rst b/zip-0068.rst index 0765c4e31..e53930513 100644 --- a/zip-0068.rst +++ b/zip-0068.rst @@ -1,8 +1,11 @@ -.. +:: ZIP: 68 Title: Relative lock-time using consensus-enforced sequence numbers - Credits: Mark Friedenbach , BtcDrak , Nicolas Dorier , kinoshitajona + Credits: Mark Friedenbach + BtcDrak + Nicolas Dorier + kinoshitajona Category: Consensus Status: Draft Created: 2016-06-06 @@ -14,9 +17,8 @@ Terminology The key words "MUST" and "MAY" in this document are to be interpreted as described in RFC 2119. [#RFC2119]_ -The "Median Time Past" of a block in this document refers to the median time of the prior -PoWMedianBlockSpan blocks, where the constant PoWMedianBlockSpan is defined in [#protocol]_ -section 5.3, and the median function is defined as in [#protocol]_ section 7.6.3. +The "Median Time Past" of a block in this document is to be interpreted as described in +[#zip-0113]_. Abstract @@ -42,7 +44,7 @@ room for future expansion and other use cases. The transaction ``nLockTime`` is used to prevent the mining of a transaction until a certain date. ``nSequence`` will be repurposed to prevent mining of a transaction until a certain age of the spent output in blocks or timespan. This, among other uses, allows -bi-directional payment channels as used in [#HTLCs]_ and [#zip-0112]_. +bi-directional payment channels as used in [#deployable-lightning]_ and [#zip-0112]_. Specification @@ -61,40 +63,43 @@ interpreted as an encoded relative lock-time. The sequence number encoding MUST be interpreted as follows: Bit (1 << 22) determines if the relative lock-time is time-based or block based: -If the bit is set, the relative lock-time specifies a timespan in units of 512 seconds -granularity. The timespan starts from the median-time-past of the output’s previous block, +If the bit is set, the relative lock-time specifies a timespan in units of 32 seconds +granularity. The timespan starts from the Median Time Past of the output’s previous block, and ends at the Median Time Past of the previous block. If the bit is not set, the relative lock-time specifies a number of blocks. +Note: the 64-second time unit differs from Bitcoin's BIP 68, which uses a 512-second +time unit. + The flag (1 << 22) is the highest order bit in a 3-byte signed integer for use in Zcash scripts as a 3-byte ``PUSHDATA`` with ``OP_CHECKSEQUENCEVERIFY`` [#zip-0112]_. -This specification only interprets 16 bits of the sequence number as relative lock-time, -so a mask of ``0x0000FFFF`` MUST be applied to the sequence field to extract the relative -lock-time. The 16-bit specification allows for a year of relative lock-time and the -remaining bits allow for future expansion. +This specification only interprets 22 bits of the sequence number as relative lock-time, +so a mask of ``0x003FFFFF`` MUST be applied to the sequence field to extract the relative +lock-time. The 22-bit specification allows for over 8.5 years of relative lock-time. -:raw-html-m2r:`` +.. image:: encoding.png + :alt:A 32-bit field with 'Disable Flag' at bit (1 << 31), 'Type Flag' at bit (1 << 22), and 'Value' in the low 22 bits. -For time based relative lock-time, 512-second granularity was chosen because Bitcoin -blocks are generated every 600 seconds. [TODO: should this be changed for Zcash?] -So when using block-based or time-based, the same amount of time can be encoded with -the available number of bits. Converting from a sequence number to seconds is performed -by multiplying by 512 = 2^9, or equivalently shifting up by 9 bits. +For time-based relative lock-time, 64-second granularity was chosen because the block +target spacing for Zcash, after activation of the Blossom network upgrade, is 75 seconds. +So when using block-based or time-based, roughly the same amount of time can be encoded +with the available number of bits. Converting from a sequence number to seconds is +performed by multiplying by 64. When the relative lock-time is time-based, it is interpreted as a minimum block-time constraint over the input's age. A relative time-based lock-time of zero indicates an input which can be included in any block. More generally, a relative time-based -lock-time n can be included into any block produced 512 \* n seconds after the mining +lock-time n can be included into any block produced 64 \* n seconds after the mining date of the output it is spending, or any block thereafter. The mining date of the output is equal to the Median Time Past of the previous block -which mined it. +that mined it. The block produced time is equal to the Median Time Past of its previous block. When the relative lock-time is block-based, it is interpreted as a minimum block-height constraint over the input's age. A relative block-based lock-time of zero indicates an -input which can be included in any block. More generally, a relative block lock-time n +input that can be included in any block. More generally, a relative block lock-time n MAY be included n blocks after the mining date of the output it is spending, or any block thereafter. @@ -116,7 +121,7 @@ Reference Implementation * disables nLockTime. */ static const uint32_t SEQUENCE_FINAL = 0xffffffff; - /* Below flags apply in the context of ZIP 68*/ + /* Below flags apply in the context of ZIP 68. */ /* If this flag set, CTxIn::nSequence is NOT interpreted as a * relative lock-time. */ static const uint32_t SEQUENCE_LOCKTIME_DISABLE_FLAG = (1 << 31); @@ -128,16 +133,16 @@ Reference Implementation /* If CTxIn::nSequence encodes a relative lock-time, this mask is * applied to extract that lock-time from the sequence field. */ - static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; + static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x003fffff; /* In order to use the same number of bits to encode roughly the * same wall-clock duration, and because blocks are naturally - * limited to occur every 600s on average, the minimum granularity - * for time-based relative lock-time is fixed at 512 seconds. + * limited to occur every 75s on average after Blossom activation, + * the minimum granularity for time-based relative lock-time is + * fixed at 64 seconds. * Converting from CTxIn::nSequence to seconds is performed by - * multiplying by 512 = 2^9, or equivalently shifting up by - * 9 bits. */ - static const int SEQUENCE_LOCKTIME_GRANULARITY = 9; + * multiplying by 64, or equivalently shifting up by 6 bits. */ + static const int SEQUENCE_LOCKTIME_GRANULARITY = 6; /** * Calculates the block height and previous block's Median Time Past at @@ -264,17 +269,20 @@ Reference Implementation Acknowledgments =============== -This ZIP is closely based on BIP 68, authored by Mark Friedenbach, BtcDrak, Nicolas Dorier, -and kinoshitajona. +This ZIP is based on BIP 68, authored by Mark Friedenbach, BtcDrak, Nicolas Dorier, and +kinoshitajona. -Credit goes to Gregory Maxwell for providing a succinct and clear description of the behavior of -this change, which became the basis of the BIP text. +Credit goes to Gregory Maxwell for providing a succinct and clear description of the behavior +of this change, which became the basis of the BIP text. Deployment ========== -This ZIP must be deployed simultaneously with [#zip-0112]_. +At the time of writing it has not been decided which network upgrade (if any) will implement this +proposal. + +This ZIP is designed to be deployed simultaneously with [#zip-0112]_ and [#zip-0113]_. Compatibility @@ -284,32 +292,34 @@ The only use of sequence numbers by the zcashd reference client software is to d the ``nLockTime`` constraints in a transaction. The semantics of that application are preserved by this ZIP. -As can be seen from the specification section, a number of bits are undefined by this ZIP to allow for other use cases by setting bit (1 << 31) as the remaining 31 bits have no meaning under this ZIP. Additionally, bits (1 << 23) through (1 << 30) inclusive have no meaning at all when bit (1 << 31) is unset. +As can be seen from the specification section, a number of bits are undefined by this ZIP to +allow for other use cases by setting bit (1 << 31) as the remaining 31 bits have no meaning +under this ZIP. Additionally, bits (1 << 23) through (1 << 30) inclusive have no meaning at all +when bit (1 << 31) is unset. -Additionally, this ZIP specifies only 16 bits to actually encode relative lock-time meaning a further 6 are unused (1 << 16 through 1 << 21 inclusive). This allows the possibility to increase granularity by soft-fork, or for increasing the maximum possible relative lock-time in the future. +Unlike BIP 68 in Bitcoin, all of the low 22 bits are used for the value. This reflects the fact +that blocks are more frequent (75 seconds instead of 600 seconds), and so more bits are needed +to obtain approximately the same range of time. -The most efficient way to calculate sequence number from relative lock-time is with bit masks and shifts: +The most efficient way to calculate sequence number from relative lock-time is with bit masks +and shifts: .. hightlight::c++ - // 0 <= nHeight < 65,535 blocks (1.25 years) + // 0 <= nHeight <= 4194303 blocks (~10 years at post-Blossom block target spacing) nSequence = nHeight; - nHeight = nSequence & 0x0000ffff; + nHeight = nSequence & 0x003fffff; - // 0 <= nTime < 33,554,431 seconds (1.06 years) - nSequence = (1 << 22) | (nTime >> 9); - nTime = (nSequence & 0x0000ffff) << 9; + // 0 <= nTime <= 268435392 seconds (~8.5 years) + nSequence = (1 << 22) | (nTime >> 6); + nTime = (nSequence & 0x003fffff) << 6; References ========== -Bitcoin mailing list discussion: https://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg07864.html - -BIP9: https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki - -ZIP112: https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki - -BIP113: https://github.com/bitcoin/bips/blob/master/bip-0113.mediawiki - -Hashed Timelock Contracts (HTLCs): https://github.com/ElementsProject/lightning/raw/master/doc/deployable-lightning.pdf +.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels `_ +.. [#mailing-list] `Bitcoin mailing list discussion `_ +.. [#zip-0112] `ZIP 112: CHECKSEQUENCEVERIFY `_ +.. [#zip-0113] `ZIP 113: Median Time Past as endpoint for lock-time calculations `_ +.. [#deployable-lightning] `Reaching The Ground With Lightning (draft 0.2) `_ diff --git a/zip-0112.rst b/zip-0112.rst index d13146a52..a6c130c0b 100644 --- a/zip-0112.rst +++ b/zip-0112.rst @@ -1,9 +1,11 @@ -.. +:: ZIP: 112 Title: CHECKSEQUENCEVERIFY Author: Daira Hopwood - Credits: BtcDrak , Mark Friedenbach , Eric Lombrozo + Credits: BtcDrak + Mark Friedenbach + Eric Lombrozo Category: Consensus Status: Draft Created: 2019-06-06 diff --git a/zip-0113.rst b/zip-0113.rst new file mode 100644 index 000000000..d7d657a4c --- /dev/null +++ b/zip-0113.rst @@ -0,0 +1,122 @@ +:: + + ZIP: 113 + Title: Median Time Past as endpoint for lock-time calculations + Author: Daira Hopwood + Credits: Thomas Kerin + Mark Friedenbach + Gregory Maxwell + Category: Consensus + Status: Draft + Created: 2019-06-07 + License: MIT + + +Terminology +=========== + +The key words "MUST" and "MAY" in this document are to be interpreted as described in +RFC 2119. [#RFC2119]_ + + +Abstract +======== + +This ZIP is a proposal to redefine the semantics used in determining a time-locked +transaction's eligibility for inclusion in a block. The median of the last PoWMedianBlockSpan +(11) blocks is used instead of the block's timestamp, ensuring that it increases +monotonically with each block. + + +Motivation +========== + +At present, transactions are excluded from inclusion in a block if the present time or block +height is less than or equal to that specified in the locktime. Since the consensus rules +do not mandate strict ordering of block timestamps, this has the unfortunate outcome of +creating a perverse incentive for miners to lie about the time of their blocks in order to +collect more fees by including transactions that by wall clock determination have not yet +matured. + +This ZIP proposes comparing the locktime against the median of the past PoWMedianBlockSpan +blocks' timestamps, rather than the timestamp of the block including the transaction. +Existing consensus rules guarantee this value to monotonically advance, thereby removing +the capability for miners to claim more transaction fees by lying about the timestamps of +their block. + +This proposal seeks to ensure reliable behaviour in locktime calculations as required by +[#bip-0065]_ (CHECKLOCKTIMEVERIFY) and matching the behavior of [#zip-0112]_ (CHECKSEQUENCEVERIFY). +This also matches the use of Median Time Past in difficulty adjustment as specified in +section 7.6.3 of [#protocol]_. + + +Specification +============= + +Let PoWMedianBlockSpan be as defined in [#protocol]_ section 5.3, and let the median +function be as defined in [#protocol]_ section 7.6.3. + +The Median Time Past of a block is specified as the median of the timestamps of the +prior PoWMedianBlockSpan blocks, as calculated by MedianTime(height) in [#protocol]_ +section 7.6.3. + +The values for transaction locktime remain unchanged. The difference is only in the +calculation determining whether a transaction can be included. After activation of +this ZIP, lock-time constraints of a transaction MUST be checked according to the +Median Time Past of the transaction's block. + +[FIXME make this a proper specification, independent of the zcashd implementation.] + +Lock-time constraints are checked by the consensus method ``IsFinalTx()``. This method +takes the block time as one parameter. This ZIP proposes that after activation calls to +``IsFinalTx()`` within consensus code use the return value of ``GetMedianTimePast(pindexPrev)`` +instead. + +The new rule applies to all transactions, including the coinbase transaction. + + +Reference Implementation +======================== + +This will be based on `Bitcoin PR 6566 `_. + + +Acknowledgements +================ + +This ZIP is based on BIP 113, authored by Thomas Kerin and Mark Friedenbach. + +Mark Friedenbach designed and authored the reference implementation for Bitcoin. + +Gregory Maxwell came up with the original idea, in #bitcoin-wizards on +`2013-07-16 `_ and +`2013-07-17 `_. + + +Deployment +========== + +At the time of writing it has not been decided which network upgrade (if any) will implement this +proposal. + +This ZIP is designed to be deployed simultaneously with [#zip-0068]_ and [#zip-0112]_. + + +Compatibility +============= + +At the post-Blossom block target spacing of 75 seconds, transactions generated using time-based +lock-time will take approximately 7.5 minutes longer to confirm than would be expected under the +old rules. This is not known to introduce any compatibility concerns with existing protocols. +This delay is less than in Bitcoin due to the faster block target spacing in Zcash. + + +References +========== + +.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels `_ +.. [#protocol] `Zcash Protocol Specification, Version 2019.0.1 or later [Overwinter+Sapling+Blossom] `_ +.. [#bip-0065] `BIP 65: OP_CHECKLOCKTIMEVERIFY `_ +.. [#zip-0068] `ZIP 68: Consensus-enforced transaction replacement signaled via sequence numbers `_ +.. [#zip-0112] `ZIP 112: CHECKSEQUENCEVERIFY `_ + From ffee09963f5e2ccb83448ebd5a062369bfc27530 Mon Sep 17 00:00:00 2001 From: Daira Hopwood Date: Tue, 18 Jun 2019 20:06:03 +0100 Subject: [PATCH 003/231] Move ZIP 68 into directory and add encoding diagram. Signed-off-by: Daira Hopwood --- zip-0068/encoding.png | Bin 0 -> 6252 bytes zip-0068.rst => zip-0068/zip-0068.rst | 0 2 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 zip-0068/encoding.png rename zip-0068.rst => zip-0068/zip-0068.rst (100%) diff --git a/zip-0068/encoding.png b/zip-0068/encoding.png new file mode 100644 index 0000000000000000000000000000000000000000..a244e6ba64f9254121dfe8040843146468ca0c25 GIT binary patch literal 6252 zcmd5=2T)V%wvJu#Af5xr0TfUWq$@>4nl$OXNeGEYr3ON8LBNVil`bGXgdR!=5Fl_A z=}iJjfKXHjEf5Zbnn3dKo|!vu=FXdU=gpgWZ_n(t*6h94{A+#t``6y{y|Xko;N%eC z0001-MuvLU0Kjq5QCxlY^pU;&aC7{~oql9upm*etj(y#hs3UjI*U&Kl0J!kw=Xoqm z`hwum#WR6MX8LF5PM23t{R>^tYXA6Mu0!Ky!bmE`( z74#^4=F53xFQSh9(|%Sy3hy=jZ1`{c|2_Rbx9~qG{$J94UC{0Lp$Aluwm25Hf-jB@ zqD@T{E1%&KTHVarrjkNDG>nJ{Bkl~67qK$4AKKta=%XmB1ulKV!#{ocB)3j~cGmGV z)wyLx)~O+I-+zZTO{ORdM{HRgWsKMS@$ODEu-^UqX4a=$Yap;N7(otBfUc0_JJ%pk=r9hCM%PwTg^e&;A(VFcXwKf&kJd4pW-xhrG67&E~{#|vttA|kW{Z4`jw7FSGIW}rA zCG(8!WJ~ht!)hp35r2H~eV?Y_Y0@;6hnaQ#5B@SEvxKXj*66*}^(dDwgR$K@M_d5> zCusj>#J}+3AA|z@_q6&i(j6q3(#2vxs`$2PazC=B=4Nb6jK9a=CZ)fn)iqPW1q8H6 zPLp*aQj|M7I-J315Y}I=nxCJ484ni{5|UdF2$6R*Y|1l%rc2rNQTq9R{q+drk&zLH zwOc&@h_(NquwG@M)`YtUJ1EN2ttUX>QBjcriZV{8ucgg(+5HI5U;GYl@%;W1>xUks z_4Vg}Bz3ZW_or|9g_(u4Tgz21^Yd68K^<+DdN|m9EtEH*-CH>O8=rDwe8Cal#^x*T-T}6R>6D4TA(56M3&;#IB|PgP4ZFNK=h! zSqb6pq`4+(#C+;V}Mj!N8`O6)oo8}aAbmL_lDJzx%TCW{d0ZS zK#QEG+<4Rq%YdBBd%bJ9M2Xl(m{qc)0N5vS<}%TG4f58e3ybXCV0>dC%ki{R-t>(} z!n*ImZN}A#-Uu8es^{Ux8;F>n)QyRBjCgAhij;@K1YI#&W zjLpqQ%3cjJTz}le)yadR*iF^thnwdb4BUh4p0IU{D0`)*6rhyN!{D8QQ;a+294jOG z%1`AAr}c^3e^GwEPt)ZyME7dMl6cyD$~v!)T{Ef>;S3BAaegVa$Bm*Je;jozXq4At zCcbSDQ1*G=75q*{yjS?>))#JvrA)GrWu*@vj_z;p`c4LkW=%c;u>4376Sk>S=i&KL zbWozCvj6I2uddNuxOm9}*?`*LW8`ZFZ9C<_`Ks&jB;EL(6%Y@0wLJ3fMp`bz*JQh(E;|3kUV$ z3k{;(gLN*b{elv4H2$EXBTJpG;yQfeG1R3BuZg^Ob^RRQf8V*Yus@t)US})L$knSH0J^Z8VM`s#$CD&6t z#H_{aYcF}0h_vGlG6}2hcMmCBI|jI++A6JVXJ;_E+rfgoao3b~4tH2jvo#|bFfXF6 zOYVMo6i-vJ+*Pl&?52Zb=EXhdHgU<+vC^`fE5#kds8Kym?77V{OcbRLV$r^ zb^W=l_e4;{n_wHYC1SlWAH5yuzMh`|Ju+9Bh)OJ| zdiMuvxAZ%*pQg0KoIlL}Xmv~6d3Q?p2oUN^6@yZ8y+aU{8oM=<58Mn83)5Rd-WuikLDX@Xu1=LZJGz$Q zR_fz2Oo#Zl+O-wT@-|cV%UMNv9^kw!h3jW zL{-mh+h_L)G&N&qrdn)INby7#_NZ<(OzLrcU0Lw)T*n_P|He9_Vka>6we(|c%&Zdd z7TC)b(5VSJQ|d_%_Joj_BvlPKCtlQ|b7ewqZF?XhhX)*hMqVJJ|A_ zEOX7yCMXXx^4JTJ*J*GXN7XFYd4+vj;?|st5t(); zx!KvW7NOWAhpT6nP+a2f`sVpCL4plsdcq3;suw0z9ry{ft$bIQd9qR6d=)%uZ?IhOJG0+lm&Q@%S0oe~o|0Vo zr*KeR;6qwbvnn!m>asyX2qJ4n(V03l&tkm*Cf^4P9 z*st4lsa093O=*RP!wr(ojU~DJuDBWOOp?v^Iq$w33*qiNnr1!=;w8575!nwOzC>1j z#eGls7|o4QQ#H462$@&PMRcyuR9~V*48i#FHa%z9%7>wM8x5JO<;(KLiD%QJt zMouN|1@RuQm5j&;{?Q+Zo;HR0cU62QTvDP1fvMnky?D_X_0g_`pw!;{mg1%gbE zmG3+7pbgixDV(!bq@wwnMJp=On@0NlWfZ7T8BfcqG1|^!0RGtfn&C{&6WtZI4xt#p}gd=A<#J7vG5Y7M@3|{k6n($qDRM2RTkTeyGoeoJPb>n zUltwy5$U>7ifB)4vwL2mH5v;M8bWKBl{lD>N@y`l>7{CR6J(Rf++F@)yOax7pw0QT$l1Yrzv&x#H$z1o3Fx`3;ACv9HwEpu+5yf-V%vRpCn zEou<=Hd}u2i5GrJD&ZrX8zZ5%ejqDggZuN~TJ!$#{I`xtM(7|>sSmcD_kE1nIk$T+ zk+lX8z@5(xrs%j(K_?sJr2J%pDY|l+yDImI-F=_>R^GZqpBhR{ z6-`!AmYI)oF|uKNB9F6imIg@U#BoNw7Le^Oh-EZgzkZ!pp%`e7%3+7dQeP0IcgIVJ zBPI)k@1knSbf*?Xny##I7?qbxf?bkklz_<&mtvnUgRXt&P;gcpYRGY#nR>hv<^ZvI zpylL$*GHI$-e^&vd!=&UZp_Z>t-|1<#1<78ajfIUF{`f}iA0-)>09jDC#nW(quCK= zLf&2rpHdo{Ap9wXmBU+|4Sz6UWL`e#Iw!7cipLYTm{$Xmo)0yjbx(9iFQc6#=cJVc z!#`>`%dQ{%LRKxtgqhf5Nhh6ZJgV1bFF}1~ow}!Zy&9kT_S7v)eT%VOeeTE#<~D z2EI6{vH+612m5#@EA{XUw8=tVh0EsQ!-rRGqzBbO;}N!r=YiF6CuXHJZOY+~bzd(m zFF!jSQVyN?E%4M5`JV3m+vz`$?vo5#BWqgHepa1`8G#o5VDRN;eY{ukrKGSnucebx zNd4=QAsIxW0blLB5y(BiMKCH`2q&t?&yKJ|@3Mo{-GmQxs)sCW{_-vw(~z=^??&O% z_k@+^+Jeg=MKR?>k1|b*1p9!Tu@@6gX>ir^N%h9;NO-2gqg+C3CRf?xX?MHI3F|}g zC}eGrIg~b+f>F(BJC4#5>Vz-XB}&8kC3{PXAARQF&-v8SAmsR4_2g(bv}@H-K$WUm zY{CH@a}tFJRevW*cs_b_UjU1&W5nel_{iNiu8lhg8J!R8R%Y9RKEa>=%F)-TKSgoq zbHMxMn5(g|??28zHcsE7n~?f;i$B|o7`G(48Ktq?2QQ;EeQ+gF_OkQROSEDh8|3?n zm^g2NsAJT$6n~wv1{%I*e*oga4$2xOuj8%RS&W{mpD>U-yhJHl{QL0SW zp86EFHFCiBAXU6SLT{*u=u^eDP|SrYLV~VA+J`Jzw2X_}8*?_CAQPCWS;0M4$Yx;EPfYnNwl2Ihbww9I>_Dr<2 zVe;%tx{{lvOAa6VihNoEt4MqPfOi^^284WB;^rs2dN58^W@KBuf4W7^8G(>QnDrO( z(s@As!M2Sx=J3j>)P*)pFrCX5-_{GI%tj=@svkV$Ggl3Z6wWnGnyT22@+&!~~M_Nsw7_LX`eXDii#d{(7aq2j2LsN5x$jh zYZ3_**ng_>In3A?tu?-FQy@7 z(Mae^G4%1-8>;oD(sa$=-iI4!XRM{QdRByW(mPT^0@W~Q(|)_1W0{$Sv1{4EO5W!^ zF*WaiO8y+-%;v10+EV1J2hUIh%j!he;^T*RDIZgtCbB{hh_&)a&pTdC0#9T|Vuv-W zJji{^w4^m&dtbT`xSVn>4DSsqye&5^9cD`k;%$vZ45Nlv$74RYxois&j5H0j@Lwnr zL7s#sxQuVbJEGwgy~FAxmAuYKbf0 zPM^kR=KU#xoytzQT|m@ZQtn|m54+>CStu>w%C0*y*W1T>6yPyezrpaN{&86ER;3xS zX}=AgG`*sW7~8X(^xsQW7az$M|C(dFnqux8oL8YZs!iFxQC_KUzgjuXPx&(5-%0f#b^D4syd$e?GO^4^bay29C4Q( zrNfvDnl6nhaz3m@R=Ytd!CmdEc&8BBtG^*{K?`<8nwb+ZD2>nB&US@z505hzVKonX zdEnU*0fA@@J2PaLjrYDpN0oQ|`ozwRRq$bcqm0~sM)4gVp0kSmW_3-w1`jftT?VNc zE`KGIx;Y!Px@D*M%Lml+Fmu|&pHqYSWSD#Y;Ts8X`$;o#tYhZssK$&-sG3^L5UhJG z^ZLX3_v*j3(_n3z^$m*WpN_JoZpVc6wZV6^;%a+rGPg7N9GC6A zJn%gjmp4#NZwbB`D|9sSA8~^w{P7kOBcdfT5OFnJ593vjy4q9M?f?_~9`Mat2a1LC z-zCQ&v^eUw_yFii-014vUc~*|lC$Gse4!)Zs|UO3t8*saUb|VB5NN(n+Jw8~Z!PR6 z;*biR>UnNGUrlgFKS^Q%191hb)wWoPt)RS>6^368!!**YLyc(paK$`=xe_6;qa8!b zZ8uX?<6}c^^lJtfGI!R-@XH;6vx0BN9`D#k_#3EFm@ytibKF>d^nFU4-2rkWW}~nj z0QeJmN&(t7%=|uHXnfdjrrIe+fl8{oypX-!Y z%GR^|{mPV` Date: Tue, 17 Nov 2020 14:00:18 -0500 Subject: [PATCH 004/231] Initial commit of Draft ZIP. --- zip-liberated-payments.rst | 410 +++++++++++++++++++++++++++++++++++++ 1 file changed, 410 insertions(+) create mode 100644 zip-liberated-payments.rst diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst new file mode 100644 index 000000000..da6ec84ce --- /dev/null +++ b/zip-liberated-payments.rst @@ -0,0 +1,410 @@ +:: + + ZIP: XXXX + Title: URI-Encapsulated Payments + Owners: Ian Miers + Eran Tromer + Credits: Sean Bowe + Deirdre Connolly + Kevin Gorham + Jack Grigg + Daira Hopwood + Linda Naeun Lee + George Tankersley + Henry de Valence + Status: Draft + Category: Standards Track + Created: 2019-07-17 + License: MIT + +Terminology +=========== + +The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to be +interpreted as described in RFC 2119. [#RFC2119]_ + +Zcash protocol terms are as defined in the Zcash Protocol Specification. [#protocol]_ + +.. _applink: +By “applink” we mean a platform mechanism for triggering local applications using HTTP links, such as `App Links`_ on Android, `Universal Links`_ on iOS, or `App URI handlers`_ on Windows. + + +Abstract +======== + +This proposal defines a mechanism for sending a Zcash payment encapsulated in a URI string. This enables sending Zcash funds over any secure channel, such as via a messaging app, even if the recipient does not yet have Zcash software installed and does not have a Zcash payment address. This is implemented by having the URI convey the secret spending key of an ephemeral Zcash “wallet address”, to which the funds have been transferred. Anyone who learns the URI can accept this payment, by a “finalization” process which uses the key given in the URI to transfer the encapsulated funds to their own wallet. After the payment is finalized, via a suitable on-chain transaction by the recipient, it becomes irrevocable. + +The proposal specifies the syntax and semantics of Payment-Encapsulating URIs, the workflow of generating and handling them, and requisite infrastructure. + +At its core, a URI encapsulated payment communicates the existence of a transaction (specifically a note committing to an amount of funds) to a receiving client. The URI encode contains the amount of the payment and a key used to derive all necessary randomness for note construction including the address and secret key needed to spend it. + +Usage Story +----------- + +Alice wants to send a 1.23 ZEC payment to Bob. She generates a fresh ephemeral spending key and sends 1.2301 ZEC to it. This transaction is submitted to the network and mined. Meanwhile, Alice constructs a URI containing the ephemeral spending key and appropriate metadata, and sends it to Bob over an end-to-end encrypted channel such as Signal or WhatsApp. + +Bob receives a message, which looks as follows: + + This message contains a Zcash payment. + Click the following link to view it and receive the funds using your Zcash wallet app: + +https://pay.withzcash.com:65536/v1#amount=1.23&desc=Payment+for+foo&key=... + If you do not yet have a Zcash wallet app, see: https://z.cash/wallets + +Bob clicks the link. His Zcash mobile wallet app (which is already installed and has configured itself as a handler for URLs of that form) shows up, and tells Bob that a payment of 1.23 ZEC awaits him. The wallet app confirms the existence of the pertinent transactions on the blockchain, and then offers to finalize the payment. Bob clicks the “Finalize” button, and his wallet app generates a transaction moving the money to his own address (using the extra 0.0001 ZEC he has received to pay the transaction fee). When this transaction is confirmed on-chain, Alice’s and Bob’s wallets both indicate that the payment is finalized, and thus Bob can send the funds to other parties. + + +Motivation +========== + +This proposal enables sending of funds without exposing users to the notion of payment addresses and their secure distribution. Instead, funds can be sent using any pre-existing communication channel, by a single message sent unidirectionally from the sender to the recipient. This message conveys all the information needed to obtain control of the funds, compactly expressed as a textual URI. + +Consequently, all functionality related to contact discovery and secure-channel establishment can be delegated to the message app(s) with which the user is already familiar, and in which the user has already established communication channels to many of their contacts. + +Moreover, funds can be sent to users who have not yet installed wallet software and who do not yet have a payment address. The recipient can collect the funds at any later time by installing suitable software. + +The proposal is complementary to ZIP 321 [#zip321]_, which will standardize *Payment Request URIs* using which the payment *recipient* can convey their persistent payment address to the *sender*, for subsequent fund transfers (to be done using the normal on-chain mechanism rather than the encapsulated payments described in the current proposal). + + +Requirements +============ + +This proposal’s specification of Payment-Encapsulating URIs, and the intended protocols for using them, is meant to fulfill the following requirements: + +* The protocol must not require the sender of a payment to stay online until the recipient receives the URI (let alone finalizes the payment). + +* The URIs should be short, for convenient rendering and to maximize compatibility with length-limited messaging platforms. + +* It must not be feasible for someone who has not seen the URI (or has compromised a party who has) to collect the funds. + +* The URIs and protocol should minimize the likelihood of inadvertent misuse, and in particular the risks discussed in “Security Considerations” below. + +* The protocol must not leak any information (sender identity, recipient identity, amount, description) to third parties, other than inevitable metadata about the existence of a transaction, the inevitable network communication around sending/receipts of transactions, whatever leakage is induced by the communication channel used to transmit the URI, and whatever is voluntarily shared by the parties. + +* The URIs should allow for future modifications and expansion of the format, without risk of ambiguous parsing. + +* The on-chain footprint of payments that use this mechanism should be indistinguishable from normal fully-shielded transactions (except, possibly, for the statistics of the number of shielded inputs and outputs). + +* Don’t lose funds, even if wallets crash. + +Non-requirements +================ + +* It is outside the scope of this proposal to establish a secure communication channel for transmission of Payment-Encapsulating URIs, or to protect the parties’ devices from security compromise. + +* Finalizing the payment may involve significant wait times, on the scale of minutes, as the requisite on-chain transactions are generated, mined and confirmed. This proposal does not try to solve this (though it does try to avoid imposing significant additional delays, and it does address how the intermediate state is conveyed to the user). + + +Specification +============= + +A Payment-Encapsulating URI represents the capability to claim the Zcash funds from specific on-chain transactions, as long as they’re unspent. See `Usage Story`_ for an example. + +Syntax +------ + +A Payment-Encapsulating URI is a Universal Resource Locator (URL), as defined in RFC 3986 [#RFC3986]_, of the following form. + +Scheme: ``https`` + +Host: ``pay.withzcash.com`` + +Port: ``65536`` (this is intentionally not a valid TCP/IP port number) + +Path: ``payment/v1`` + +Fragment parameters: these attribute-value pairs, in this order, separated by ``&``, and with all values percent-encoded where necessary: + +* ``amount=...`` where the attribute is a decimal number representing the amount of ZEC included in the payment. MUST be present. + If a decimal fraction is present then a period (.) MUST be used as the separating character to separate the whole number from the decimal fraction, and both the whole number and the decimal fraction MUST be nonempty. No other separators (such as commas for grouping or thousands) are permitted. Leading zeros in the whole number or trailing zeros in the decimal fraction are ignored. There MUST NOT be more than 8 digits in the decimal fraction. +* ``desc=...`` where the attribute is a human-readable string associated with the payment. MAY be present. + If present, it MUST be encoded as “textual data consisting of characters from the Universal Character Set” as specified in RFC 3986 section 2.5. +* ``key=``is a 128 bit random number encoded with Bech32 as specified in Section 5.6.9 of the Zcash Protocol Specification [#protocol]_). MUST be present. + + +Semantics +--------- + +The values of ``key’’ and ``amount`` deterministically imply a unique *payment note* corresponding to this URI, which is a Zcash Sapling note that carries the given amount and is spendable by a Sapling spending key derived from ``key`. The derivation of this note is done by the following procedure: +*DerivePaymentNote(key,amount)*: + Derive the master extended spending key *(ask, nsk, ovk, dk, c)* according to ZIP 32 [#zip32]_ `Sapling master key generation`_, with *key* as *sk_m*. + Fix the diversified d = XXX. + Derive *rcm = PRF^expand(sk_m || [0x11])* as specified in [#protocol]_ Section 5.4.2. + Derive *pk_d* from *ask* and *nsk* as specified in [#protocol]_ Section 4.2.2. + Define the corresponding *payment note* as *n = (d, pk_d, amount, rcm)* (see [#protocol]_ Section 3.2). + Define the corresponding a *payment note commitment* as *cm = NoteCommit^Sapling_rcm (repr_J (dk),repr_J (pk_d ), value)* as specified in [#protocol]_ Section 5.4.7.2. + +Construct a shielded zcash transaction containing that note as an output. + +The payment note SHOULD be unspent at the time it is intended to be received by the recipient. + +Clients MAY generate and send the URI before the transaction is built, sent, or confirmed. + +The ``amount`` parameter MUST match the total amount of ZEC in the payment note plus the standard transaction fee for fully-shielded transactions (currently 0.0001 ZEC). + +There MUST NOT exist any other notes on the blockchain, beyond the payment note derived from the Payment URI, that are addressed to a payment address derived from ``key``. Such notes MAY be generated within an implementation (e.g., as speculative pre-generation) but MUST NOT be broadcast for mining. + +The ``desc`` parameter MAY convey a human-readable description of the payment, entered manually by the user or generated by the application in any reasonable manner. + +The encrypted memo fields in the output description containing the payment note commitment MUST be either empty (all-zero), or identical to the ``desc`` parameter (padded with zeros). + +When conveying payment to users, the sender’s and recipient’s wallet software MAY convey the description encoded in the ``desc`` parameter. + +The recipient’s wallet software SHOULD convey to the user that the ``amount`` and ``desc`` values are merely a claim made by the party who sent the URI, and may be tentative, inaccurate or malicious. + +In particular, the recipient’s wallet software SHOULD convey to the user that the amount of ZEC they can successfully transfer to their wallet may be different than that given by the ``amount`` parameter, and may change (possibly to zero), until the finalization process has been completed. + + +Centralized Deployment +---------------------- + +The owner of the ``withzcash.com`` domain name MUST NOT create a DNS record for the ``pay.withzcash.com`` domain name, nor a TLS certificate for it. All feasible means SHOULD be taken to ensure this, and to prevent unintended transfer of ownership or control over the ``withzcash.com`` domain name. (See `Rationale for URI Format`_ and `Security Considerations`_ below for discussion.) + +Applink_ mechanisms let domain name owners provide a whitelist, specifying which apps are authorized to handle URLs with that domain name. This is implemented by serving suitable files at well-known paths on the web server of that domain or, in the case of a subdomain, its parent domain. Thus, the owner of the ``withzcash.com`` domain effectively controls the whitelist of apps that may be launched by users’ platform to handle URI-Encapsulated Payments (see `Security Implications`_). This whitelist should protect users from installing rogue apps that intercept incoming payments. Thus, the domain owner MUST do the following: +* Maintain such a whitelist and serve it as needed for the applink_ mechanisms of major platforms. +* Publish a policy for inclusion of apps in this whitelist. +* Use all feasible means to whitelist only apps that comply with the published policy. +* Publish the whitelist’s content in human-readable form. +* Provide clear and effective means for rapid removal of apps from the whitelist when required as security response. +* Use all feasible means to protect the whitelist’s integrity (in particular, this includes protecting the web server that serves the whitelist, the domain’s TLS certificate, and the means by which the whitelist is modified). +* Use effective means for keeping a precise, irrevocable and public history of the whitelist (e.g., using a timestamped Git repository, or an accountability mechanism akin to Certificate Transparency). + +They also SHOULD: +* Strive for the whitelist to include all apps that would not place the user at any greater security risk than reputable state-of-the-art wallet apps. + + +Testing +------- + +For testing purposes, all of the above specification is duplicated for the Zcash testnet network, substituting ``TAZ`` (Zcash testnet coins) for ``ZEC`` and ``testzcash.com`` for ``withzcash.com``. + +A separate “testnet whitelist” MUST be maintained by the owner of the ``testzcash.com`` domain name, with a separate policy that SHOULD allow any legitimate third-party developer to add their work-in-progress wallet for testing purposes. Integrity and availability MAY be looser. + +Wallets apps MAY support just one type of payments (ZEC or TAZ), and if they support both then they MUST keep separate accounting and must clearly distinguish the type when payments or balances are conveyed to users. + + +Rationale for URI Format +------------------------ + +The URI format ``https://pay.withzcash.com:65536/v1#...`` was chosen to allow automatic triggering of wallet software on mobile devices, using the platform’s applink_ mechanism, while minimizing the risk of payment information being intercepted by third parties. The latter is prevented by a defense in depth, where any of the following suffices to prevent the payment information from being exposed over the network: +* The ``pay.withzcash.com`` domain should not resolve. +* A valid TLS certificate for ``pay.withzcash.com`` should not exist.. +* The port number ``65536`` is not valid for the TCPv4, TCPv6 or UDP protocols. Empirically, the common behavior in browsers and messaging apps, when following HTTPS links with port number port number 65536, is to render an empty or `about:blank` page rather than a DNS error; a network fetch is not triggered. (This may change if a network proxy protocol is used, but SOCKS5 also cannot represent port 65536.) +* The contents of the fragment identifier are specified by HTTP as being resolved locally, rather than sent over the network (but see the caveat about active JavaScript attacks below). + +The downside is that if the user follows the link prior to installing a suitable wallet app, they get a weird-looking DNS error or a blank page. Also, the URL looks weird due to the port number. + +Several alternatives were considered, but found to have inferior usability and/or security ramifications: + +1. ``https://pay.withzcash.com/v1#...``: similar to above, but without the port number, and backed by a DNS record, TLS certificate and web server for ``pay.withzcash.com`` that serves an informative HTML page (e.g., “Please install a wallet to receive this payment”). This still allows handling by wallet apps using an applink_ mechanism, and provides a friendlier fallback in case the user follows the link prior to installing a suitable app. However, it creates a security risk. If the web server serving that web page is compromised, or impersonated using an DNS+TLS attack, then the attacker can capture they payment parameters and steal the funds. (Note that the sensitive information is in the fragment following the ``#``, which is not sent in an HTTP GET request; but the malicious server can serve JavaScript code which retrieves the fragment.) + +2. ``zcash-data:payment/v1?amount=1.23&desc=Payment+for+foo&key=...&seedcmu=...``: a custom URI scheme, such as ``zcash-data``. This still allows for triggering application action (e.g., using Mobile Deep Links). However, on most platorms, *any* app installed on the device is able to register to handle links from (almost) any custom URI scheme. If the request is received by a rogue party, then the funds could be stolen. Even if received by an honest operator, funds could be stolen if they are compromised. Also, custom URI schemes are not linkified when displayed in some messaging apps. + + Note the use of the ``zcash-data`` URI scheme, rather than the more elegant ``zcash``, because URIs of the form ``zcash:address?...`` are already used to specify Zcash addresses and payment requests in ZIP 321 [#zip321]_, by analogy to the ``bitcoin`` URIs of BIP 21. An alternative is to use ``zcash:v1/payment?...``; legacy software may parse this as a payment request to the address ``v1``, which is invalid. Another alternative is to use ``zcash-payment:v1?...``, which is appealing in terms of length and readability, but may be gratuitous pollution of the URI scheme namespace. + +Another option, which can be added to any of the above, is to add a confirmation code outside the URI that needs to be manually entered by the user into the wallet app, so that merely intercepting the URI link would not suffice. This does not seem to significantly reduce risk in the scenarios considered, and so deemed to not justify the reduced usability. + + +Lifecycle Specification +======================= + +The lifecycle of a Payment-Encapsulating URI consists of several stages, which in the usual case culminate in the funds being irrevocably deposited into the recipient’s personal wallet irrevocably: + +Generating the notes and URI +---------------------------- +The sender’s Zcash wallet app creates an ephemeral spending key, sends ZEC funds to the payment addressed derived from that key, and creates a Payment-Encapsulating URI that contains this ephemeral spending key and the newly-generated note commitments. + +URI Transmission +---------------- +The sender conveys the Payment-Encapsulating URI to the intended recipient, over some secure channel (e.g., an end-to-end encrypted messaging platform such as Signal, WhatsApp or Magic Wormhole; or a QR code scanned in person). + +If transmitted via a human-readable medium, such as a messaging app, the Payment-Encapsulating URI MAY be accompanied by a contextual explanation that the URI encapsulates a payment, and a suggested action by the recipient to complete the process (see Usage Story above for an example). + +When sent via a human-readable medium that consists of discrete messages, the message that contains the URI SHOULD NOT contain any payment-specific or manually-entered information outside the URI itself, since this information may not be visible to the recipient (see “Message Rendering” below). + +From this point, and until finalization or cancellation (see below), from the sender’s perspective the payment is “in progress”; it SHOULD be conveyed as such to the sender; and MUST NOT be conveyed as “finalized” or other phrasing that conveys successful completion. + +Message Rendering +----------------- +The recipient’s device renders the Payment-Encapsulating URI, or an indication of its arrival, along with the aforementioned contextual explanation (if any). The user has the option of “opening” the URI (i.e., by clicking it), which results in the device opening a Zcash wallet app, using the local platforms app link mechanism. + +A messaging app MAY recognize Payment-Encapsulating URIs, and render them in a way that conveys their nature more clearly than raw URI strings. If the messaging medium consists of discrete messages, and a message contains one or more Payment-Encapsulating URIs, then the messaging app MAY assume that all other content in that message is automatically generated and contains no payment-specific or manually-generated information, and thus may be discarded during rendering. + + +Payment Rendering and Blockchain Lookup +--------------------------------------- +The recipient’s Zcash wallet app SHOULD present the payment amount and MAY present the description, as conveyed in the URI, along with an indication of the tentative nature of this information. + +In parallel, the wallet app SHOULD retrieve the relevant transactions from the Zcash blockchain, by looking up the transactionnote commitments given by the ``seedcmu`` parameter (this MAY use an efficient index, perhaps assisted by a server), and check whether: +* such transactions are indeed present on the blockchain +* the notes are unspent +* the notes can be spent using an ephemeral spending keys given by the ``key`` parameter. + +The wallet conveys to the user one of the following states: + +* *Ready-to-finalize*: The tests all verify, and the payment is ready to be finalized. The wallet SHOULD present the user with an option to finalize the payment (e.g., a “Finalize” button). +* *Invalid*: The tests fail irreversibly (e.g., some of the notes are already spent, or the amounts to not add up). The wallet MAY convey the reason to the user, but in any case MUST convey that the funds cannot be received. +* *Pending*: The tests fail in a way that may be remedied in the future, namely, some of the notes are not yet present on the blockchain (and no other tests are violated). + +Within the *Pending* state, the wallet app MAY also consider “0 confirmations” transactions (i.e., transactions that have been broadcast on the node network but are neither mined nor expired), and convey their existence to the user. These do not suffice for entering the *Ready-to-finalize* state (since unmined notes cannot be immediately spent.) + +The aforementioned conditions may change over time (e.g., the transactions may be spent by someone else in the interim), so the status SHOULD be updated periodically. + +Finalization +------------ +When the recipient chooses to finalize the payment, the wallet app generates transactions that spends the aforementioned notes (using the ephemeral spending key) and send these Zcash funds to the user’s own persistent payment address. These transactions carry the default expiry time (currently 100 blocks). + +The recipient’s wallet app SHOULD convey the payment status as “Finalizing…” starting at the time that the uses initiates the finalization process. It MAY in addition convey the specific action done or event waited. + +The sender’s wallet SHOULD convey the payment status as “Finalizing…” as soon as it detects that relevant transactions have been broadcast on the peer-to-peer network, or mined to the blockchain. + +Once these transactions are confirmed (to an extent considered satisfactory by the local wallet app; currently 10 confirmations is common practice), their status SHOULD be conveyed as “Finalized”, by both the sender’s wallet app and the recipient’s wallet app. Both wallets MUST NOT convey the payment as “finalized”, or other phrasing that conveys irrevocability, until this point. + +If these transactions expire unmined, or are otherwise rendered irrevocably invalidated (e.g., by a rollback), then both wallets’ status SHOULD convey this, and the recipient’s wallet SHOULD revert to the “Payment Rendering and Blockchain Lookup” stage above. + +Payment Cancellation +-------------------- +At any time prior to the payment being finalized, the sender is capable of cancelling the payment, by themselves finalizing the payment into their own wallet (thereby “clawing back” the funds). If the wallet has not yet sent, for inclusion in the blockchain, any of the transactions associated with the ephemeral spending key, then cancellation can also be done by discarding these transactions or aborting their generation. The sender’s wallet app SHOULD offer this feature, and in this case, MUST appropriately handle the race condition where the recipient initiated finalization concurrently. + + +Status View +------------ + +Wallet apps SHOULD let the user view the status of all payments they have generated, as well as all inbound payment (i.e., Payment-Encapsulating URIs that have been sent to the app, e.g., by invocation from messaging apps). The status includes the available metadata, and the payment’s current state. When pertinent, the wallet app SHOULD offer the ability to finalize any *Pending* inbound payment, and MAY offer the ability to cancel any outbound payment. + +Wallet apps SHOULD actively alert the user (e.g., via status notifications) if a payment that they sent has not been finalized within a reasonable time period (e.g., 1 week), and offer to cancel the payment. + + +Security Considerations +======================= + +* Anyone who intercepts the Payment-Encapsulating URIs may steal the encapsulated funds. Therefore, Payment-Encapsulating URIs should be sent over a secure channel, and should be kept secret from anyone but the intended recipient. + The Payment-Encapsulating URI is like a magic spell that will teleport the money to the first person that clicks it and then does "finalize". + +* Payment-Encapsulating URIs may be captured by malicious local apps on the sender or receiver’s platform, e.g., by screen capturing or clipboard eavesdropping. Wallet apps should use the platform’s interaction and communication facilities in a way that minimizes these risks (e.g., use the “Share” API rather than a clipboard that is visible to all apps). + +* Likewise, if the URI is transferred by presenting and optically scanning a QR code, anyone who observes this QR code may be able to finalize the payment and thus take ownership of the funds before the intended recipient. For example, an attacker may use a telephoto lens aimed at a point-of-sale terminal to steal QR-encoded payments sent to that terminal. + +* Users may have casually-established communication channels (e.g., they have entered the phone number of a new contact without bothering to double-check it), but may later mistakenly consider these to be adequately-authenticated secure channels for the purpose of sending Payment-Encapsulating URI. Wallet apps should mitigate this where feasible, e.g., by indicating that the chosen messaging channel is previously-unused and thus should be more carefully checked. + +* Users may incorrectly believe that the payment has been irrevocably received even though they have not invoked the finalization procedure, or even though the finalization procedure has failed. Wallet software should correctly convey the status and set expectations, as discussed above. + +* Payment recipients may not notice the incoming payment notification and act on it (i.e., invoke finalization) in a timely fashion. By the time they see it, the payment may have been cancelled by the sender. + +* Users may not understand that Payment-Encapsulating URIs are for one-time use, and attempt to use the same URI for multiple people or payments, resulting in race conditions on who receives the funds. + +* Users may confuse Payment-Encapsulating URIs (as specified in the current ZIP) with Payment Request URIs of the form ``zcash:payment-address?amount=...``. (The latter are a de facto standard, and will be specified in the forthcoming ZIP 321 [#zip321issue]_). Normally these serve different workflows, and work in opposing directions (send vs. receive of funds), and thus ought to not arise in ambiguous context. Wallet apps should take care to not create or send a Payment-Encapsulating URI (which is for *sending* funds) in a context where the user may be intending to *receive* funds. + +* Users may attempt to use a Payment-Encapsulating URI as a “cold wallet”, e.g., by writing the URI on paper and putting it in a safe. This is dangerous. The spending key is known to the sending wallet at the time when the URI is produced, and possibly also at other times (e.g., if there are storage remnants, or if deterministic derivation is used; see “Ephemeral key derivation” below). Thus, an adversary who compromises the sending wallet may drain the cold wallet. + +* The act and timing of finalizing a payment is visible to the sender, which may be a privacy leak. Likewise, if the on-chain transactions are sent in advance, their timing can be linked to the later payment, which may be a privacy leak. + +* The payment amount is readily visible to anyone who observes the Payment-Encapsulating URI, even in retrospect after payment has already been finalized (e.g., if their device or chat log backups are later compromised). This may be a privacy concern, and in particular may put recipients of large payments at risk of undesired attention. + +* Users attempting to follow Payment-Encapsulating URIs as a regular HTTPS hyperlink may inadvertently leak the payment information to a remote attacker, if all layers of defense listed in `Rationale for URI Format`_ are somehow breached. + +* The owner of the ``withzcash.com`` domain effectively controls the whitelist of apps that may be launched by users’ platform to handle URI-Encapsulated Payments using the applink_ mechanism. If the whitelist is too *permissive* and includes a malicious or vulnerable app, and a user installs that app (which itself may be subject to the platform vendor’s app review mechanism), then the user is placed at risk of having their payments intercepted by an attacker. Conversely, if the whitelist is too *restrictive*, or altogether unavailable, then users would not be able to trigger desirable wallet apps by simply following links, and would need to instead ”share” the message containing the URI into their wallet app (note that, as discussed above, clipboard copy-and-paste is insecure). + +* Usage of Payment-Encapsulating URIs may train users to, generally, click on other types of URI/URL links sent in other messaging contexts. Malicious links sent via unauthenticated messaging channels (e.g., emails and SMS texts) are a common attack vector, used for exploiting vulnerabilities in the apps triggered to handle these links. Even though the fault for vulnerabilities lies with those other apps, and even though this ZIP uses deep link URIs in the way intended, there are none the less these negative externalities to encouraging such use. + +* + +Design Decisions and Rationale +============================== + +See `Rationale for URI Format`_ above. Moreover: + +1. The metadata (amount and description) is provided within the URI. An alternative would be to encode the description in the encrypted memo fields of the associated shielded transactions, and compute the amount from those transactions. However, in that case the metadata would not be available for presentation to the user until the transactions have been retrieved from the blockchain. + +2. We support multiple spending keys and multiple notes in one URI, because these payments may be speculatively generated and mined before the payment amount is determined (to allow payments with no latency). For example, the sending wallets may pre-generate transactions for powers-of-2 amounts, and then include only a subset of them in the URI, totalling to the desired amount. + +3. We do not include the sender or receiver’s identity in the URI, because the sending wallet many not know the name of who it is sending to (or even from). Moreover there is the risk that fraudulent sender/recipient information could be used. If necessitated by circumstances (e.g., the `Travel Rule`_), claimed sender and recipient identity can be included in ``desc`` parameter. + + +Open Questions +============== + +Ephemeral Key Derivation +------------------------ +Specify how the ephemeral spending keys can be derived from a seed a la ZIP 32, so that if a wallet is recovered from backup, sent-but-unfinalized payments can be reclaimed. +This requires a deterministic key-derivation mechanism, and means to find payments that can be recovered given just the wallet seed and the blockchain ledger. + +Sketch: +Use a ZIP 32 derivation pathway to obtain a child extended spending key from path m_Sapling/zip_number'/coin_type'/payment_index' +Implementations need to remember which payment_index values they have used (in range 0..2^31), and not reuse them. +Convert the child ExtSK into a “URI seed” (e.g. hash the entire key), and provide that to the recipient. +The intent of this step is to shorten the value that goes into the URI, while keeping it linked to the sender’s backed-up wallet seed for recovery purposes. +The length of this URI seed can be “short” (e.g. 128 bits instead of 256) given that funds are not intended to be stored long-term underneath this secret (one of the sender or recipient is expected to scrape funds back into a long-term address) +Use PRF^expand on the URI seed with to-be-defined domain separation to obtain 64 bytes. Split this into two 32-byte values. +First 32-byte value is sk_m; derive spending key from this as in the spec / ZIP 32. +Second 32-byte value is the root for deriving randomness following ZIP 212. + + +URI Usability +------------- +The URI could be changed in several ways due to usability concerns: + +1. It may be desirable to prevent the ``amount`` and ``desc`` parameters from being human readable. This is to discourage people from just looking at the URI, seeing the numbers and text, and mistakenly thinking this is already a confirmation of successful receipt (without going through the finalization process). + +2. Perhaps the URI should be contain the phrase “password” early on (e.g., ``zcash-data:/payment/v1/password=``, as a cue that this string must be kept secret. (Note that technically nothing here is a password in the usual sense of the term.) + +3. Perhaps we should actually use BIP 39 words as an actual password. So you could memorize it or read it over the phone. The BIP 39 words can be embedded in the URI itself (which is highly unusual): + ``zcash-data:payment/v1/password=witch+collapse+practice+feed+shame+open+despair+creek+road+again+ice+least`` + or + ``zcash-data:payment/v1/password=WitchCollapsePracticeFeedShameOpenDespairCreekRoadAgainIceLeast`` + This provides an additional cue that the URI contains a sensitive password (for users who are accustomed to BIP 39 style word lists; to others the Base 64 encoding may be more evocative of a password). Moreover, users may discover the fact that they can manually send these words to recipients, in writing or verbally, as a way to send money without a textual messaging service. + Alternatively, the BIP 39 words can be used as an alternative syntax for the encapsulation, without the confusing-to-humans URI syntax (but generating this alternative syntax this may complicate the UI). + + +Identifying Notes +----------------- +The recipient’s wallet needs to identify the notes related to the payment (and the on-chain transactions that contain them), in order to verify their validity and then (during the finalization process) spend them. + +In the above description, we explicitly list the notes involved in the payment (which are easily mapped to the transactions containing them, using a suitable index). This results in long URIs when multiple notes are involved (e.g., when using the aforementioned “powers-of-2 amounts” technique). + +Instead, we can have the nodes be implicitly identified by the spending key (or similar) included in the URI. This can make URI shorter, thus less scary and less likely to run into length limits (consider SMS). The following alternatives are feasible: + +0. Explicitly list the note commitments within the URI. + +1. Include only the spending key(s) in the URI, and have the recipient scan the blockchain using the existing mechanism (trial decryption of the encrypted memo field). This is very slow, and risks denial-of-service attacks. Would be faster in the nominal case if the scanning is done backwards (newest block first), or if told by the sender when the transactions were mined; but scanning the whole chain for nonexistent transactions (perhaps induced by a DoS) would still take very long. + +2. Derive a tag from a seed included in the URI, and put this tag within the encrypted memo field of the output descriptors in the associated transactions. Put the tag plaintext within the space reserved for the memo field ciphertext (breaking the AEAD abstraction). The recipient’s wallet (or the service assisting it) would maintain an index of such tags, and efficiently look up the tags derived from the URI. + The tags are publicly-visible and thus may leak information on the payment amount (e.g., when using the powers-of-2 pre-generation technique). + +3. Similarly to the above, but place the tag in an additional zero-value output descriptor added to each pertinent transaction. The recipient can recompute this note commitment and use that as the identifier, to be looked up in an index in order to locate the transaction. + Here too, the tags are publicly-visible and thus may leak information on the payment amount (e.g., when using the powers-of-2 pre-generation technique). + +4. Have the URI include a seed and the amount of the (single) output note. Let the seed determine not only the spending key, but also all randomness involved in the generation of the note. Thus, the recipient can deterministically derive the note commitment from the seed and amount, and look it up to find the relevant transaction. This requires the recipient (or the server assisting them) to maintain an index mapping note commitments (of output descriptors that are the first in their transaction) to the transaction that contains them. Additional notes can be included in the same transaction. + + + + +Other Questions +--------------- +Should senders delay admitting a generated transaction by a random amount to prevent traffic analysis (i.e., so the messaging service operator cannot correlate messages with on-chain transactions)? + +Consider the behavior in case a chain reorgs invalidates a sent payment. Should we specify a Merkle root or block hash to help detect this reason for payment failure? Or have some servers that maintain a cache of payments that were invalidated by reorgs? + + +References +========== + +.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels `_ +.. [#RFC3986] `Uniform Resource Identifier (URI): Generic Syntax `_ +.. [#protocol] `Zcash Protocol Specification, Version 2019.0.3 or later [Overwinter+Sapling] `_ +.. [#zip321] `ZIP 321: Payment Request URIs + `_ + +.. _Sapling master key generation: https://zips.z.cash/zip-0032#sapling-master-key-generation +.. _App Links: https://developer.android.com/training/app-links +.. _Universal Links: _https://developer.apple.com/documentation/uikit/inter-process_communication/allowing_apps_and_websites_to_link_to_your_content +.. _App URI handlers: https://docs.microsoft.com/en-us/windows/uwp/launch-resume/web-to-app-linking +.. _Travel Rule: https://www.fincen.gov/sites/default/files/2019-05/FinCEN%20Guidance%20CVC%20FINAL%20508.pdf + + +Publication Blockers +==================== +* Register all domains mentioned in this draft (``withzcash.com``, ``testzcash.com``). +* Clean up semantics. + + From 08d34f6b8dd033f0103e69a933282f7e04c4d85a Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 08:53:58 -0500 Subject: [PATCH 005/231] Typo correction. Co-authored-by: Deirdre Connolly --- zip-liberated-payments.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index da6ec84ce..c5eb2bf2b 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -36,7 +36,7 @@ This proposal defines a mechanism for sending a Zcash payment encapsulated in a The proposal specifies the syntax and semantics of Payment-Encapsulating URIs, the workflow of generating and handling them, and requisite infrastructure. -At its core, a URI encapsulated payment communicates the existence of a transaction (specifically a note committing to an amount of funds) to a receiving client. The URI encode contains the amount of the payment and a key used to derive all necessary randomness for note construction including the address and secret key needed to spend it. +At its core, a URI encapsulated payment communicates the existence of a transaction (specifically a note committing to an amount of funds) to a receiving client. The URI encodes the amount of the payment and a key used to derive all necessary randomness for note construction including the address and secret key needed to spend it. Usage Story ----------- @@ -407,4 +407,3 @@ Publication Blockers * Register all domains mentioned in this draft (``withzcash.com``, ``testzcash.com``). * Clean up semantics. - From 94f81896da2150f26cbf81e1d163b056c7b12250 Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 08:56:17 -0500 Subject: [PATCH 006/231] Correct for fee change in ZIP-313 Co-authored-by: Deirdre Connolly --- zip-liberated-payments.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index c5eb2bf2b..7f557d8e2 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -41,7 +41,7 @@ At its core, a URI encapsulated payment communicates the existence of a transact Usage Story ----------- -Alice wants to send a 1.23 ZEC payment to Bob. She generates a fresh ephemeral spending key and sends 1.2301 ZEC to it. This transaction is submitted to the network and mined. Meanwhile, Alice constructs a URI containing the ephemeral spending key and appropriate metadata, and sends it to Bob over an end-to-end encrypted channel such as Signal or WhatsApp. +Alice wants to send a 1.23 ZEC payment to Bob. She generates a fresh ephemeral spending key and sends 1.23001 ZEC to it. This transaction is submitted to the network and mined. Meanwhile, Alice constructs a URI containing the ephemeral spending key and appropriate metadata, and sends it to Bob over an end-to-end encrypted channel such as Signal or WhatsApp. Bob receives a message, which looks as follows: @@ -406,4 +406,3 @@ Publication Blockers ==================== * Register all domains mentioned in this draft (``withzcash.com``, ``testzcash.com``). * Clean up semantics. - From 55bdf1480bb5c4379d325cf40f057f54f86fdc85 Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 08:57:56 -0500 Subject: [PATCH 007/231] Another correction for the fee change in ZIP-313 Co-authored-by: Deirdre Connolly --- zip-liberated-payments.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index 7f557d8e2..d2ef61045 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -51,7 +51,7 @@ Bob receives a message, which looks as follows: https://pay.withzcash.com:65536/v1#amount=1.23&desc=Payment+for+foo&key=... If you do not yet have a Zcash wallet app, see: https://z.cash/wallets -Bob clicks the link. His Zcash mobile wallet app (which is already installed and has configured itself as a handler for URLs of that form) shows up, and tells Bob that a payment of 1.23 ZEC awaits him. The wallet app confirms the existence of the pertinent transactions on the blockchain, and then offers to finalize the payment. Bob clicks the “Finalize” button, and his wallet app generates a transaction moving the money to his own address (using the extra 0.0001 ZEC he has received to pay the transaction fee). When this transaction is confirmed on-chain, Alice’s and Bob’s wallets both indicate that the payment is finalized, and thus Bob can send the funds to other parties. +Bob clicks the link. His Zcash mobile wallet app (which is already installed and has configured itself as a handler for URLs of that form) shows up, and tells Bob that a payment of 1.23 ZEC awaits him. The wallet app confirms the existence of the pertinent transactions on the blockchain, and then offers to finalize the payment. Bob clicks the “Finalize” button, and his wallet app generates a transaction moving the money to his own address (using the extra 0.00001 ZEC he has received to pay the transaction fee). When this transaction is confirmed on-chain, Alice’s and Bob’s wallets both indicate that the payment is finalized, and thus Bob can send the funds to other parties. Motivation From 346b84d48cb353d1cca759011191aab3803417b6 Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 09:39:20 -0500 Subject: [PATCH 008/231] Typo correction. Co-authored-by: Deirdre Connolly --- zip-liberated-payments.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index d2ef61045..cf07ba764 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -79,7 +79,7 @@ This proposal’s specification of Payment-Encapsulating URIs, and the intended * The URIs and protocol should minimize the likelihood of inadvertent misuse, and in particular the risks discussed in “Security Considerations” below. -* The protocol must not leak any information (sender identity, recipient identity, amount, description) to third parties, other than inevitable metadata about the existence of a transaction, the inevitable network communication around sending/receipts of transactions, whatever leakage is induced by the communication channel used to transmit the URI, and whatever is voluntarily shared by the parties. +* The protocol must not leak any information (sender identity, recipient identity, amount, description) to third parties, other than inevitable metadata about the existence of a transaction, the inevitable network communication around sending/receipt of transactions, whatever leakage is induced by the communication channel used to transmit the URI, and whatever is voluntarily shared by the parties. * The URIs should allow for future modifications and expansion of the format, without risk of ambiguous parsing. From f20bdfcb793435229118e7c4824d7f6d69f3b27d Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 09:42:42 -0500 Subject: [PATCH 009/231] Addresses PR suggestion https://github.com/zcash/zips/pull/420#discussion_r533493323 --- zip-liberated-payments.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index cf07ba764..70ac4998a 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -34,7 +34,7 @@ Abstract This proposal defines a mechanism for sending a Zcash payment encapsulated in a URI string. This enables sending Zcash funds over any secure channel, such as via a messaging app, even if the recipient does not yet have Zcash software installed and does not have a Zcash payment address. This is implemented by having the URI convey the secret spending key of an ephemeral Zcash “wallet address”, to which the funds have been transferred. Anyone who learns the URI can accept this payment, by a “finalization” process which uses the key given in the URI to transfer the encapsulated funds to their own wallet. After the payment is finalized, via a suitable on-chain transaction by the recipient, it becomes irrevocable. -The proposal specifies the syntax and semantics of Payment-Encapsulating URIs, the workflow of generating and handling them, and requisite infrastructure. +The proposal specifies the syntax and semantics of URI-Encapsulated Payments, the workflow of generating and handling them, and requisite infrastructure. At its core, a URI encapsulated payment communicates the existence of a transaction (specifically a note committing to an amount of funds) to a receiving client. The URI encodes the amount of the payment and a key used to derive all necessary randomness for note construction including the address and secret key needed to spend it. From fd26084810b87e52d1ab56f47481069ac1020857 Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 09:45:02 -0500 Subject: [PATCH 010/231] Clarification. Co-authored-by: Deirdre Connolly --- zip-liberated-payments.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index 70ac4998a..f04daee77 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -85,7 +85,7 @@ This proposal’s specification of Payment-Encapsulating URIs, and the intended * The on-chain footprint of payments that use this mechanism should be indistinguishable from normal fully-shielded transactions (except, possibly, for the statistics of the number of shielded inputs and outputs). -* Don’t lose funds, even if wallets crash. +* Don’t lose funds, even if wallets crash, or everything but the sending wallet master secret is lost. Non-requirements ================ From aa8f52924dffe3dc5df73402bcc2002d493d01c3 Mon Sep 17 00:00:00 2001 From: Kevin Gorham Date: Wed, 9 Dec 2020 09:45:21 -0500 Subject: [PATCH 011/231] Formatting. Co-authored-by: Deirdre Connolly --- zip-liberated-payments.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/zip-liberated-payments.rst b/zip-liberated-payments.rst index f04daee77..76d7920e5 100644 --- a/zip-liberated-payments.rst +++ b/zip-liberated-payments.rst @@ -48,7 +48,8 @@ Bob receives a message, which looks as follows: This message contains a Zcash payment. Click the following link to view it and receive the funds using your Zcash wallet app: -https://pay.withzcash.com:65536/v1#amount=1.23&desc=Payment+for+foo&key=... + https://pay.withzcash.com:65536/v1#amount=1.23&desc=Payment+for+foo&key=... + If you do not yet have a Zcash wallet app, see: https://z.cash/wallets Bob clicks the link. His Zcash mobile wallet app (which is already installed and has configured itself as a handler for URLs of that form) shows up, and tells Bob that a payment of 1.23 ZEC awaits him. The wallet app confirms the existence of the pertinent transactions on the blockchain, and then offers to finalize the payment. Bob clicks the “Finalize” button, and his wallet app generates a transaction moving the money to his own address (using the extra 0.00001 ZEC he has received to pay the transaction fee). When this transaction is confirmed on-chain, Alice’s and Bob’s wallets both indicate that the payment is finalized, and thus Bob can send the funds to other parties. From 386626af5a477d235e5ff83fd48b0fb5a6b5c3bb Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Fri, 11 Dec 2020 23:42:18 +0000 Subject: [PATCH 012/231] Working draft of ZIP 311 --- zip-0311.rst | 259 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 257 insertions(+), 2 deletions(-) diff --git a/zip-0311.rst b/zip-0311.rst index 94b7b0e88..d23f74c0f 100644 --- a/zip-0311.rst +++ b/zip-0311.rst @@ -2,7 +2,262 @@ ZIP: 311 Title: Sapling Payment Disclosure - Owners: Deirdre Connolly - Status: Reserved + Owners: Jack Grigg + Status: Draft Category: Standards / RPC / Wallet Discussions-To: + + +Abstract +======== + +This ZIP describes a mechanism and format for disclosing information about Sapling spends +and outputs within a transaction. In the typical case, this means enabling a sender to +present a proof that they transferred funds to a recipient's Sapling address. + + +Motivation +========== + +Payment disclosure is useful in a number of situations, for example: + +- A sender may need to prove that their payment was sent and received by a recipient. +- A third party needs to verify that a payment between sender and recipient was executed + successfully. + +When a transaction involves only transparent addresses, the sender and recipient can verify payment by examining the blockchain. A third party can also perform this verification if they know the transparent addresses of the involved parties. + +However, if the transaction involves shielded addresses, the blockchain by itself does not contain enough information to allow a record of the payment to be reconstructed and verified. + +Let us examine the types of transaction that might occur and when a method of payment disclosure would be useful: + +transparent --> transparent + The source, destination and amount are visible on the public blockchain. Payment + disclosure is not required to view the transaction. + +transparent --> shielded + The destination address and amount sent to that address cannot be confirmed. Payment + disclosure is required. + +shielded --> transparent + The recipient can see the amount received at their destination address, but cannot + identify the sender. Requires either an interactive payment disclosure, or a + non-interactive payment disclosure that reveals the sender's address. + +shielded --> shielded + The sender, recipient and amount are unknown. Payment disclosure required. + + +Use cases +========= + +Managing contributions for an event +----------------------------------- +Alice runs an event on behalf of Bob and Carol, who both agree to split the cost equally. +Alice receives a single payment for half the amount, and wants proof of who it came from +(so she knows which person to follow up with for the remaining amount). Carol can provide +a payment disclosure that reveals: + +- The correct amount was sent to the Alice's recipient address in the given transaction. +- Carol was the sender of that transaction (more precisely, Carol controls the spend + authority used in that transaction). + +and does not reveal: + +- Carol's payment address. +- Any of Carol's other diversified payment addresses. +- Any of Carol's other transactions. + +Donation drive +-------------- + +Diana is a well-known individual with a public Sapling payment address (for receiving +donations). She runs a matching donation drive to the non-profit Totally Legit Bunnies, +and wants to prove to her followers that she correctly matched their dontations. Diana +can create a payment disclosure, and post it to her social media account, that reveals: + +- The correct amount was sent to the Totally Legit Bunnies' recipient address in the + given transaction. +- The sender of the transaction is the payment address that is associated with Diana. + +and does not reveal: + +- Any of Diana's other diversified payment addresses. +- Any of Diana's other transactions. + +Merchant dispute +---------------- + +Edward goes to CoinCafe to buy a sandwich, and sends funds from his Sapling address to +CoinCafe's transparent address. They can see a payment has been made, but claim that they +have not received a payment from Edward. He can create a payment disclosure that reveals: + +- Edward was the sender of the given transaction. + +and does not reveal: + +- Edward's payment address. +- Any of Edward's other diversified payment addresses. +- Any of Edward's other transactions. + +Shielded withdrawals from an exchange +------------------------------------- + +CorpBux is an exchange with a transparent hot wallet, and enables customers to withdraw +their funds to Sapling addresses. They can create a payment disclosure, that they can give +to their customers, that reveals: + +- The correct amount was sent to the customer's recipient address in the given transaction. +- CorpBux was the sender of the given transaction. + + +Requirements +============ + +- The payment disclosure may disclose the contents of zero or more Sapling outputs within + a single transaction. (Zero outputs is useful for t2z transactions.) +- Payment disclosures are non-malleable, and can only be created by a sender of the + transaction. +- A payment disclosure may be tied to a challenge, creating an interactive proof. +- Senders with a Sapling address may choose to provably reveal that address as part of the + payment disclosure. +- Senders are not required to remember any ephemeral data from the creation of a transaction + in order to produce a payment disclosure for that transaction. (Performance may be + improved if they do cache witness data for spent notes.) + + +Specification +============= + +Payment disclosure data structure +--------------------------------- + +A payment disclosure has the following fields: + +- `txid`: Transaction id for the given transaction `tx`. + +- `msg`: A message field, which could contain a challenge value from the party to whom + the payment disclosure is directed. + +- `sapling_outputs`: A vector of the Sapling Outputs that we are disclosing + [0..tx.shieldedOutputs.len()] + + - `index`: An index into `tx.shieldedOutputs`. + - `ock`: The outgoing cipher key that allows this output to be recovered. + +- `sapling_spends`: A vector of the Sapling Spends for which we are proving spend authority + [0..tx.shieldedSpends.len()] + + - `index`: An index into `tx.shieldedSpends`. + - `cv` + - `rk` + - `zkproof_spend` + - [Optional] A payment address proof: + - Any `(d, pk_d)` such that `pk_d = [ivk] GH(d)` + - `nullifier_addr` + - `zkproof_addr` + - `spendAuthSig` + +- `transparent_inputs`: A vector of the transparent inputs for which we are proving spend + authority [0..tx.vin.len()] + + - `index`: An index into `tx.vin`. + - `sig`: A BIP 322 signature. + + - TODO: `zcashd` currently only supports the legacy format defined in BIP 322. We may + want to backport full BIP 322 support before having transparent input support in + this ZIP, to ensure it does what we need. + - TODO: BIP 322 specifies consensus rule checks as part of the signature verification + process. We will likely need to migrate it over to an equivalent ZIP that specifies + these for Zcash (which has a different set of script validation consensus rules). + +Creating a payment disclosure +----------------------------- + +The inputs to a payment disclosure are: + +- The transaction. +- The SLIP-44 [#slip-0044]_ coin type. +- The message :math:`msg` to be included (which may be empty). +- A vector of `(output_index, ock)` tuples. +- A vector of tuples of: + + - A Sapling spend index. + - Its corresponding expanded spending key :math:`(\mathsf{ask}, \mathsf{nsk}, \mathsf{ovk})`. + - (Optional) An associated payment address :math:`(\mathsf{d}, \mathsf{pk_d})`. + +- A vector of tuples of: + + - `index`: An index into `tx.vin`. + - The inputs to a BIP 322 signature (excluding `message_data`). + +The payment disclosure is created as follows: + +- For each Sapling spend index: + + - Create a Sapling spend proof for the note that was spent in `tx.shieldedSpends[index]`, + using the same anchor, to obtain `(cv, rk, zkproof_spend)` as well as the random α + that was generated internally. + + - [Optional] If an associated payment address was provided for this spend index, create + a ZIP 304 signature proof for that payment address, using α and `rk` from the + previous step. We obtain `(nullifier_addr, zkproof_addr)` from this step. + +- For each transparent input index: + + - TODO: Prepare BIP 322 signature inputs. + +- Construct an unsigned payment disclosure from the disclosed Sapling outputs, and the + above data for the Sapling spends and transparent inputs. + +- For each Sapling spend index: + + - Let :math:`\mathsf{rsk} = \mathsf{SpendAuthSig.RandomizePrivate}(α, \mathsf{ask})`. + + - Let :math:`coinType` be the 4-byte little-endian encoding of the coin type in its + index form, not its hardened form (i.e. 133 for mainnet Zcash). + + - Let :math:`digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, payment_disclosure_digest)`. + + - Let :math:`spendAuthSig = \mathsf{SpendAuthSig.Sign}(\mathsf{rsk}, digest)`. + +- For each transparent input index: + + - TODO: Create a BIP 322 signature. + +- Return the payment disclosure. + +Verifying a payment disclosure +------------------------------ + +UIs displaying a payment disclosure MUST support three levels of assurance in a payment disclosure: + +- Level 1: The payment disclosure contains valid `ock`s that disclose the specified outputs in the specified transaction. + - Full Sapling note decryption MUST succeed for all specified outputs. More precisely, all specified outputs MUST have valid note commitments when recovered using the specified corresponding `ock` via the process in Section 4.17.3. + - A payment disclosure that only satisfies level 1 may have been: + - modified to be a subset of some other payment disclosure; or + - merged with another level 1 payment disclosure for the same transaction; or + - extracted from a level 2+ payment disclosure. +- Level 2: In addition to level 1, the payment disclosure has been created by one or more of the senders of the original transaction. + +- Level 3: In addition to level 2, one or more of the proven spend authorities is bound to a + +Random notes below here + + +Rationale +========= + +If a sender elects, at transaction creation time, to use an `ovk` of :math:`\bot` for a +specific Sapling output, then they are unable to subsequently create a payment disclosure +that discloses that output. This maintains the semantics of `ovk`, in that the sender +explicitly chose to lose the capability to recover that output. + +Payment disclosures that prove Sapling spend authority are not required to reveal a +sender address. This is because it is impossible: we can "prove" the transaction came from +any of the diversified addresses linked to the spending key. Fundamentally, the "sender" +of a transaction is anyone who has access to the corresponding spend authority; in the +case of Sapling, a spend authority corresponds to multiple diversified addresses. In +situations where a sender address is already known to the verifier of the payment +disclosure, it may still be useful to have the option of linking the payment disclosure to +that address. From dbf56abbf485d1860b9bc5993709c64c1968ddcd Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Fri, 18 Dec 2020 23:26:02 +0000 Subject: [PATCH 013/231] ZIP 311: Add payment disclosure verification, general draft improvements --- zip-0311.rst | 311 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 240 insertions(+), 71 deletions(-) diff --git a/zip-0311.rst b/zip-0311.rst index d23f74c0f..f52ce9360 100644 --- a/zip-0311.rst +++ b/zip-0311.rst @@ -19,33 +19,38 @@ present a proof that they transferred funds to a recipient's Sapling address. Motivation ========== -Payment disclosure is useful in a number of situations, for example: +There are various situations where a proof-of-payment may be desired. For example: -- A sender may need to prove that their payment was sent and received by a recipient. -- A third party needs to verify that a payment between sender and recipient was executed - successfully. +- A sender may need to prove that their payment was sent to a recipient, and available to + be received (following the Zcash protocol in-band). +- A third party may need to verify that a payment between a given sender and recipient was + executed successfully. -When a transaction involves only transparent addresses, the sender and recipient can verify payment by examining the blockchain. A third party can also perform this verification if they know the transparent addresses of the involved parties. +When a transaction involves only transparent addresses, proof-of-payment is simple: The +sender provides the transaction ID, and the recipent examines the blockchain to confirm +that the transaction was mined. A third party can also perform this verification if they +know the transparent addresses of the involved parties. -However, if the transaction involves shielded addresses, the blockchain by itself does not contain enough information to allow a record of the payment to be reconstructed and verified. +However, if the transaction involves shielded addresses, the blockchain by itself does not +contain enough information to allow a record of the payment to be reconstructed and +verified: -Let us examine the types of transaction that might occur and when a method of payment disclosure would be useful: +=========== =========== ====== ========= ====== ================== +Source Destination Sender Recipient Amount Payment dislosure? +=========== =========== ====== ========= ====== ================== +Transparent Transparent 🔍 🔍 🔍 Not required +Transparent Shielded 🔍 🔐 🔐 Required +Shielded Transparent 🔒 🔍 🔍 Required +Shielded Shielded 🔒 🔐 🔐 Required +=========== =========== ====== ========= ====== ================== -transparent --> transparent - The source, destination and amount are visible on the public blockchain. Payment - disclosure is not required to view the transaction. +- 🔍 = Visible on the block chain. +- 🔐 = Cannot be confirmed without information that is not visible on the block chain. +- 🔒 = Requires either an interactive payment disclosure, or a non-interactive payment + disclosure that reveals the sender's address. -transparent --> shielded - The destination address and amount sent to that address cannot be confirmed. Payment - disclosure is required. - -shielded --> transparent - The recipient can see the amount received at their destination address, but cannot - identify the sender. Requires either an interactive payment disclosure, or a - non-interactive payment disclosure that reveals the sender's address. - -shielded --> shielded - The sender, recipient and amount are unknown. Payment disclosure required. +Enabling proof-of-payment for all of these transaction variants (where it is possible to +do so) is the subject of this ZIP. Use cases @@ -73,7 +78,7 @@ Donation drive Diana is a well-known individual with a public Sapling payment address (for receiving donations). She runs a matching donation drive to the non-profit Totally Legit Bunnies, -and wants to prove to her followers that she correctly matched their dontations. Diana +and wants to prove to her followers that she correctly matched their donations. Diana can create a payment disclosure, and post it to her social media account, that reveals: - The correct amount was sent to the Totally Legit Bunnies' recipient address in the @@ -88,8 +93,8 @@ and does not reveal: Merchant dispute ---------------- -Edward goes to CoinCafe to buy a sandwich, and sends funds from his Sapling address to -CoinCafe's transparent address. They can see a payment has been made, but claim that they +Edward goes to CarsRFast to buy a Lamborghini, and sends funds from his Sapling address to +CarsRFast's transparent address. They can see a payment has been made, but claim that they have not received a payment from Edward. He can create a payment disclosure that reveals: - Edward was the sender of the given transaction. @@ -115,7 +120,7 @@ Requirements ============ - The payment disclosure may disclose the contents of zero or more Sapling outputs within - a single transaction. (Zero outputs is useful for t2z transactions.) + a single transaction. (Zero outputs is useful for z2t transactions.) - Payment disclosures are non-malleable, and can only be created by a sender of the transaction. - A payment disclosure may be tied to a challenge, creating an interactive proof. @@ -126,6 +131,33 @@ Requirements improved if they do cache witness data for spent notes.) +Conventions +=========== + +The following functions used in this ZIP are defined in the Zcash protocol specification: +[#protocol]_ + +- :math:`\mathsf{SpendAuthSig.RandomizePrivate}(α, \mathsf{sk})`, + :math:`\mathsf{SpendAuthSig.Sign}(\mathsf{sk}, m)`, and + :math:`\mathsf{SpendAuthSig.Verify}(\mathsf{vk}, m, σ)` [#protocol-concretespendauthsig]_ + +We reproduce some notation and functions from [#protocol]_ here for convenience: + +- :math:`[k] P` means scalar multiplication of the elliptic curve point :math:`P` by the + scalar :math:`k`. + +- :math:`\mathsf{BLAKE2b}\text{-}\mathsf{256}(p, x)` refers to unkeyed BLAKE2b-256 in + sequential mode, with an output digest length of 32 bytes, 16-byte personalization + string :math:`p`, and input :math:`x`. + +We also define the following notation here: + +- :math:`[a..b]` means the sequence of values inclusive of :math:`a` and exclusive of + :math:`b`. + +- :math:`a\text{.len}()` means the length of the vector :math:`a`. + + Specification ============= @@ -139,30 +171,33 @@ A payment disclosure has the following fields: - `msg`: A message field, which could contain a challenge value from the party to whom the payment disclosure is directed. -- `sapling_outputs`: A vector of the Sapling Outputs that we are disclosing +- :math:`\mathsf{saplingOutputs}`: A vector of the Sapling Outputs that we are disclosing [0..tx.shieldedOutputs.len()] - - `index`: An index into `tx.shieldedOutputs`. - - `ock`: The outgoing cipher key that allows this output to be recovered. + - :math:`\mathsf{index}`: An index into `tx.shieldedOutputs`. + - :math:`\mathsf{ock}`: The outgoing cipher key that allows this output to be recovered. + [#protocol-saplingencrypt]_ + +- :math:`\mathsf{saplingSpends}`: A vector of the Sapling Spends for which we are proving + spend authority [0..tx.shieldedSpends.len()] + + - :math:`\mathsf{index}`: An index into `tx.shieldedSpends`. + - :math:`\mathsf{cv}`: A value commitment to the spent note. + - :math:`\mathsf{rk}`: A randomized public key linked to the spent note. + - :math:`\mathsf{zkproof_{spend}}`: A Sapling spend proof. + - [Optional] A payment address proof `addr_proof`: -- `sapling_spends`: A vector of the Sapling Spends for which we are proving spend authority - [0..tx.shieldedSpends.len()] + - Any :math:`(\mathsf{d, pk_d})` such that :math:`\mathsf{pk_d} = [\mathsf{ivk}] GH(\mathsf{d})` + - :math:`\mathsf{nullifier_{addr}}`: A nullifier for a ZIP 304 fake note. [#zip-0304]_ + - :math:`\mathsf{zkproof_{addr}}`: A Sapling spend proof. - - `index`: An index into `tx.shieldedSpends`. - - `cv` - - `rk` - - `zkproof_spend` - - [Optional] A payment address proof: - - Any `(d, pk_d)` such that `pk_d = [ivk] GH(d)` - - `nullifier_addr` - - `zkproof_addr` - - `spendAuthSig` + - :math:`\mathsf{spendAuthSig}` -- `transparent_inputs`: A vector of the transparent inputs for which we are proving spend - authority [0..tx.vin.len()] +- :math:`\mathsf{transparentInputs}`: A vector of the transparent inputs for which we are + proving spend authority [0..tx.vin.len()] - - `index`: An index into `tx.vin`. - - `sig`: A BIP 322 signature. + - :math:`\mathsf{index}`: An index into `tx.vin`. + - :math:`\mathsf{sig}`: A BIP 322 signature. [#bip-0322]_ - TODO: `zcashd` currently only supports the legacy format defined in BIP 322. We may want to backport full BIP 322 support before having transparent input support in @@ -171,6 +206,10 @@ A payment disclosure has the following fields: process. We will likely need to migrate it over to an equivalent ZIP that specifies these for Zcash (which has a different set of script validation consensus rules). +TODO: Decide on payment disclosure versioning. + +TODO: Define encodings for unsigned and signed payment disclosures. + Creating a payment disclosure ----------------------------- @@ -179,45 +218,51 @@ The inputs to a payment disclosure are: - The transaction. - The SLIP-44 [#slip-0044]_ coin type. - The message :math:`msg` to be included (which may be empty). -- A vector of `(output_index, ock)` tuples. -- A vector of tuples of: +- A vector of :math:`(\mathsf{outputIndex}, \mathsf{ock})` tuples (which may be empty). +- A vector of Sapling spend tuples (which may be empty) containing: - A Sapling spend index. - Its corresponding expanded spending key :math:`(\mathsf{ask}, \mathsf{nsk}, \mathsf{ovk})`. - - (Optional) An associated payment address :math:`(\mathsf{d}, \mathsf{pk_d})`. + - [Optional] An associated payment address :math:`(\mathsf{d}, \mathsf{pk_d})`. -- A vector of tuples of: +- A vector of transparent input tuples (which may be empty) containing: - - `index`: An index into `tx.vin`. + - :math:`\mathsf{index}`: An index into `tx.vin`. - The inputs to a BIP 322 signature (excluding `message_data`). +The caller MUST provide at least one input tuple of any type (either a Sapling spend tuple +or a transparent input tuple). + The payment disclosure is created as follows: - For each Sapling spend index: - - Create a Sapling spend proof for the note that was spent in `tx.shieldedSpends[index]`, - using the same anchor, to obtain `(cv, rk, zkproof_spend)` as well as the random α - that was generated internally. + - Create a Sapling spend proof for the note that was spent in + :math:`\mathsf{tx.shieldedSpends[index]}`, using the same anchor, to obtain + :math:`(\mathsf{cv}, \mathsf{rk}, \mathsf{zkproof_{spend}})` as well as the random + :math:`\alpha` that was generated internally. - [Optional] If an associated payment address was provided for this spend index, create - a ZIP 304 signature proof for that payment address, using α and `rk` from the - previous step. We obtain `(nullifier_addr, zkproof_addr)` from this step. + a ZIP 304 signature proof for that payment address, [#zip-0304]_ using :math:`\alpha` + and :math:`\mathsf{rk}` from the previous step. We obtain + :math:`(\mathsf{nullifier_{addr}}, \mathsf{zkproof_{addr}})` from this step. - For each transparent input index: - TODO: Prepare BIP 322 signature inputs. - Construct an unsigned payment disclosure from the disclosed Sapling outputs, and the - above data for the Sapling spends and transparent inputs. + above data for the Sapling spends and transparent inputs. Define the encoding of this as + :math:`unsignedPaymentDisclosure`. - For each Sapling spend index: - - Let :math:`\mathsf{rsk} = \mathsf{SpendAuthSig.RandomizePrivate}(α, \mathsf{ask})`. + - Let :math:`\mathsf{rsk} = \mathsf{SpendAuthSig.RandomizePrivate}(\alpha, \mathsf{ask})`. - Let :math:`coinType` be the 4-byte little-endian encoding of the coin type in its index form, not its hardened form (i.e. 133 for mainnet Zcash). - - Let :math:`digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, payment_disclosure_digest)`. + - Let :math:`digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, unsignedPaymentDisclosure)`. - Let :math:`spendAuthSig = \mathsf{SpendAuthSig.Sign}(\mathsf{rsk}, digest)`. @@ -225,33 +270,105 @@ The payment disclosure is created as follows: - TODO: Create a BIP 322 signature. -- Return the payment disclosure. +- Return the payment disclosure as the combination of the unsigned payment disclosure and + the set of `spendAuthSig` and transparent signature values. Verifying a payment disclosure ------------------------------ -UIs displaying a payment disclosure MUST support three levels of assurance in a payment disclosure: +Given a payment disclosure :math:`\mathsf{pd}`, a transaction :math:`\mathsf{tx}`, and the +`height` of the block in which :math:`\mathsf{tx}` was mined (which we assume was verified +by the caller), the verifier proceeds as follows: + +- Perform the following structural correctness checks, returning false if any check fails: + + - :math:`\mathsf{pd.txid} = \mathsf{tx.txid}()` + + - Vector length correctness: + + - :math:`\mathsf{pd.saplingOutputs.len}() \leq \mathsf{tx.shieldedOutputs.len}()` + - :math:`\mathsf{pd.saplingSpends.len}() \leq \mathsf{tx.shieldedSpends.len}()` + - :math:`\mathsf{pd.transparentInputs.len}() \leq \mathsf{tx.vin.len}()` + + - Index uniqueness: + + - For every :math:`\mathsf{output}` in :math:`\mathsf{pd.saplingOutputs}`, + :math:`\mathsf{output.index}` only occurs once. + - For every :math:`\mathsf{spend}` in :math:`\mathsf{pd.saplingSpends}`, + :math:`\mathsf{spend.index}` only occurs once. + - For every :math:`\mathsf{input}` in :math:`\mathsf{pd.transparentInputs}`, + :math:`\mathsf{input.index}` only occurs once. + + - Index correctness: + + - For every :math:`\mathsf{output}` in :math:`\mathsf{pd.saplingOutputs}`, + :math:`\mathsf{output.index} < \mathsf{tx.shieldedOutputs.len}()` + - For every :math:`\mathsf{spend}` in :math:`\mathsf{pd.saplingSpends}`, + :math:`\mathsf{spend.index} < \mathsf{tx.shieldedSpends.len}()` + - For every :math:`\mathsf{input}` in :math:`\mathsf{pd.transparentInputs}`, + :math:`\mathsf{input.index} < \mathsf{tx.vin.len}()` + + - :math:`\mathsf{pd.saplingSpends.len}() + \mathsf{pd.transparentInputs.len}() > 0` + +- Let :math:`unsignedPaymentDisclosure` be the encoding of the payment disclosure without + signatures. -- Level 1: The payment disclosure contains valid `ock`s that disclose the specified outputs in the specified transaction. - - Full Sapling note decryption MUST succeed for all specified outputs. More precisely, all specified outputs MUST have valid note commitments when recovered using the specified corresponding `ock` via the process in Section 4.17.3. - - A payment disclosure that only satisfies level 1 may have been: - - modified to be a subset of some other payment disclosure; or - - merged with another level 1 payment disclosure for the same transaction; or - - extracted from a level 2+ payment disclosure. -- Level 2: In addition to level 1, the payment disclosure has been created by one or more of the senders of the original transaction. +- Let :math:`coinType` be the 4-byte little-endian encoding of the coin type in its index + form, not its hardened form (i.e. 133 for mainnet Zcash). -- Level 3: In addition to level 2, one or more of the proven spend authorities is bound to a +- Let :math:`digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, unsignedPaymentDisclosure)`. -Random notes below here +- For every :math:`\mathsf{spend}` in :math:`\mathsf{pd.saplingSpends}`: + + - If :math:`\mathsf{SpendAuthSig.Verify}(\mathsf{spend.rk}, digest, \mathsf{spend.spendAuthSig}) = 0`, return false. + + - [Optional] If a payment address proof :math:`\mathsf{addrProof}` is present in + :math:`\mathsf{spend}`, verify :math:`(\mathsf{addrProof.nullifier_{addr}}, \mathsf{spend.rk}, \mathsf{addrProof.zkproof_{addr}})` as a ZIP 304 proof + for :math:`(\mathsf{addrProof.d}, \mathsf{addrProof.pk_d})` [#zip-0304]_. If verification fails, return + false. + + - Decode and verify :math:`\mathsf{zkproof_{spend}}` as a Sapling spend proof + [#protocol-spendstatement]_ with primary input: + + - :math:`\mathsf{tx.shieldedSpends[spend.index].rt}` + - :math:`\mathsf{spend.cv}` + - :math:`\mathsf{tx.shieldedSpends[spend.index].nf}` + - :math:`\mathsf{spend.rk}` + + If verification fails, return false. + +- For every :math:`\mathsf{input}` in :math:`\mathsf{pd.transparentInputs}`: + + - TODO: BIP 322 verification. + +- For every :math:`\mathsf{output}` in :math:`\mathsf{pd.saplingOutputs}`: + + - Recover the Sapling note in :math:`\mathsf{tx.shieldedOutputs}[\mathsf{output.index}]` + via the process specified in [#protocol-saplingdecryptovk]_ with inputs + :math:`(height, \mathsf{output.ock})`. If recovery returns :math:`\bot`, return false. + +- Return true. + +Payment disclosure validity in UIs +---------------------------------- + +TODO: Set some standards for how UIs should display payment disclosures, and how they +should convey the various kinds of validity information: + +- One, but not all, of the spenders proved spend authority. +- All spenders of a specific type proved spend authority. +- All spenders proved spend authority. +- These, but also including optional payment address proofs. Rationale ========= -If a sender elects, at transaction creation time, to use an `ovk` of :math:`\bot` for a -specific Sapling output, then they are unable to subsequently create a payment disclosure -that discloses that output. This maintains the semantics of `ovk`, in that the sender -explicitly chose to lose the capability to recover that output. +If a sender elects, at transaction creation time, to use an :math:`\mathsf{ovk}` of +:math:`\bot` for a specific Sapling output, then they are unable to subsequently create a +payment disclosure that discloses that output. This maintains the semantics of +:math:`\mathsf{ovk}`, in that the sender explicitly chose to lose the capability to +recover that output. Payment disclosures that prove Sapling spend authority are not required to reveal a sender address. This is because it is impossible: we can "prove" the transaction came from @@ -261,3 +378,55 @@ case of Sapling, a spend authority corresponds to multiple diversified addresses situations where a sender address is already known to the verifier of the payment disclosure, it may still be useful to have the option of linking the payment disclosure to that address. + + +Security and Privacy Considerations +=================================== + +When spending Sapling notes normally in transactions, wallets select a recent anchor to +make the anonymity set of the spent note as large as possible. By contrast, Sapling spend +authority in a payment disclosure is proven using the same anchor that was used in the +transaction itself, instead of a recent anchor. We do this for efficency reasons: + +- The anchor is already encoded in the transaction, so can be omitted from the payment + disclosure encoding. +- It is necessary to have a witness for each spent note that is being included in the + payment disclosure. Using the same anchor means that the same witness can be used for + the transaction spend and the payment disclosure, which in turn means that wallets that + support payment disclosures only need to remember that witness, and do not need to + continually update witnesses for spent notes in the off-chance that they might be used + in a payment disclosure. + +There is no privacy benefit to selecting a more recent anchor; the anonymity set of the +note was "fixed" by the original spend (which revealed that the note existed as of that +anchor's height). + +We require all payment disclosures to prove spend authority for at least one input, in +order to simplify the verification UX. In particular, if payment disclosures without +spends were considered valid, an invalid payment disclosure with invalid signatures (that +would be shown as invalid by UIs) could be mutated into a payment disclosure that would be +shown as valid by UIs, by stripping off the signatures. We do not believe that this +prevents any useful use cases; meanwhile if someone is intent on obtaining Sapling output +disclosures regardless of the validity of their source, they will do so without a common +standard. + + +Reference implementation +======================== + +TBD + + +References +========== + +.. [#RFC2119] `RFC 2119: Key words for use in RFCs to Indicate Requirement Levels `_ +.. [#RFC4648] `RFC 4648: The Base16, Base32, and Base64 Data Encodings `_ +.. [#protocol] `Zcash Protocol Specification, Version 2020.1.15 or later `_ +.. [#protocol-spendstatement] `Zcash Protocol Specification, Version 2020.1.15. Section 4.15.2: Spend Statement (Sapling) `_ +.. [#protocol-saplingencrypt] `Zcash Protocol Specification, Version 2020.1.15. 4.17.1: Encryption (Sapling) `_ +.. [#protocol-saplingdecryptovk] `Zcash Protocol Specification, Version 2020.1.15. 4.17.3: Decryption using a Full Viewing Key (Sapling) `_ +.. [#protocol-concretespendauthsig] `Zcash Protocol Specification, Version 2020.1.15. Section 5.4.6.1: Spend Authorization Signature `_ +.. [#bip-0322] `BIP 322: Generic Signed Message Format `_ +.. [#slip-0044] `SLIP-0044 : Registered coin types for BIP-0044 `_ +.. [#zip-0304] `ZIP 304: Sapling Address Signatures `_ From 0076879df4924988fa00b9930b29764227f4957d Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 18 Dec 2020 23:30:12 +0000 Subject: [PATCH 014/231] Apply suggestions from code review Co-authored-by: Daira Hopwood --- zip-0311.rst | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/zip-0311.rst b/zip-0311.rst index f52ce9360..ddc157291 100644 --- a/zip-0311.rst +++ b/zip-0311.rst @@ -72,6 +72,8 @@ and does not reveal: - Carol's payment address. - Any of Carol's other diversified payment addresses. - Any of Carol's other transactions. +For this use case, it is not necessary (and not necessarily intended) that the payment disclosure +be publically verifiable. Donation drive -------------- @@ -112,7 +114,8 @@ CorpBux is an exchange with a transparent hot wallet, and enables customers to w their funds to Sapling addresses. They can create a payment disclosure, that they can give to their customers, that reveals: -- The correct amount was sent to the customer's recipient address in the given transaction. +- The correct amount was sent to the customer's shielded recipient address in the given + transaction. - CorpBux was the sender of the given transaction. @@ -121,8 +124,10 @@ Requirements - The payment disclosure may disclose the contents of zero or more Sapling outputs within a single transaction. (Zero outputs is useful for z2t transactions.) -- Payment disclosures are non-malleable, and can only be created by a sender of the - transaction. +- Payment disclosures can only be created by a sender of the transaction, + and are non-malleable (that is, no-one else can take one or more payment + disclosures and construct a new one that they would not have been able + to make independently). - A payment disclosure may be tied to a challenge, creating an interactive proof. - Senders with a Sapling address may choose to provably reveal that address as part of the payment disclosure. @@ -166,7 +171,7 @@ Payment disclosure data structure A payment disclosure has the following fields: -- `txid`: Transaction id for the given transaction `tx`. +- `txid`: Transaction id for the transaction `tx` being disclosed. - `msg`: A message field, which could contain a challenge value from the party to whom the payment disclosure is directed. @@ -249,7 +254,7 @@ The payment disclosure is created as follows: - For each transparent input index: - - TODO: Prepare BIP 322 signature inputs. + - TODO: Prepare BIP 322 signature inputs using `msg` as the `message_data`. - Construct an unsigned payment disclosure from the disclosed Sapling outputs, and the above data for the Sapling spends and transparent inputs. Define the encoding of this as @@ -259,7 +264,7 @@ The payment disclosure is created as follows: - Let :math:`\mathsf{rsk} = \mathsf{SpendAuthSig.RandomizePrivate}(\alpha, \mathsf{ask})`. - - Let :math:`coinType` be the 4-byte little-endian encoding of the coin type in its + - Let :math:`coinType` be the 4-byte little-endian encoding of the SLIP 44 coin type in its index form, not its hardened form (i.e. 133 for mainnet Zcash). - Let :math:`digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, unsignedPaymentDisclosure)`. @@ -268,7 +273,7 @@ The payment disclosure is created as follows: - For each transparent input index: - - TODO: Create a BIP 322 signature. + - TODO: Create a BIP 322 signature using `msg` as the `message_data`. - Return the payment disclosure as the combination of the unsigned payment disclosure and the set of `spendAuthSig` and transparent signature values. @@ -376,7 +381,7 @@ any of the diversified addresses linked to the spending key. Fundamentally, the of a transaction is anyone who has access to the corresponding spend authority; in the case of Sapling, a spend authority corresponds to multiple diversified addresses. In situations where a sender address is already known to the verifier of the payment -disclosure, it may still be useful to have the option of linking the payment disclosure to +disclosure (or publically), it may still be useful to have the option of linking the payment disclosure to that address. From 28bc4e16115d39ccf35a54b50cc84633198c12cf Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Fri, 18 Dec 2020 23:31:14 +0000 Subject: [PATCH 015/231] ZIP 311: Regenerate HTML --- README.rst | 2 +- index.html | 2 +- zip-0311.html | 603 +++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 603 insertions(+), 4 deletions(-) diff --git a/README.rst b/README.rst index 3f25986f9..99d0b7046 100644 --- a/README.rst +++ b/README.rst @@ -112,7 +112,7 @@ Index of ZIPs 308 Sprout to Sapling Migration Final 309 Blind Off-chain Lightweight Transactions (BOLT) Reserved 310 Security Properties of Sapling Viewing Keys Draft - 311 Sapling Payment Disclosure Reserved + 311 Sapling Payment Disclosure Draft 312 Shielded Multisignatures using FROST Reserved 313 Reduce Default Transaction Fee Reserved 321 Payment Request URIs Proposed diff --git a/index.html b/index.html index f33ed0501..b34100e7e 100644 --- a/index.html +++ b/index.html @@ -85,7 +85,7 @@ 308 Sprout to Sapling Migration Final 309 Blind Off-chain Lightweight Transactions (BOLT) Reserved 310 Security Properties of Sapling Viewing Keys Draft - 311 Sapling Payment Disclosure Reserved + 311 Sapling Payment Disclosure Draft 312 Shielded Multisignatures using FROST Reserved 313 Reduce Default Transaction Fee Reserved 321 Payment Request URIs Proposed diff --git a/zip-0311.html b/zip-0311.html index 9d6a86e98..6ae8faba3 100644 --- a/zip-0311.html +++ b/zip-0311.html @@ -3,15 +3,614 @@ ZIP 311: Sapling Payment Disclosure + 
ZIP: 311
 Title: Sapling Payment Disclosure
-Owners: Deirdre Connolly <deirdre@zfnd.org>
-Status: Reserved
+Owners: Jack Grigg <jack@electriccoin.co>
+Status: Draft
 Category: Standards / RPC / Wallet
 Discussions-To: <https://github.com/zcash/zips/issues/387>
+

Abstract

+

This ZIP describes a mechanism and format for disclosing information about Sapling spends and outputs within a transaction. In the typical case, this means enabling a sender to present a proof that they transferred funds to a recipient's Sapling address.

+
+

Motivation

+

There are various situations where a proof-of-payment may be desired. For example:

+
    +
  • A sender may need to prove that their payment was sent to a recipient, and available to be received (following the Zcash protocol in-band).
    • +
  • A third party may need to verify that a payment between a given sender and recipient was executed successfully.
    • +
+

When a transaction involves only transparent addresses, proof-of-payment is simple: The sender provides the transaction ID, and the recipent examines the blockchain to confirm that the transaction was mined. A third party can also perform this verification if they know the transparent addresses of the involved parties.

+

However, if the transaction involves shielded addresses, the blockchain by itself does not contain enough information to allow a record of the payment to be reconstructed and verified:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SourceDestinationSenderRecipientAmountPayment dislosure?
TransparentTransparent🔍🔍🔍Not required
TransparentShielded🔍🔐🔐Required
ShieldedTransparent🔒🔍🔍Required
ShieldedShielded🔒🔐🔐Required
+
    +
  • 🔍 = Visible on the block chain.
    • +
  • 🔐 = Cannot be confirmed without information that is not visible on the block chain.
    • +
  • 🔒 = Requires either an interactive payment disclosure, or a non-interactive payment disclosure that reveals the sender's address.
    • +
+

Enabling proof-of-payment for all of these transaction variants (where it is possible to do so) is the subject of this ZIP.

+
+

Use cases

+

Managing contributions for an event

+

Alice runs an event on behalf of Bob and Carol, who both agree to split the cost equally. Alice receives a single payment for half the amount, and wants proof of who it came from (so she knows which person to follow up with for the remaining amount). Carol can provide a payment disclosure that reveals:

+
    +
  • The correct amount was sent to the Alice's recipient address in the given transaction.
    • +
  • Carol was the sender of that transaction (more precisely, Carol controls the spend authority used in that transaction).
    • +
+

and does not reveal:

+
    +
  • Carol's payment address.
    • +
  • Any of Carol's other diversified payment addresses.
    • +
  • Any of Carol's other transactions.
    • +
+
+

System Message: WARNING/2 (zip-0311.rst line 75)

+

Bullet list ends without a blank line; unexpected unindent.

+
+

For this use case, it is not necessary (and not necessarily intended) that the payment disclosure be publically verifiable.

+
+

Donation drive

+

Diana is a well-known individual with a public Sapling payment address (for receiving donations). She runs a matching donation drive to the non-profit Totally Legit Bunnies, and wants to prove to her followers that she correctly matched their donations. Diana can create a payment disclosure, and post it to her social media account, that reveals:

+
    +
  • The correct amount was sent to the Totally Legit Bunnies' recipient address in the given transaction.
    • +
  • The sender of the transaction is the payment address that is associated with Diana.
    • +
+

and does not reveal:

+
    +
  • Any of Diana's other diversified payment addresses.
    • +
  • Any of Diana's other transactions.
    • +
+
+

Merchant dispute

+

Edward goes to CarsRFast to buy a Lamborghini, and sends funds from his Sapling address to CarsRFast's transparent address. They can see a payment has been made, but claim that they have not received a payment from Edward. He can create a payment disclosure that reveals:

+
    +
  • Edward was the sender of the given transaction.
    • +
+

and does not reveal:

+
    +
  • Edward's payment address.
    • +
  • Any of Edward's other diversified payment addresses.
    • +
  • Any of Edward's other transactions.
    • +
+
+

Shielded withdrawals from an exchange

+

CorpBux is an exchange with a transparent hot wallet, and enables customers to withdraw their funds to Sapling addresses. They can create a payment disclosure, that they can give to their customers, that reveals:

+
    +
  • The correct amount was sent to the customer's shielded recipient address in the given transaction.
    • +
  • CorpBux was the sender of the given transaction.
    • +
+
+
+

Requirements

+
    +
  • The payment disclosure may disclose the contents of zero or more Sapling outputs within a single transaction. (Zero outputs is useful for z2t transactions.)
    • +
  • Payment disclosures can only be created by a sender of the transaction, and are non-malleable (that is, no-one else can take one or more payment disclosures and construct a new one that they would not have been able to make independently).
    • +
  • A payment disclosure may be tied to a challenge, creating an interactive proof.
    • +
  • Senders with a Sapling address may choose to provably reveal that address as part of the payment disclosure.
    • +
  • Senders are not required to remember any ephemeral data from the creation of a transaction in order to produce a payment disclosure for that transaction. (Performance may be improved if they do cache witness data for spent notes.)
    • +
+
+

Conventions

+

The following functions used in this ZIP are defined in the Zcash protocol specification: 3

+
    +
  • + \(\mathsf{SpendAuthSig.RandomizePrivate}(α, \mathsf{sk})\) + , + \(\mathsf{SpendAuthSig.Sign}(\mathsf{sk}, m)\) + , and + \(\mathsf{SpendAuthSig.Verify}(\mathsf{vk}, m, σ)\) + 7
    • +
+

We reproduce some notation and functions from 3 here for convenience:

+
    +
  • + \([k] P\) + means scalar multiplication of the elliptic curve point + \(P\) + by the scalar + \(k\) + .
    • +
  • + \(\mathsf{BLAKE2b}\text{-}\mathsf{256}(p, x)\) + refers to unkeyed BLAKE2b-256 in sequential mode, with an output digest length of 32 bytes, 16-byte personalization string + \(p\) + , and input + \(x\) + .
    • +
+

We also define the following notation here:

+
    +
  • + \([a..b]\) + means the sequence of values inclusive of + \(a\) + and exclusive of + \(b\) + .
    • +
  • + \(a\text{.len}()\) + means the length of the vector + \(a\) + .
    • +
+
+

Specification

+

Payment disclosure data structure

+

A payment disclosure has the following fields:

+
    +
  • txid: Transaction id for the transaction tx being disclosed.
    • +
  • msg: A message field, which could contain a challenge value from the party to whom the payment disclosure is directed.
    • +
  • + \(\mathsf{saplingOutputs}\) + : A vector of the Sapling Outputs that we are disclosing [0..tx.shieldedOutputs.len()] +
      +
    • + \(\mathsf{index}\) + : An index into tx.shieldedOutputs.
      • +
    • + \(\mathsf{ock}\) + : The outgoing cipher key that allows this output to be recovered. 5
      • +
    +
    • +
  • + \(\mathsf{saplingSpends}\) + : A vector of the Sapling Spends for which we are proving spend authority [0..tx.shieldedSpends.len()] +
      +
    • + \(\mathsf{index}\) + : An index into tx.shieldedSpends.
      • +
    • + \(\mathsf{cv}\) + : A value commitment to the spent note.
      • +
    • + \(\mathsf{rk}\) + : A randomized public key linked to the spent note.
      • +
    • + \(\mathsf{zkproof_{spend}}\) + : A Sapling spend proof.
      • +
    • [Optional] A payment address proof addr_proof: +
        +
      • Any + \((\mathsf{d, pk_d})\) + such that + \(\mathsf{pk_d} = [\mathsf{ivk}] GH(\mathsf{d})\) +
        • +
      • + \(\mathsf{nullifier_{addr}}\) + : A nullifier for a ZIP 304 fake note. 10
        • +
      • + \(\mathsf{zkproof_{addr}}\) + : A Sapling spend proof.
        • +
      +
      • +
    • + \(\mathsf{spendAuthSig}\) +
      • +
    +
    • +
  • + \(\mathsf{transparentInputs}\) + : A vector of the transparent inputs for which we are proving spend authority [0..tx.vin.len()] +
      +
    • + \(\mathsf{index}\) + : An index into tx.vin.
      • +
    • + \(\mathsf{sig}\) + : A BIP 322 signature. 8 +
        +
      • TODO: zcashd currently only supports the legacy format defined in BIP 322. We may want to backport full BIP 322 support before having transparent input support in this ZIP, to ensure it does what we need.
        • +
      • TODO: BIP 322 specifies consensus rule checks as part of the signature verification process. We will likely need to migrate it over to an equivalent ZIP that specifies these for Zcash (which has a different set of script validation consensus rules).
        • +
      +
      • +
    +
    • +
+

TODO: Decide on payment disclosure versioning.

+

TODO: Define encodings for unsigned and signed payment disclosures.

+
+

Creating a payment disclosure

+

The inputs to a payment disclosure are:

+
    +
  • The transaction.
    • +
  • The SLIP-44 9 coin type.
    • +
  • The message + \(msg\) + to be included (which may be empty).
    • +
  • A vector of + \((\mathsf{outputIndex}, \mathsf{ock})\) + tuples (which may be empty).
    • +
  • A vector of Sapling spend tuples (which may be empty) containing: +
      +
    • A Sapling spend index.
      • +
    • Its corresponding expanded spending key + \((\mathsf{ask}, \mathsf{nsk}, \mathsf{ovk})\) + .
      • +
    • [Optional] An associated payment address + \((\mathsf{d}, \mathsf{pk_d})\) + .
      • +
    +
    • +
  • A vector of transparent input tuples (which may be empty) containing: +
      +
    • + \(\mathsf{index}\) + : An index into tx.vin.
      • +
    • The inputs to a BIP 322 signature (excluding message_data).
      • +
    +
    • +
+

The caller MUST provide at least one input tuple of any type (either a Sapling spend tuple or a transparent input tuple).

+

The payment disclosure is created as follows:

+
    +
  • For each Sapling spend index: +
      +
    • Create a Sapling spend proof for the note that was spent in + \(\mathsf{tx.shieldedSpends[index]}\) + , using the same anchor, to obtain + \((\mathsf{cv}, \mathsf{rk}, \mathsf{zkproof_{spend}})\) + as well as the random + \(\alpha\) + that was generated internally.
      • +
    • [Optional] If an associated payment address was provided for this spend index, create a ZIP 304 signature proof for that payment address, 10 using + \(\alpha\) + and + \(\mathsf{rk}\) + from the previous step. We obtain + \((\mathsf{nullifier_{addr}}, \mathsf{zkproof_{addr}})\) + from this step.
      • +
    +
    • +
  • For each transparent input index: +
      +
    • TODO: Prepare BIP 322 signature inputs using msg as the message_data.
      • +
    +
    • +
  • Construct an unsigned payment disclosure from the disclosed Sapling outputs, and the above data for the Sapling spends and transparent inputs. Define the encoding of this as + \(unsignedPaymentDisclosure\) + .
    • +
  • For each Sapling spend index: +
      +
    • Let + \(\mathsf{rsk} = \mathsf{SpendAuthSig.RandomizePrivate}(\alpha, \mathsf{ask})\) + .
      • +
    • Let + \(coinType\) + be the 4-byte little-endian encoding of the SLIP 44 coin type in its index form, not its hardened form (i.e. 133 for mainnet Zcash).
      • +
    • Let + \(digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, unsignedPaymentDisclosure)\) + .
      • +
    • Let + \(spendAuthSig = \mathsf{SpendAuthSig.Sign}(\mathsf{rsk}, digest)\) + .
      • +
    +
    • +
  • For each transparent input index: +
      +
    • TODO: Create a BIP 322 signature using msg as the message_data.
      • +
    +
    • +
  • Return the payment disclosure as the combination of the unsigned payment disclosure and the set of spendAuthSig and transparent signature values.
    • +
+
+

Verifying a payment disclosure

+

Given a payment disclosure + \(\mathsf{pd}\) + , a transaction + \(\mathsf{tx}\) + , and the height of the block in which + \(\mathsf{tx}\) + was mined (which we assume was verified by the caller), the verifier proceeds as follows:

+
    +
  • Perform the following structural correctness checks, returning false if any check fails: +
      +
    • + \(\mathsf{pd.txid} = \mathsf{tx.txid}()\) +
      • +
    • Vector length correctness: +
        +
      • + \(\mathsf{pd.saplingOutputs.len}() \leq \mathsf{tx.shieldedOutputs.len}()\) +
        • +
      • + \(\mathsf{pd.saplingSpends.len}() \leq \mathsf{tx.shieldedSpends.len}()\) +
        • +
      • + \(\mathsf{pd.transparentInputs.len}() \leq \mathsf{tx.vin.len}()\) +
        • +
      +
      • +
    • Index uniqueness: +
        +
      • For every + \(\mathsf{output}\) + in + \(\mathsf{pd.saplingOutputs}\) + , + \(\mathsf{output.index}\) + only occurs once.
        • +
      • For every + \(\mathsf{spend}\) + in + \(\mathsf{pd.saplingSpends}\) + , + \(\mathsf{spend.index}\) + only occurs once.
        • +
      • For every + \(\mathsf{input}\) + in + \(\mathsf{pd.transparentInputs}\) + , + \(\mathsf{input.index}\) + only occurs once.
        • +
      +
      • +
    • Index correctness: +
        +
      • For every + \(\mathsf{output}\) + in + \(\mathsf{pd.saplingOutputs}\) + , + \(\mathsf{output.index} < \mathsf{tx.shieldedOutputs.len}()\) +
        • +
      • For every + \(\mathsf{spend}\) + in + \(\mathsf{pd.saplingSpends}\) + , + \(\mathsf{spend.index} < \mathsf{tx.shieldedSpends.len}()\) +
        • +
      • For every + \(\mathsf{input}\) + in + \(\mathsf{pd.transparentInputs}\) + , + \(\mathsf{input.index} < \mathsf{tx.vin.len}()\) +
        • +
      +
      • +
    • + \(\mathsf{pd.saplingSpends.len}() + \mathsf{pd.transparentInputs.len}() > 0\) +
      • +
    +
    • +
  • Let + \(unsignedPaymentDisclosure\) + be the encoding of the payment disclosure without signatures.
    • +
  • Let + \(coinType\) + be the 4-byte little-endian encoding of the coin type in its index form, not its hardened form (i.e. 133 for mainnet Zcash).
    • +
  • Let + \(digest = \mathsf{BLAKE2b}\text{-}\mathsf{256}(\texttt{"ZIP311Signed"}\,||\,coinType, unsignedPaymentDisclosure)\) + .
    • +
  • For every + \(\mathsf{spend}\) + in + \(\mathsf{pd.saplingSpends}\) + : +
      +
    • If + \(\mathsf{SpendAuthSig.Verify}(\mathsf{spend.rk}, digest, \mathsf{spend.spendAuthSig}) = 0\) + , return false.
      • +
    • [Optional] If a payment address proof + \(\mathsf{addrProof}\) + is present in + \(\mathsf{spend}\) + , verify + \((\mathsf{addrProof.nullifier_{addr}}, \mathsf{spend.rk}, \mathsf{addrProof.zkproof_{addr}})\) + as a ZIP 304 proof for + \((\mathsf{addrProof.d}, \mathsf{addrProof.pk_d})\) + 10. If verification fails, return false.
      • +
    • Decode and verify + \(\mathsf{zkproof_{spend}}\) + as a Sapling spend proof 4 with primary input: +
        +
      • + \(\mathsf{tx.shieldedSpends[spend.index].rt}\) +
        • +
      • + \(\mathsf{spend.cv}\) +
        • +
      • + \(\mathsf{tx.shieldedSpends[spend.index].nf}\) +
        • +
      • + \(\mathsf{spend.rk}\) +
        • +
      +

      If verification fails, return false.

      +
      • +
    +
    • +
  • For every + \(\mathsf{input}\) + in + \(\mathsf{pd.transparentInputs}\) + : +
      +
    • TODO: BIP 322 verification.
      • +
    +
    • +
  • For every + \(\mathsf{output}\) + in + \(\mathsf{pd.saplingOutputs}\) + : +
      +
    • Recover the Sapling note in + \(\mathsf{tx.shieldedOutputs}[\mathsf{output.index}]\) + via the process specified in 6 with inputs + \((height, \mathsf{output.ock})\) + . If recovery returns + \(\bot\) + , return false.
      • +
    +
    • +
  • Return true.
    • +
+
+

Payment disclosure validity in UIs

+

TODO: Set some standards for how UIs should display payment disclosures, and how they should convey the various kinds of validity information:

+
    +
  • One, but not all, of the spenders proved spend authority.
    • +
  • All spenders of a specific type proved spend authority.
    • +
  • All spenders proved spend authority.
    • +
  • These, but also including optional payment address proofs.
    • +
+
+
+

Rationale

+

If a sender elects, at transaction creation time, to use an + \(\mathsf{ovk}\) + of + \(\bot\) + for a specific Sapling output, then they are unable to subsequently create a payment disclosure that discloses that output. This maintains the semantics of + \(\mathsf{ovk}\) + , in that the sender explicitly chose to lose the capability to recover that output.

+

Payment disclosures that prove Sapling spend authority are not required to reveal a sender address. This is because it is impossible: we can "prove" the transaction came from any of the diversified addresses linked to the spending key. Fundamentally, the "sender" of a transaction is anyone who has access to the corresponding spend authority; in the case of Sapling, a spend authority corresponds to multiple diversified addresses. In situations where a sender address is already known to the verifier of the payment disclosure (or publically), it may still be useful to have the option of linking the payment disclosure to that address.

+
+

Security and Privacy Considerations

+

When spending Sapling notes normally in transactions, wallets select a recent anchor to make the anonymity set of the spent note as large as possible. By contrast, Sapling spend authority in a payment disclosure is proven using the same anchor that was used in the transaction itself, instead of a recent anchor. We do this for efficency reasons:

+
    +
  • The anchor is already encoded in the transaction, so can be omitted from the payment disclosure encoding.
    • +
  • It is necessary to have a witness for each spent note that is being included in the payment disclosure. Using the same anchor means that the same witness can be used for the transaction spend and the payment disclosure, which in turn means that wallets that support payment disclosures only need to remember that witness, and do not need to continually update witnesses for spent notes in the off-chance that they might be used in a payment disclosure.
    • +
+

There is no privacy benefit to selecting a more recent anchor; the anonymity set of the note was "fixed" by the original spend (which revealed that the note existed as of that anchor's height).

+

We require all payment disclosures to prove spend authority for at least one input, in order to simplify the verification UX. In particular, if payment disclosures without spends were considered valid, an invalid payment disclosure with invalid signatures (that would be shown as invalid by UIs) could be mutated into a payment disclosure that would be shown as valid by UIs, by stripping off the signatures. We do not believe that this prevents any useful use cases; meanwhile if someone is intent on obtaining Sapling output disclosures regardless of the validity of their source, they will do so without a common standard.

+
+

Reference implementation

+

TBD

+
+

References

+ + + + + + + +
1RFC 2119: Key words for use in RFCs to Indicate Requirement Levels
+ + + + + + + +
2RFC 4648: The Base16, Base32, and Base64 Data Encodings
+ + + + + + + +
3Zcash Protocol Specification, Version 2020.1.15 or later
+ + + + + + + +
4Zcash Protocol Specification, Version 2020.1.15. Section 4.15.2: Spend Statement (Sapling)
+ + + + + + + +
5Zcash Protocol Specification, Version 2020.1.15. 4.17.1: Encryption (Sapling)
+ + + + + + + +
6Zcash Protocol Specification, Version 2020.1.15. 4.17.3: Decryption using a Full Viewing Key (Sapling)
+ + + + + + + +
7Zcash Protocol Specification, Version 2020.1.15. Section 5.4.6.1: Spend Authorization Signature
+ + + + + + + +
8BIP 322: Generic Signed Message Format
+ + + + + + + +
9SLIP-0044 : Registered coin types for BIP-0044
+ + + + + + + +
10ZIP 304: Sapling Address Signatures
+
\ No newline at end of file From fa4b1ddcf09174c49c21e42a298ec97d63886c5e Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Mon, 25 Jan 2021 11:04:26 +0000 Subject: [PATCH 016/231] ZIP 311: Use length(a) instead of a.len() --- zip-0311.html | 41 ++++++++++++++++++++++++++--------------- zip-0311.rst | 30 +++++++++++++++--------------- 2 files changed, 41 insertions(+), 30 deletions(-) diff --git a/zip-0311.html b/zip-0311.html index 6ae8faba3..4fd5b260f 100644 --- a/zip-0311.html +++ b/zip-0311.html @@ -175,7 +175,7 @@

System Message: WARNING/2 (zip-0311.rst line 75)

\(b\) .
  • - \(a\text{.len}()\) + \(\mathsf{length}(a)\) means the length of the vector \(a\) .
    • @@ -189,11 +189,14 @@

    System Message: WARNING/2 (zip-0311.rst line 75)

  • msg: A message field, which could contain a challenge value from the party to whom the payment disclosure is directed.
  • \(\mathsf{saplingOutputs}\) - : A vector of the Sapling Outputs that we are disclosing [0..tx.shieldedOutputs.len()] + : A vector of the Sapling Outputs that we are disclosing + \([0..\mathsf{length}(\mathsf{tx.shieldedOutputs})]\)
    • \(\mathsf{index}\) - : An index into tx.shieldedOutputs.
      • + : An index into + \(\mathsf{tx.shieldedOutputs}\) + .
    • \(\mathsf{ock}\) : The outgoing cipher key that allows this output to be recovered. 5
      • @@ -201,11 +204,14 @@

      System Message: WARNING/2 (zip-0311.rst line 75)

    • \(\mathsf{saplingSpends}\) - : A vector of the Sapling Spends for which we are proving spend authority [0..tx.shieldedSpends.len()] + : A vector of the Sapling Spends for which we are proving spend authority + \([0..\mathsf{length}(\mathsf{tx.shieldedSpends})]\)
      • \(\mathsf{index}\) - : An index into tx.shieldedSpends.
        • + : An index into + \(\mathsf{tx.shieldedSpends}\) + .
      • \(\mathsf{cv}\) : A value commitment to the spent note.
        • @@ -237,11 +243,14 @@

        System Message: WARNING/2 (zip-0311.rst line 75)

      • \(\mathsf{transparentInputs}\) - : A vector of the transparent inputs for which we are proving spend authority [0..tx.vin.len()] + : A vector of the transparent inputs for which we are proving spend authority + \([0..\mathsf{length}(\mathsf{tx.vin})]\)
        • \(\mathsf{index}\) - : An index into tx.vin.
          • + : An index into + \(\mathsf{tx.vin}\) + .
        • \(\mathsf{sig}\) : A BIP 322 signature. 8 @@ -282,7 +291,9 @@

          System Message: WARNING/2 (zip-0311.rst line 75)

          • \(\mathsf{index}\) - : An index into tx.vin.
            • + : An index into + \(\mathsf{tx.vin}\) + .
          • The inputs to a BIP 322 signature (excluding message_data).
          • @@ -357,13 +368,13 @@

          System Message: WARNING/2 (zip-0311.rst line 75)

        • Vector length correctness:
          • - \(\mathsf{pd.saplingOutputs.len}() \leq \mathsf{tx.shieldedOutputs.len}()\) + \(\mathsf{length}(\mathsf{pd.saplingOutputs}) \leq \mathsf{length}(\mathsf{tx.shieldedOutputs})\)
          • - \(\mathsf{pd.saplingSpends.len}() \leq \mathsf{tx.shieldedSpends.len}()\) + \(\mathsf{length}(\mathsf{pd.saplingSpends}) \leq \mathsf{length}(\mathsf{tx.shieldedSpends})\)
          • - \(\mathsf{pd.transparentInputs.len}() \leq \mathsf{tx.vin.len}()\) + \(\mathsf{length}(\mathsf{pd.transparentInputs}) \leq \mathsf{length}(\mathsf{tx.vin})\)
          • @@ -399,26 +410,26 @@

          System Message: WARNING/2 (zip-0311.rst line 75)

          in \(\mathsf{pd.saplingOutputs}\) , - \(\mathsf{output.index} < \mathsf{tx.shieldedOutputs.len}()\) + \(\mathsf{output.index} < \mathsf{length}(\mathsf{tx.shieldedOutputs})\)
        • For every \(\mathsf{spend}\) in \(\mathsf{pd.saplingSpends}\) , - \(\mathsf{spend.index} < \mathsf{tx.shieldedSpends.len}()\) + \(\mathsf{spend.index} < \mathsf{length}(\mathsf{tx.shieldedSpends})\)
        • For every \(\mathsf{input}\) in \(\mathsf{pd.transparentInputs}\) , - \(\mathsf{input.index} < \mathsf{tx.vin.len}()\) + \(\mathsf{input.index} < \mathsf{length}(\mathsf{tx.vin})\)
      • - \(\mathsf{pd.saplingSpends.len}() + \mathsf{pd.transparentInputs.len}() > 0\) + \(\mathsf{length}(\mathsf{pd.saplingSpends}) + \mathsf{length}(\mathsf{pd.transparentInputs}) > 0\)
      • diff --git a/zip-0311.rst b/zip-0311.rst index ddc157291..6ff6e103e 100644 --- a/zip-0311.rst +++ b/zip-0311.rst @@ -160,7 +160,7 @@ We also define the following notation here: - :math:`[a..b]` means the sequence of values inclusive of :math:`a` and exclusive of :math:`b`. -- :math:`a\text{.len}()` means the length of the vector :math:`a`. +- :math:`\mathsf{length}(a)` means the length of the vector :math:`a`. Specification @@ -177,16 +177,16 @@ A payment disclosure has the following fields: the payment disclosure is directed. - :math:`\mathsf{saplingOutputs}`: A vector of the Sapling Outputs that we are disclosing - [0..tx.shieldedOutputs.len()] + :math:`[0..\mathsf{length}(\mathsf{tx.shieldedOutputs})]` - - :math:`\mathsf{index}`: An index into `tx.shieldedOutputs`. + - :math:`\mathsf{index}`: An index into :math:`\mathsf{tx.shieldedOutputs}`. - :math:`\mathsf{ock}`: The outgoing cipher key that allows this output to be recovered. [#protocol-saplingencrypt]_ - :math:`\mathsf{saplingSpends}`: A vector of the Sapling Spends for which we are proving - spend authority [0..tx.shieldedSpends.len()] + spend authority :math:`[0..\mathsf{length}(\mathsf{tx.shieldedSpends})]` - - :math:`\mathsf{index}`: An index into `tx.shieldedSpends`. + - :math:`\mathsf{index}`: An index into :math:`\mathsf{tx.shieldedSpends}`. - :math:`\mathsf{cv}`: A value commitment to the spent note. - :math:`\mathsf{rk}`: A randomized public key linked to the spent note. - :math:`\mathsf{zkproof_{spend}}`: A Sapling spend proof. @@ -199,9 +199,9 @@ A payment disclosure has the following fields: - :math:`\mathsf{spendAuthSig}` - :math:`\mathsf{transparentInputs}`: A vector of the transparent inputs for which we are - proving spend authority [0..tx.vin.len()] + proving spend authority :math:`[0..\mathsf{length}(\mathsf{tx.vin})]` - - :math:`\mathsf{index}`: An index into `tx.vin`. + - :math:`\mathsf{index}`: An index into :math:`\mathsf{tx.vin}`. - :math:`\mathsf{sig}`: A BIP 322 signature. [#bip-0322]_ - TODO: `zcashd` currently only supports the legacy format defined in BIP 322. We may @@ -232,7 +232,7 @@ The inputs to a payment disclosure are: - A vector of transparent input tuples (which may be empty) containing: - - :math:`\mathsf{index}`: An index into `tx.vin`. + - :math:`\mathsf{index}`: An index into :math:`\mathsf{tx.vin}`. - The inputs to a BIP 322 signature (excluding `message_data`). The caller MUST provide at least one input tuple of any type (either a Sapling spend tuple @@ -291,9 +291,9 @@ by the caller), the verifier proceeds as follows: - Vector length correctness: - - :math:`\mathsf{pd.saplingOutputs.len}() \leq \mathsf{tx.shieldedOutputs.len}()` - - :math:`\mathsf{pd.saplingSpends.len}() \leq \mathsf{tx.shieldedSpends.len}()` - - :math:`\mathsf{pd.transparentInputs.len}() \leq \mathsf{tx.vin.len}()` + - :math:`\mathsf{length}(\mathsf{pd.saplingOutputs}) \leq \mathsf{length}(\mathsf{tx.shieldedOutputs})` + - :math:`\mathsf{length}(\mathsf{pd.saplingSpends}) \leq \mathsf{length}(\mathsf{tx.shieldedSpends})` + - :math:`\mathsf{length}(\mathsf{pd.transparentInputs}) \leq \mathsf{length}(\mathsf{tx.vin})` - Index uniqueness: @@ -307,13 +307,13 @@ by the caller), the verifier proceeds as follows: - Index correctness: - For every :math:`\mathsf{output}` in :math:`\mathsf{pd.saplingOutputs}`, - :math:`\mathsf{output.index} < \mathsf{tx.shieldedOutputs.len}()` + :math:`\mathsf{output.index} < \mathsf{length}(\mathsf{tx.shieldedOutputs})` - For every :math:`\mathsf{spend}` in :math:`\mathsf{pd.saplingSpends}`, - :math:`\mathsf{spend.index} < \mathsf{tx.shieldedSpends.len}()` + :math:`\mathsf{spend.index} < \mathsf{length}(\mathsf{tx.shieldedSpends})` - For every :math:`\mathsf{input}` in :math:`\mathsf{pd.transparentInputs}`, - :math:`\mathsf{input.index} < \mathsf{tx.vin.len}()` + :math:`\mathsf{input.index} < \mathsf{length}(\mathsf{tx.vin})` - - :math:`\mathsf{pd.saplingSpends.len}() + \mathsf{pd.transparentInputs.len}() > 0` + - :math:`\mathsf{length}(\mathsf{pd.saplingSpends}) + \mathsf{length}(\mathsf{pd.transparentInputs}) > 0` - Let :math:`unsignedPaymentDisclosure` be the encoding of the payment disclosure without signatures. From ff6a98ff652d73ffe597331ea7ba1572520272db Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Mon, 25 Jan 2021 11:05:16 +0000 Subject: [PATCH 017/231] ZIP 311: Replace "vector" with "sequence" --- zip-0311.html | 16 ++++++++-------- zip-0311.rst | 16 ++++++++-------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/zip-0311.html b/zip-0311.html index 4fd5b260f..0564e2e53 100644 --- a/zip-0311.html +++ b/zip-0311.html @@ -176,7 +176,7 @@

      System Message: WARNING/2 (zip-0311.rst line 75)

      .
    • \(\mathsf{length}(a)\) - means the length of the vector + means the length of the sequence \(a\) .
    @@ -189,7 +189,7 @@

    System Message: WARNING/2 (zip-0311.rst line 75)

  • msg: A message field, which could contain a challenge value from the party to whom the payment disclosure is directed.
  • \(\mathsf{saplingOutputs}\) - : A vector of the Sapling Outputs that we are disclosing + : A sequence of the Sapling Outputs that we are disclosing \([0..\mathsf{length}(\mathsf{tx.shieldedOutputs})]\)