Rework of wallet output and transaction state update process

[yeastplume]

Placeholder issue for definition of how wallets should update themselves from a node. We should take the opportunity to finalize 'best practices' here, and define a process that keeps the wallet and related transactions in sync as well as possible.

Right now, the process is roughly (will update with more detail as needed):

Coinbase

• Create a potential coinbase output for a mined block
• If the block is accepted the output is detected on the chain, create a transaction log entry of type 'Confirmed Coinbase', with 'confirmed' set to true

Transacting (payer -> payee)

• The sender creates a new 'blank' transaction Slate, adding inputs and change outputs to the slate.
• The sender generates a random kernel offset
• The sender sends the slate to the payee (via file or http). If sending synchronously, (e.g via http) the associated transaction is saved to the log after a response from the payee's listener. If sending asynchronously, (e.g. via file), the transaction is saved immediately.
• When saved, at this stage, the associated transaction is set to type 'Sent Tx' with a status of 'unconfirmed', inputs are locked internally and a change output is added with status 'unconfirmed'.
• The payee receives the slate, creates (an) output(s) for the received amount with status unconfirmed, and immediately stores a transaction in their log of type 'Received Tx' with confirmed set to false.
• The payee calculates and saves the transaction kernel for later reference.
• The slate is returned to the payer for completion, who calculates and saves the transaction kernel.

(Invoicing is similar to the above, with the roles of payer/payee and sender/receiver swapped, so I won't detail it separately here)

Updating Transaction/Output State

• On every info/send/txs command, query all 'unconfirmed' outputs in the user's wallet against the node.
• If an 'unconfirmed' output in the user's wallet appears on the chain, change the status to 'confirmed', and update the associated transaction status to 'confirmed'.
• If the transaction has no change outputs and the wallet is the payer, also perform a kernel lookup when performing a 'txs' command.

Check-Repair

Check-Repair currently fixes most of the potential issues that come up via the current updating model, namely:

• Removing transactions that have never confirmed, setting their state to cancelled, unlocking and/or deleting associated outputs as needed.
• Adding outputs that exist on the chain but for some reason or another are not on the chain, creating transactions for them
• Removing outputs that may exist as 'Unspent' but don't exist on the chain (via a fork, for instance)
• Note this process is identical to the restore process

At the moment, this is a manual process that needs to be initiated by the user.

[yeastplume]

So, random thoughts here:

• Does this need an RFC? (I'd probably say yes since we're defining best practice and this changes the user/developer experience)
• We likely want to include the current 'check-repair' functionality as part of the general update process, rough thoughts on how this would work
  • On wallet init, scan (check-repair) the entire chain for outputs (or allow the user to skip this step?)
  • Keep track of the last height scanned locally
  • Periodically run the scan process from the last block scanned - a threshold
  • Add an 'expiry height' to a transaction, after which the scanning process will cancel transactions and remove/unlock associated outputs.
  • Do we want to change the default selection method to 'smallest' so users can have outstanding transactions without locking all their inputs?
  • Do we want to run this on a timed thread?
• We should change the update process to work via transaction kernel lookup instead (far fewer hits on the server overall). Outputs will be double-checked during the scan process
  • We should provide a mechanism to allow users to manually perform a full chain scan, or provide the number of blocks to scan back.
• It may also be desirable to implement a global file lock to preserve atomicity of wallet operations in a database-independent way.

[yeastplume]

Done in #253, #261, #273