[all] Introduction of AppRegistry and significant code improvements to facilitate it

[snario]

Introduction

This pull request includes several changes to the codebase, all spawned originally from the introduction of a new contract: AppRegistry.sol. This contract entirely replaces AppInstance.sol. The motivation for this change was to remove the necessity to have to deploy a contract in the event of a protocol breakdown (i.e. a "dispute"). Instead, they simply submit a transaction to the global singletonAppRegistry contract with some metadata pertaining to the particular AppInstance they are referring to.

NOTE: I have written a small glossary at the bottom of this document to help articulate the differences between some core terms.

Core Concept Changes

Updated machine middleware signature

Middle ware now takes three arguments:

• message: A ProtocolMessage object
• context: An object that persists through execution of a flow that stores "remembered" data from prior middleware execution including a stateChannel object, a commitment object, and potentially a signatures object
• stateChannel: The state channel that was initially passed into the execution of the middleware. The idea is that the execution of a protocol is intended to transform this StateChannel into a new version of a StateChannel representative of the updates the protocol made to the object

You can see how STATE_TRANSITION_PROPOSE is implemented for the Setup Protocol for example:

monorepo/packages/machine/src/protocol/setup.ts

Lines 62 to 69 in 578a7e2

	function proposeStateTransition(
	message: ProtocolMessage,
	context: Context,
	state: StateChannel
	) {
	context.stateChannel = state.setupChannel(context.network);
	context.operation = constructSetupOp(context.network, context.stateChannel);
	}

cfaddress no longer exists

The notion of a cfaddress is now gone. Replacing it is the unique AppInstanceId which is described in the glossary below. In cf.js I did re-implement the cfAddress() method to compute the AppInstanceId instead, but I did not change the public API.

Introduction of AppRegistry

The actual AppRegistry.sol file is pretty simple because it uses a "mixin" architecture:

monorepo/packages/contracts/contracts/AppRegistry.sol

Lines 12 to 22 in 578a7e2

	contract AppRegistry is
	MixinAppRegistryCore,
	MixinSetState,
	MixinSetStateWithAction,
	MixinCancelChallenge,
	MixinSetResolution
	{
	constructor ()
	public
	{}
	}

Each mixin implements a public facing method of the AppRegistry contract.

• MixinAppRegistryCore implements some helper external / accessor methods:

  • getAppChallenge(bytes32 _id) returns (LibStateChannelApp.AppChallenge)
  • isStateFinalized(bytes32 _id) returns (bool)
  • getResolution(bytes32 _id) returns (Transfer.Transaction)

• MixinSetState implements the setState method

• MixinSetStateWithAction implements the setStateWithAction method

• MixinSetResolution implements the setResolution method

• MixinCancelChallenge implements the cancelChallenge method

Naming Changes

CfOperations

I renamed CfOperations to "commitments". This is what they are as described in the specs. They are individual transactions which could be sent to the blockchain that have a hashToSign(). The folder these things now live in is under the ethereum folder in the machine package since these particular commitments are coupled to the Ethereum blockchain. Here is the InstallCommitment for example and the specifications that you can cross-reference for implementation correctness.

PeerBalance

This was a class that basically described a tuple of (a) "who" and (b) "how much" abstractly. It was re-used for install sub-deposits and uninstall resolution proposals. Hidden in its obscure description and name however was the fact that this thing hard-coded Wei balance changes to the ETH Free Balance of the underlying state channel. So, I've gotten rid of its usage and replaced it with arguments to the install protocol and uninstall protocol that are far more explicit about their usage: aliceBalanceIncrement and aliceBalanceDecrement (and the Bob counterparts). You can see usage here.

Pull Request Changes

I tried to order this in order of importance and relevance...

Introduction of StateChannel and AppInstance models

We were desperately needing clearly defined immutable data structures that represent a state channel and an app instance. This is what these were added for. They replace StateChannelInfoImpl which was an attempt at this object and the sometimes used legacy.AppInstance.

As alluded to above, these classes have been implemented to be immutable meaning all methods on the classes return a new instance of the class with some values changed. This makes the implementation details of the machine much easier to reason about in my opinion.

Note these classes do not use immutable.js yet although they would benefit from it

Recommendation: Read the tests!

Removing the global ganache-cli

I felt like we were unnecessarily coupling the test environments of the packages by requiring there to be a global ganache-cli blockchain running for all of the tests. Instead, I've separated out the execution of the ganache-cli blockchain to be local to each package. So, the contracts repo spins its own instance up, runs migrations, runs tests, spins it down. See here.

The apps package

I moved out some contract code for apps like TicTacToe, Nim, etc into a new package called apps which just is a collection of random apps. For this repo I used a new tool that replaces Truffle for testing called Waffle. Using it made the tests way cleaner since it is based on ethers and they run really fast (like 1 second).

I tried using Waffle on the main contracts repo too but hit some bugs. I have a branch off this PR that implements it. I'm waiting on Waffle to mature more for that PR but for the apps repo the tests are basic enough that we don't encounter any issues.

Significantly better testing

Instead of constructing CfOperations / Commitments and then checking if the hashToSign() method returned a string that was equal to the manually constructed version of it in a test file, now we construct the actual commitment (the transaction) and then decode the calldata piece by piece and check that it is equal to the expected function call to a contract. It is easy to now read a file like the install commitment test file and cross-reference it with the specifications of the install protocol to verify its correctness.

Code Cleanliness

Remove all use of legacy from machine

I was getting really frustrated with this bizarre dependency so I removed every use of anything to do with cf.js/legacy methods and defined my own replacements to all of them.

The @counterfactual/types package

I added a new package to store some types I constantly was reusing. I also added a file in the machine called protocol-types-tbd. Both of these probably ought to be merged in some way with the common-types package that @mvanderh added.

Stop manually specifying test files in yarn test for contracts

As @Alonski found out, it is frustrating to have to specify the names of the files you are testing. As you can see here I have removed this package.json config entirely and now everything "just works" because the tsc build process actually places the built files inside the test/ directory instead of creating a new dist/ subdirectory. This is where the truffle test expects the files to be. There is no downside to this as far as I can tell.

Stop relying on Truffle migrations

I have the opinion that it is bad practice for the test files of the contracts repo to rely on the migrations being run for them to work (e.g, using .deployed() method on TruffleContract) and that, instead, we should deploy and manually link every contract inline in the test. This de-couples the tests from Truffle's migrations framework which I think is good practice.

Proper use of type, interface, and class

We had been using class to define too many kinds of things which has no business being classes. Here is a classic example of that. Instead I followed this rule:

• Use type when defining the structure of some kind of data structure that will be passed around
• Use interface when defining the structure of an object that is required on a public API method
• Use class when there is a requirement for the thing to be uniquely identifiable and persistent

Use ethereum-waffle for chai plugins

We used to use three separate plugins chai-bignumber, chai-as-promised, chai-string or something like that in order to add some brevity to our test files with custom matchers like .to.be.bignumber.eq. It turns out the ethereum-waffle project implemented many of the same things and exported their own chai matcher which we now solely re-use.

Remove AbstractContract

Since we no longer really ever need to do any linking any place except test code (and even then it is entirely optional because Truffle could do it by itself as described above) the AbstractContract class and its associated utilities were mostly just complicating the code in a way that I felt was unhelpful.

Importing ethers utils and constants as modules

Pretty much everywhere I've stopped importing ethers itself and instead import the utility methods and constants individually. For example,

import { hexlify, getAddress, randomBytes } from "ethers/utils";
import { Zero, AddressZero, HashZero } from "ethers/constants";

I think it really helped clean the code up.

Replace rm -rf <hard coded folders> with git clean -Xdf

For the yarn clean command we relied on manually selecting folders to delete when executing the command, but I think using gits built in clean command achieves the same thing automatically.

Glossary

AppInterface

The equivalent of a contract "ABI" for an off-chain application on Counterfactual.

Properties

• addr: The address of an on-chain contract where the contract with the following interface is deployed.

• applyAction, resolve, isStateTerminal, getTurnTaker: Four bytes4 sighashes which are used by the AppRegistry to make staticcall functions to the address of the contract. It is very important to understand that this is solely needed because the contracts use a "hack" to automatically decode ABIEncoderV2 encoded structs as bytes data types in the calldata of function calls. With Solidity 0.5, these fields are no longer necessary because of the introduction of abi.decode.

• stateEncoding: A string representation of the data type that represents the state. For example, tuple(address foo, bytes32 bar).

• actionEncoding: A string representation of the data types that represents an action on the state. This can be undefined for applications that do not define an applyAction method.

Terms

An object that encodes metadata regarding the amount of blockchain state (e.g., funds) that an application has control over in the context of the Counterfactual framework.

Properties

• assetType: An enum representing some well-established asset types e.g., ETH, ERC20

• limit: An integer representing some notion of the amount of that asset class being controlled by this application.

• token: If ERC20, the address of the token on-chain that represents the token.

AppIdentity

An object which represents a unique pointer to an off-chain application in a particular state channel.

Properties

• owner / multisigAddress: The address of an on-chain contract for which, if a transaction is submitted to the blockchain to update the state of this AppInstance where msg.sender == owner then it is considered an "authorized" transaction.

• signingKeys: An array of addresses for which, if msg.sender is not the owner, then any transaction submitted to the blockchain that updates the state of the AppInstance must have also a signatures argument which includes a copy of a signature from every key in signingKeys to be considered "authorized".

• appInterfaceHash: The hash of an AppInterface object.

• terms: The hash of a Terms object.

• defaultTimeout: The number of blocks by default (it can be overrided) that the AppRegistry will wait for newer state in the case where a "dispute" is triggered.

• id(): A computed property, also known as "AppInstanceId" that is computed as keccak256(abi.encodePacked(owner, signingKeys, appInterfaceHash, terms, defaultTimeout)).

AppInstance

An instance of an off-chain application in a particular state channel.

Properties

• latestState: The latest state of the AppInstance in the channel e.g., { foo: 0x, bar: 0x }

• latestNonce: The latest nonce / version number of this state

• latestTimeout: The latest agreed upon timeout for which this state would have on-chain (defaults to identity.defaultTimeout.

• appSeqNo: The sequence number / global unique nonce inside the channel for this AppInstance

TODO: Not implemented yet:

• rootNonceDependency: The value of the root nonce dependency for this app

• uninstallKey(): The computed "key" that would be used for the NonceRegistry that a conditional transaction would rely on not having been set to "UNINSTALLED" (i.e. 1). Computed as keccak256(appSeqNo).

• identity(): An AppIdentity object, as described above.

• id(): The computed id() method of the identity above.

StateChannel

An instance of a state channel.

Properties

• multisigAddress: The on-chain address of a single multisignature wallet that stores the funds.

• multisigOwners: The addresses of the public keys which are the owners of the multisig wallet.

• appInstances: A mapping of AppInstance.ids to AppInstances that are currently active

• freeBalanceIndexes: A mapping from AssetType to the unique AppInstance.id in appInstances

TODO: Need a better name:

• monotonicallyIncreasingSeqNo: A monotonically increasing number representing the number of apps ever installed all-time.

TODO: Not implemented yet:

• rootNonceNumber: The value of the root nonce at the present

[snario]

Tests currently pass for me with the recent changes but there are some caveats...

• Haven't really carefully evaluated security properties of this approach yet, still TODO
• Hard-coded in a 1337 timeout versus using a variable in createDispute because of a stack too deep error. Need to find a workaround still.

Still would appreciate any extra feedback @IIIIllllIIIIllllIIIIllllIIIIllllIIIIll

[ldct]

-1 on the name AppInstanceAdjudicator; maybe AppInstanceRegistry?

[ldct]

LGTM other than existing comments and the FIXME

[mitchelljustin]

@IIIIllllIIIIllllIIIIllllIIIIllllIIIIll I prefer "adjudicator" to "registry" tbh. It describes the raison d'etre of the contract, which is to adjudicate disputes on-chain.

[ldct]

@mvanderh that explanation doesn't support the Instance in the name though

[mitchelljustin]

@IIIIllllIIIIllllIIIIllllIIIIllllIIIIll Why not? The contract adjudicates app instances.

[cf19drofxots]

@snario Status update?

[snario]

@snario Status update?

In progress.

[ebryn]

It doesn't feel right to me that we return new instances of StateChannel objects from each of these methods. It seems like the StateChannel object reference should be stable across all these operations.

[snario]

Do you mean that the class should not be immutable or that the implementation should hide the fact that it is immutable?

[ebryn]

@snario It depends. It could just be a naming issue. Are these objects purely internal to the machine or are they vended out to the Node / Playground Server as well?

[snario]

At the moment, purely internal. However, the intention is to use them in the Node. It is unclear how they will be distributed. My recommendation is to make the machine a subfolder of Node and hoist the models folder up into the node source code

[ebryn]

@snario Then I think they should probably be like how CF.js's AppInstance objects work in that state channels are singletons. API consumers only see one consistent state channel object.

stale