all: simple checkpoint syncing

[rjl493456442]

In this PR, we introduce a more flexible approach to update the checkpoint to fasten light client syncing instead of relying on hardcoded one.

Basially the idea is we can deploy a smart contract on the blockchain and put all stable checkpoints in there. Light client can verify the correntness of advertised checkpoint which announced by remote peer through the contract.

For the security consideration, we rely on the elliptic curve cryptography.

Pre-requisites

• Deploy a checkpoint registrar contract on the mainnet.

the code of contract should looks like:

pragma solidity ^0.5.2;

contract Registrar {
    struct Vote {
        address addr;
        bytes   sig;
    }

    struct PendingProposal {
        uint index; // Checkpoint section index
        uint count; // Number of signers who have submitted checkpoint announcement
        mapping(address => bytes32) usermap; // map between signer address and advertised checkpoint hash
        mapping(bytes32 => Vote[]) votemap; // map between checkpoint hash and relative signer announcements.
    }

    modifier OnlyAuthorized() {
        require(admins[msg.sender]);
        _;
    }

    event NewCheckpointEvent(uint indexed index, bytes32 checkpointHash, bytes signature);

    constructor(address[] memory _adminlist, uint _sectionSize, uint _processConfirms, uint _threshold) public {
        for (uint i = 0; i < _adminlist.length; i++) {
            admins[_adminlist[i]] = true;
            adminList.push(_adminlist[i]);
        }
        sectionSize = _sectionSize;
        processConfirms = _processConfirms;
        threshold = _threshold;
    }

    function SetCheckpoint(
        uint _sectionIndex,
        bytes32 _hash,
        bytes memory _sig
    )
    OnlyAuthorized
    public
    returns(bool)
    {
        if (block.number < (_sectionIndex+1)*sectionSize+processConfirms || block.number >= (_sectionIndex+2)*sectionSize) {
            return false;
        }
        // Delete stale pending proposal silently
        if (pending_proposal.index != _sectionIndex) {
            deletePending();
        }
        bytes32 old = pending_proposal.usermap[msg.sender];
        // Filter out duplicate announcement
        if (old == _hash) {
            return false;
        }
        bool isNew = (old == "");
        pending_proposal.usermap[msg.sender] = _hash;

        if (!isNew) {
            // Checkpoint modification
            Vote[] storage votes = pending_proposal.votemap[old];
            for (uint i = 0; i < votes.length - 1; i++) {
                if (votes[i].addr == msg.sender) {
                    votes[i] = votes[votes.length - 1];
                    break;
                }
            }
            delete votes[votes.length-1];
            votes.length -= 1;
            pending_proposal.votemap[_hash].push(Vote({
                addr: msg.sender,
                sig:  _sig
            }));
        } else {
            // New checkpoint announcement
            pending_proposal.count += 1;
            pending_proposal.index = _sectionIndex;
            pending_proposal.votemap[_hash].push(Vote({
                addr: msg.sender,
                sig:  _sig
            }));
        }
        if (pending_proposal.votemap[_hash].length < threshold) {
           return true;
        }
        bytes memory sigs;
        for (uint idx = 0; idx < threshold; idx++) {
            sigs = abi.encodePacked(sigs, pending_proposal.votemap[_hash][idx].sig);
        }
        emit NewCheckpointEvent(_sectionIndex, _hash, sigs);
        deletePending();
        return true;
    }

    PendingProposal pending_proposal;

    mapping(address => bool) admins;
    
    address[] adminList;
    
    uint sectionSize;

    uint processConfirms;

    uint threshold;
}

• Hardcode contract address and trusted signers for different public chains(mainnet, ropsten, rinkeby) in the codebase.
• Register stable checkpoint by trusted signers at a specified rate(32768 blocks).

Besides, we provide a registrar help tool to help operator to register checkpoint correctly.

So all in all, this action just like updating hard code checkpoint before release by core developers, but it is definitely more flexible and same secure as the hardcoded one.

Simple checkpoint syncing

Server side

• Advertise a stable checkpoint(which should generated locally and announced by trusted signers) in the handshake package when shaking with a remote light client.

Note the persisted checkpoint should containes two things:

1. Checkpoint(SectionIndex,SectionHead, ChtRoot, BloomRoot);
2. The block number corresponding to the new checkpoint event;

Light client side

• Select a server peer which has the highest advertised total difficulty as the target peer.
• If there is no checkpoint provided by target peer(e.g some customized chain doesn't have the registrar contract deployed and no hardcoded checkpoint), use normal light syncing to fetch all headers from the genesis.
• If the checkpoint syncing is not activated, use normal light syncing to fetch all headers from the genesis.
• Validate the checkpoint annouced by target peer:
  • Fetch the corresponding block header where checkpoint is registered.
  • Fetch the logs of the block.
  • Verify the correctness of checkpoint through the checkpoint signature.
• Request the last block header which covered by server peer's announced stable checkpoint and set as the current head header
  • For the ethash consensus engine, the start header is the block header of the checkpoint
  • For the clique consensus engine, the start header is the block header of the latest epoch covered by checkpoint.
• Sync the last a few thousands blocks by light syncing

Admin side

• Sign for checkpoint with the private key and register checkpoint after (index+1)*32768+256 blocks;
• Can modify the checkpoint in the block range [(index+1)*32768+256, (index+2)*32768];

Security model

For the simple checkpoint syncing, the security is guaranteed by ECDSA.

1. Contract address and initial target admin addresses are hardcoded in the codebase.
2. Light client only accept checkpoint announcement given by trusted signers.
3. If we can find a checkpoint equal with the advertised checkpoint, it means the advertised checkpoint is valid and we can sync based on that just like a new genesis block.

Weakness

1. Admins private key files are the heart of security.

TODO

• Deploy contact on the rinkeby testnet
• Deploy contact on the ropsten testnet
• Deploy contact on the goerli testnet

[rjl493456442]

@zsfelfoldi Btw I have tested the latest PR by connecting the client and server and running a light syncing based on the checkpoint registered in the contract, it works.

[rjl493456442]

@zsfelfoldi I remove the annoying callback stuff and move the backend setting logic after the node start. Hope it makes sense to you :))

[zsfelfoldi]

PR is approved, please resolve merge conflicts and fix the linter warnings:
cmd/registrar/common.go:36:2:warning: unused variable or constant errInvalidURL (varcheck)
cmd/registrar/common.go:37:2:warning: unused variable or constant errNoSpecifiedAddr (varcheck)

Note: the registrar contract seems fine to me but we need a more careful security examination before deploying it on the mainnet. In the meantime we can test it on Rinkeby.

[karalabe]

So, we've been thinking that there's no real reason (correct us if wrong) to maintain a set of old checkpoints. We only ever care about the latest. That restriction would probably reduce the contract complexity a bit.

[karalabe]

Wondering if this is worth the complexity.

[karalabe]

We had this idea for reorg protection: When voting, also pass in the current block number and hash (or maybe one from 10 blocks). That way if there's a reorg, the votes will fail on that "chain" and we don't serve up bad CHTs, rather revert to an older set one.

[karalabe]

Swap so guy with the F is happy :D

[holiman]

Please take a look at https://eips.ethereum.org/EIPS/eip-191 , see if we can use that prefix scheme for the signatures

[holiman]

Would it be possible to simplify the whole thing, contract-wise, with some variation of this:

// SetCheckpoint sets  a new checkpoint. It accepts a list of signatures
// @_recentNumber: a recent blocknumber, for replay protection
// @_recentHash : the hash of `_recentNumber`
// @_hash : the hash to set at _sectionIndex
// @_sectionIndex : the section index to set
// @v : the list of v-values
// @r : the list or r-values
// @s : the list of s-values 
function SetCheckpoint(uint _recentNumber, 
	bytes32 _recentHash, 
	bytes32 _hash, 
	uint _sectionIndex , 
	uint8 []v, bytes32 []r, bytes32 []s)
	public
	returns (bytes32 transactionHash)
{

	// This checks replay protection, so it cannot be replayed on forks,
	// accidentally or intentionally
    require(admins[msg.sender]);
    require(blockhash(_recentNumber) == _recentHash);
    
    assert(v.length == r.length)
    assert(v.length == s.length)

	// Filter out "future" checkpoint.
	if (block.number < (_sectionIndex+1)*sectionSize+processConfirms) {
		return false;
	}
	// Filter out "old" announcement
	if (_sectionIndex < sectionIndex) {
		return false;
	}
	// Filter out "stale" announcement
	if (_sectionIndex == sectionIndex && (_sectionIndex != 0 || height != 0)) {
		return false;
	}
	// Filter out "invalid" announcement
	if (_hash == ""){
		return false
	}
	

	// EIP 191 style signatures

	// Arguments when calculating hash to validate
	// 1: byte(0x19) - the initial 0x19 byte
	// 2: byte(0) - the version byte (data with intended validator) 
	// 3: this - the validator address
	// 4,5 : Application specific data

	signedHash = keccak256(byte(0x19),byte(0),this, _hash, _sectionIndex);

	// Tally up the votes
	int memory votes = 0
	mapping(address => bool) memory voted

	for (uint idx = 0; idx < v.length; idx++){
		var signer = ecrecover(signedHash, v[idx], r[idx], s[idx]);
		if admins[signer] && !voted[signer]{
			votes ++
			if votes >= threshold{
				// Sufficient signatures present
						// Update latest checkpoint
				hash = _hash;
				height = block.number;
				sectionIndex = _sectionIndex;
				// TODO! Build the 'sigs' struct
				emit NewCheckpointEvent(_sectionIndex, _hash, sigs);

			}
		}
	}
}

The code above has the following differences:

• No concept of pending, it instead takes all signatures in one go,
• Thus no need to remember a lot of state, and clear pending etc,
• Only one transaction is needed to set the checkpoint,
• The contract verifies that the signatures are valid

If we decide that we don't need the replay protection, we can even make the method totally public, so it doesn't even require admins to call it -- only to sign a eip-191 data structure and send to the submitter.

[holiman]

To clarify, the EIP 191 version 0 is 'data with intended validator'. In this case, the intended validator is the contract at the specified address. After that, the rest of the data is in whatever form the validator is hardcoded to accept.
So this means that the signatures are valid for consumption by that contract and that contract only, which gives us cross-network replay protection (if we so desire).

[holiman]

That would give us a concrete way to try out the UI of clef. I imagine the UX should be something like

Do you want to sign EIP-191 formatted data:
- EIP 191 version: 0
- Intended validator [address]: 0xdeax...ff 
- Application-specific data : 0x00000000012deadbeefdeadbeef

[holiman]

(though, miight be nicer to use bytes65 []sigs as the parameter)

[holiman]

Here's a more complete version, which I think is a lot simpler than the one in this PR.

pragma solidity ^0.5.2;

/**
 * @title Registrar
 * @author Gary Rong<garyrong@ethereum.org>, Martin Swende <martin.swende@ethereum.org>
 * @dev Implementation of the blockchain checkpoint registrar.
 */
contract Registrar {
    
    /*
        Fields
    */
    // A map of admin users who have the permission to update CHT and bloom Trie root
    mapping(address => bool) admins;

    // Latest stored section id
    uint sectionIndex;

    // The block height associated with latest registered checkpoint.
    uint height;

    // The hash of latest registered checkpoint.
    bytes32 hash;

    // The frequency for creating a checkpoint
    //
    // The default value should be the same as the checkpoint size(32768) in the ethereum.
    uint8 sectionSize;

    // The number of confirmations needed before a checkpoint can be registered.
    // We have to make sure the checkpoint registered will not be invalid due to
    // chain reorg.
    //
    // The default value should be the same as the checkpoint process confirmations(256) 
    // in the ethereum.
    uint processConfirms;
    
    // The required signatures to finalize a stable checkpoint.
    uint8 threshold;
    /*
        Events
    */

    // NewCheckpointVote is emitted when a new checkpoint proposal receives a vote.
    event NewCheckpointVote(uint8 indexed index, bytes32 checkpointHash, uint8 v, bytes32 r, bytes32 s);

    /*
        Public Functions
    */
    constructor(address[] memory _adminlist, uint8 _sectionSize, uint _processConfirms, uint8 _threshold) public {
        for (uint i = 0; i < _adminlist.length; i++) {
            admins[_adminlist[i]] = true;
        }
        sectionSize = _sectionSize;
        processConfirms = _processConfirms;
        threshold = _threshold;
    }
	
    // SetCheckpoint sets  a new checkpoint. It accepts a list of signatures
    // @_recentNumber: a recent blocknumber, for replay protection
    // @_recentHash : the hash of `_recentNumber`
    // @_hash : the hash to set at _sectionIndex
    // @_sectionIndex : the section index to set
    // @v : the list of v-values
    // @r : the list or r-values
    // @s : the list of s-values 
    function SetCheckpoint(uint _recentNumber, 
    	bytes32 _recentHash, 
    	bytes32 _hash, 
    	uint8 _sectionIndex ,
    	uint8[] memory v, 
    	bytes32[] memory r, 
    	bytes32[] memory s)
    	public
    	returns (bool)
    {
    
    	// These checks replay protection, so it cannot be replayed on forks,
    	// accidentally or intentionally
        require(admins[msg.sender]);
        require(blockhash(_recentNumber) == _recentHash);
        
        assert(v.length == r.length);
        assert(v.length == s.length);
        
    	// Filter out "future" checkpoint.
    	if (block.number < (_sectionIndex+1)*sectionSize+processConfirms) {
    		return false;
    	}
    	// Filter out "old" announcement
    	if (_sectionIndex < sectionIndex) {
    		return false;
    	}
    	// Filter out "stale" announcement
    	if (_sectionIndex == sectionIndex && (_sectionIndex != 0 || height != 0)) {
    		return false;
    	}
    	// Filter out "invalid" announcement
    	if (_hash == ""){
    		return false;
    	}
    	
    	// EIP 191 style signatures
   		//
    	// Arguments when calculating hash to validate
    	// 1: byte(0x19) - the initial 0x19 byte
    	// 2: byte(0) - the version byte (data with intended validator) 
    	// 3: this - the validator address
    	// --  Application specific data
    	// 4 : section index (uint8)
    	// 5 : hash (bytes32)
    
    	bytes32 signedHash = keccak256(abi.encodePacked(byte(0x19),byte(0),this,_sectionIndex, _hash ));

    	// Tally up the votes
    	address lastVoter = address(0);
    	// In order for us not to have to maintain a mapping
    	// of who has already voted, and we don't want to count a 
    	// vote twice, the signatures must be submitted in 
    	// strict ordering
    	for (uint idx = 0; idx < v.length; idx++){
    		address signer = ecrecover(signedHash, v[idx], r[idx], s[idx]);
    		require( admins[signer] );
    		require( uint256(signer) >= uint256(lastVoter));
    		lastVoter = signer;
    		emit NewCheckpointVote(_sectionIndex, _hash, v[idx], r[idx], s[idx]);
    		if (idx+1 >= threshold){
    			// Sufficient signatures present
    			// Update latest checkpoint
    			hash = _hash;
    			height = block.number;
    			sectionIndex = _sectionIndex;
                return true;
    		}
    	}
    	// We shouldn't wind up here, reverting un-emits the events
        revert();
    }
}

A couple of things to note:

• It requires the callers to collect signatures, and sort them in ascending order before sending to the contract.
• There is very minimal state in the contract,
• The contract emits one event per successful vote, but if the votes do not tally up, then the call revert()s, so the events are thus never actually emitted on bad data.
• I tested it with the EIP-191 data in accounts,signer: better support for EIP-191 intended validator #19523 , seems to work correctly.