Update develop branch with latest commits from 2.1.0

.github/ISSUE_TEMPLATE.md

@@ -2,9 +2,9 @@
 
 This issue tracker is only for technical issues related to bitcoin-core.
 
-General bitcoin questions and/or support requests and are best directed to the [Bitcoin StackExchange](https://bitcoin.stackexchange.com).
+General ravencoin questions and/or support requests and are best directed to the [Ravencoin Discord](https://discord.gg/GwtXdyc).
 
-For reporting security issues, please read instructions at [https://bitcoincore.org/en/contact/](https://bitcoincore.org/en/contact/).
+For reporting security issues, please direct message one of the core developers in discord.
 
 ### Describe the issue
 

.travis.yml

@@ -18,7 +18,6 @@ env:
     - CCACHE_COMPRESS=1
     - BASE_OUTDIR=$TRAVIS_BUILD_DIR/out
     - SDK_URL=https://ravencoin.org/depends-sources/sdks
-    - PYTHON_DEBUG=1
     - WINEDEBUG=fixme-all
   matrix:
 # ARM
@@ -75,7 +74,7 @@ script:
     - export LD_LIBRARY_PATH=$TRAVIS_BUILD_DIR/depends/$HOST/lib
     - if [ "$RUN_TESTS" = "true" ]; then travis_wait 30 make $MAKEJOBS check VERBOSE=1; fi
     - if [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then extended="--extended --exclude pruning,dbcrash"; fi
-    - if [ "$RUN_TESTS" = "true" ]; then test/functional/test_runner.py --coverage --quiet ${extended}; fi
+    - if [ "$RUN_TESTS" = "true" ]; then test/functional/test_runner.py --combinedlogslen=4000 --coverage --quiet ${extended}; fi
 after_script:
     - echo $TRAVIS_COMMIT_RANGE
     - echo $TRAVIS_COMMIT_LOG

CONTRIBUTING.md

@@ -76,7 +76,7 @@ the pull request affects. Valid areas as:
 
 Examples:
 
-    Consensus: Add new opcode for BIP-XXXX OP_CHECKAWESOMESIG
+    Consensus: Add new opcode for RIP-XXXX OP_CHECKAWESOMESIG
     Net: Automatically create hidden service, listen on Tor
     Qt: Add feed bump button
     Trivial: Fix typo in init.cpp
@@ -193,7 +193,7 @@ In general, all pull requests must:
 
 Patches that change Raven consensus rules are considerably more involved than
 normal because they affect the entire ecosystem and so must be preceded by
-extensive mailing list discussions and have a numbered BIP. While each case will
+extensive mailing list discussions and have a numbered RIP. While each case will
 be different, one should be prepared to expend more time and effort than for
 other kinds of patches because of increased peer review and consensus building
 requirements.
@@ -234,7 +234,7 @@ of consensus critical code.
 
 Where a patch set proposes to change the Raven consensus, it must have been
 discussed extensively on the mailing list and IRC, be accompanied by a widely
-discussed BIP and have a generally widely perceived technical consensus of being
+discussed RIP and have a generally widely perceived technical consensus of being
 a worthwhile change based on the judgement of the maintainers.
 
 ### Finding Reviewers

README.md

@@ -101,5 +101,5 @@ Bitcoin is and always should be focused on its goals of being a better form of m
 
 In the new global economy, borders and jurisdictions will be less relevant as more assets are tradable and trade across borders is increasingly frictionless. In an age where people can move significant amounts of wealth instantly using Bitcoin, global consumers will likely demand the same efficiency for their securities and similar asset holdings.
 
-For such a global system to work it will need to be independent of regulatory jurisdictions.  This is not due to ideological belief but practicality: if the rails for blockchain asset transfer are not censorship resistance and jurisdiction agnostic, any given jurisdiction may be in conflict with another.  In legacy systems, wealth was generally confined in the jurisdiction of the holder and therefor easy to control based on the policies of that jurisdiction. Because of the global nature of blockchain technology any protocol level ability to control wealth would potentially place jurisdictions in conflict and will not be able to operate fairly.  
+For such a global system to work it will need to be independent of regulatory jurisdictions.  This is not due to ideological belief but practicality: if the rails for blockchain asset transfer are not censorship resistance and jurisdiction agnostic, any given jurisdiction may be in conflict with another.  In legacy systems, wealth was generally confined in the jurisdiction of the holder and therefore easy to control based on the policies of that jurisdiction. Because of the global nature of blockchain technology any protocol level ability to control wealth would potentially place jurisdictions in conflict and will not be able to operate fairly.  
 

assets/tools/README.md

@@ -0,0 +1,24 @@
+## Tools for Asset Issuance
+
+### Bulk Issuance
+Issue assets from a .csv file.
+* Make a copy of https://docs.google.com/spreadsheets/d/1Ym88-ggbw8yiMgVxOtVYDsCXJGNGZqlpOfgdbVK8iYU
+* Edit your own data
+* Download as .csv and put in this folder.
+* ```python issuebulk.py```
+
+### Signed Promises
+Check for assets that have signed documents.
+* Set the constants at the top of signed_promises.py
+* ```python signed_promises.py```
+
+### Block Facts
+Loops through blocks and prints out block information.
+* Set the constants at the top of blockfacts.py
+* ```python blockfacts.py```
+
+### Transaction Facts
+Loops through blocks and transactions and prints out tx information.
+* Uncomment out print lines to print out more facts
+* Set the constants at the top of txfacts.py
+* ```python txfacts.py```

assets/tools/asset_audit.py

@@ -0,0 +1,101 @@
+#!/usr/bin/env python3
+# Script to audit the assets
+# Reads the asset (amount has all issuances)
+# Reads the balances in every address for the asset.
+# Compares the two numbers to checks that qty of all assets are accounted for
+
+import subprocess
+import json
+
+
+#Set this to your raven-cli program
+cli = "raven-cli"
+
+mode =  "-testnet"
+rpc_port = 18766
+#mode =  "-regtest"
+#rpc_port = 18443
+
+#Set this information in your raven.conf file (in datadir, not testnet3)
+rpc_user = 'rpcuser'
+rpc_pass = 'rpcpass555'
+
+def listassets(filter):
+    rpc_connection = get_rpc_connection()
+    result = rpc_connection.listassets(filter, True)
+    return(result) 
+
+def listaddressesbyasset(asset):
+    rpc_connection = get_rpc_connection()
+    result = rpc_connection.listaddressesbyasset(asset)
+    return(result) 
+
+def rpc_call(params):
+    process = subprocess.Popen([cli, mode, params], stdout=subprocess.PIPE)
+    out, err = process.communicate()
+    return(out)
+
+def generate_blocks(n):
+    rpc_connection = get_rpc_connection()
+    hashes = rpc_connection.generate(n)
+    return(hashes)
+
+def get_rpc_connection():
+    from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException
+    connection = "http://%s:%[email protected]:%s"%(rpc_user, rpc_pass, rpc_port)
+    #print("Connection: " + connection)
+    rpc_connection = AuthServiceProxy(connection)
+    return(rpc_connection)
+
+def audit(filter):
+    assets = listassets(filter)
+    print("Auditing: " + filter)
+    #print(assets)
+    print("Asset count: " + str(len(assets)))
+    count = 0  
+    max_dist_asset_name = ""
+    max_dist_address_count = 0
+    for asset, properties in assets.items():
+        count=count+1
+        total_issued = 0
+        total_for_asset = 0
+
+        print("Auditing asset (" + str(count) + "): " + asset)
+        for key, value in properties.items():
+            if (key == 'amount'):
+                total_issued += value
+                print("Total issued for " + asset + " is: " + str(value))
+        address_qtys = listaddressesbyasset(asset)
+
+        address_count = 0
+        for address, qty in address_qtys.items():
+            address_count = address_count + 1
+            print(address + " -> " + str(qty))
+            total_for_asset += qty
+
+        print("Total in addresses for asset " + asset + " is " + str(total_for_asset))
+
+        #Calculate stats
+        if address_count > max_dist_address_count:
+            max_dist_asset_name = asset
+            max_dist_address_count = address_count
+
+        if (total_issued == total_for_asset):
+            print("Audit PASSED for " + asset)
+            print("")
+        else:
+            print("Audit FAILED for " + asset)
+            exit()
+
+        if len(assets) == count:
+            print("All " + str(len(assets)) + " assets audited.")
+            print("Stats:")
+            print("  Max Distribed Asset: " + max_dist_asset_name + " with " + str(max_dist_address_count) + " addresses.")
+
+
+
+if mode == "-regtest":  #If regtest then mine our own blocks
+    import os
+    os.system(cli + " " + mode + " generate 400")
+
+audit("*")  #Set to "*" for all.

assets/tools/blockfacts.py

@@ -0,0 +1,45 @@
+#Shows data from the first 1000 blocks
+
+import random
+import os
+import subprocess
+import json
+
+
+#Set this to your raven-cli program
+cli = "raven-cli"
+
+#mode = "-testnet"
+mode = ""
+rpc_port = 8766
+#Set this information in your raven.conf file (in datadir, not testnet3)
+rpc_user = 'rpcuser'
+rpc_pass = 'rpcpass555'
+
+
+def rpc_call(params):
+    process = subprocess.Popen([cli, mode, params], stdout=subprocess.PIPE)
+    out, err = process.communicate()
+    return(out)
+
+def get_blockinfo(num):
+    rpc_connection = get_rpc_connection()
+    hash = rpc_connection.getblockhash(num)
+    blockinfo = rpc_connection.getblock(hash)
+    return(blockinfo)
+
+def get_rpc_connection():
+    from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException
+    connection = "http://%s:%[email protected]:%s"%(rpc_user, rpc_pass, rpc_port)
+    #print("Connection: " + connection)
+    rpc_connection = AuthServiceProxy(connection)
+    return(rpc_connection)
+
+for i in range(1,1000):
+    dta = get_blockinfo(i)
+    print("Block #" + str(i))
+    print(dta.get('hash'))
+    print(dta.get('difficulty'))
+    print(dta.get('time'))
+    print("")
+

assets/tools/issuebulk.py

@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+# Script to issue assets on the Ravencoin platform
+# Reads from a csv file
+# Template Google Spreadsheet at:  
+#   https://docs.google.com/spreadsheets/d/1Ym88-ggbw8yiMgVxOtVYDsCXJGNGZqlpOfgdbVK8iYU
+# In Google Sheets: File->Download As->.csv
+# Prerequisite: ravend daemon to be running
+# In order to use metadata, you must install be running IPFS
+# Steps:
+#   1. Get IPFS - https://ipfs.io/
+#   2. Run the ipfs daemon
+#   3. pip install ipfsapi
+#
+# If you're signing contract_url
+# pip install python-bitcoinrpc
+
+
+import random
+import os
+import subprocess
+import csv
+import json
+import hashlib
+
+
+#Set this to your raven-cli program
+cli = "raven-cli"
+
+mode =  "-testnet"
+rpc_port = 18766
+#mode =  "-regtest"
+#rpc_port = 18443
+csv_file = "Raven Assets - Sheet1.csv"
+#Set this information in your raven.conf file (in datadir, not testnet3)
+rpc_user = 'rpcuser'
+rpc_pass = 'rpcpass555'
+
+
+def NormalizeMetaData(dict):
+    #Remove all empty items    
+    for key in list(dict.keys()):
+        if dict[key] == '':
+            del dict[key] 
+
+    del dict['asset']
+    del dict['qty']
+    del dict['units']
+    del dict['reissuable']
+
+    #Convert from text to bool
+    if dict.get('forsale') == 'TRUE': 
+        dict['forsale'] = True
+
+
+    return(dict)
+
+def issue_asset(asset, qty, units, reissuable = False, address='', ipfs_hash = ''):
+    cmd = cli + " " + mode + " issue " + asset + " " + str(qty) + " " + "\"" + address + "\"" + " " + "\"\"" + " " + str(units) + " "
+
+    if reissuable:
+        cmd += 'true'
+    else:
+        cmd += 'false'
+
+    if len(ipfs_hash) > 0:
+        cmd = cmd + " true " + ipfs_hash
+
+    print(cmd)
+    os.system(cmd)  
+
+def get_contract_hash(url):
+    import urllib2
+    print ("Downloading: " + url)
+    response = urllib2.urlopen(url)
+    rawdata = response.read()
+    #print("Len: " + len(rawdata))
+    hexdigest = hashlib.sha256(rawdata).hexdigest()
+    print(hexdigest)
+    return(hexdigest)
+
+def rpc_call(params):
+    process = subprocess.Popen([cli, mode, params], stdout=subprocess.PIPE)
+    out, err = process.communicate()
+    return(out)
+
+def get_address():
+    rpc_connection = get_rpc_connection()
+    new_address = rpc_connection.getnewaddress()
+    print("New address: " + new_address)
+    return(new_address)
+
+def sign_hash(address, hash):
+    rpc_connection = get_rpc_connection()
+    signature = rpc_connection.signmessage(address, hash)
+    return(signature)
+
+def generate_blocks(n):
+    rpc_connection = get_rpc_connection()
+    hashes = rpc_connection.generate(n)
+    return(hashes)
+
+def get_rpc_connection():
+    from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException
+    connection = "http://%s:%[email protected]:%s"%(rpc_user, rpc_pass, rpc_port)
+    print("Connection: " + connection)
+    rpc_connection = AuthServiceProxy(connection)
+    return(rpc_connection)
+
+def add_to_ipfs(file):
+    print("Adding to IPFS")
+    import ipfsapi
+    api = ipfsapi.connect('127.0.0.1', 5001)
+    res = api.add(file)
+    print(res)
+    return(res['Hash'])
+
+
+
+if mode == "-regtest":  #If regtest then mine our own blocks
+    import os
+    os.system(cli + " " + mode + " generate 400")
+
+with open(csv_file, "r") as csvfile:
+    #print(rpc_call('getbestblockhash'))
+    reader = csv.DictReader(csvfile)
+    for issue in reader:
+        if issue.get('reissuable') == 'TRUE': 
+            issue['reissuable'] = True
+        else: 
+            issue['reissuable'] = False
+        print(issue['asset'], issue['qty'], issue['units'])
+        asset = issue['asset']
+        meta = issue.copy();
+        NormalizeMetaData(meta)
+        print(issue['asset'])
+
+        #If a contract_url is present, then sign it and add the proof to the meta data
+        if meta.get('contract_url'):
+            meta['contract_hash'] = get_contract_hash(meta['contract_url'])
+            meta['contract_address'] = get_address();
+            meta['contract_signature'] = sign_hash(meta['contract_address'] , meta['contract_hash'])
+            print(meta['contract_signature'])            
+
+
+
+        #If there is metadata, then add to IPFS
+        ipfs_hash = ''
+        if (len(meta)):
+            file = issue['asset'] + '.json'
+            with open(file, 'w') as fp:
+                json.dump(meta, fp, sort_keys=True, indent=2)
+            ipfs_hash = add_to_ipfs(file)
+            print(issue['asset'] + " IPFS Hash:" + ipfs_hash)
+        else:
+            print(issue['asset'] + " No metadata")
+
+        issue_asset(issue['asset'], issue['qty'], issue['units'], issue['reissuable'], meta.get('contract_address', ''), ipfs_hash)
+
+
+
+
+