Skip to content
This repository has been archived by the owner on May 13, 2022. It is now read-only.

Testing

Jump to bottom
Carl Crott edited this page Sep 27, 2017 · 48 revisions

What's needed before starting:

An installation of Bitcoin Core (it doesn't have to be running or synchronized, as we only use regtest mode here).

You'll need to know the path to your bitcoind and bitcoin-cli commands (see below), although they will be picked up as default if you miss out the btcroot option.

Note: Commands prefaced with $ are command line commands and run from the main joinmarket dir

OSX

Create symlink to bitcoin core's data dir to simplify commands:
$ sudo ln -s ~/Library/Application\ Support/Bitcoin/ ~/.bitcoin

Create a default joinmarket config:
$ cp test/regtest_joinmarket.cfg joinmarket.cfg

Backup default bitcoin.conf in bitcoin core dir:
$ cp ~/.bitcoin/bitcoin.conf ~/.bitcoin/bitcoin_backup.conf

Edit the above bitcoin.conf to match your joinmarket.conf:

#rpc_user value from joinmarket.conf
rpcuser=bitcoinrpc
#rpc_password value from joinmarket.conf
rpcpassword=123456abcdef
#testing-specific port
rpcport=18332
walletnotify=curl -sI --connect-timeout 0.1 http://localhost:62612/walletnotify?%s
alertnotify=/usr/bin/wget -q --spider --timeout=0.5 --tries=1 http://localhost:62612/alertnotify?%s

Install miniircd (lightweight IRC server):
$ git clone https://github.com/Joinmarket-Org/miniircd

Running tests

Notes:

  1. The test should not conflict with a running bitcoind, as the testing bitcoind is run in regtest mode.
  2. Use fully expanded filepaths for all flags. Using "~/" may cause issues.
  3. Ensure --btcroot directory ends with a "/" or tests will fail.

Method 1 (jmvenv located inside joinmarket dir):

$ PYTHONPATH=.:$PYTHONPATH py.test --cov-report html --nirc=2 --ignore test/test_tumbler.py --btcroot=/path/to/bitcoin/bin/ --btcpwd=123456abcdef --btcconf=/Users/username/.bitcoin/bitcoin.conf

Method 2 (jmvenv located inside ~/.virtualenvs dir):

If method 1 fails to work, it could be due to recursive test running. This is solved by relocating the virtualenv outside the joinmarket directory:
$ mkdir ~/.virtualenvs
$ mv jmvenv ~/.virtualenvs
$ source ~/.virtualenvs/jmvenv/bin/activate #reactivate the jmvenv

Run full test suite (not including tumbler saving 5-10 minutes):
$ python -m pytest --cov-report html --nirc=2 --ignore test/test_tumbler.py --btcroot=/Users/username/python_projects/bitcoin/src/ --btcpwd=123456abcdef --btcconf=/Users/username/.bitcoin/bitcoin.conf

This (or similar) will show the coverage after the run:
$ open htmlcov/index.html

Since a full test suite takes several minutes, you may want to just run individual tests (such as one you write); in that case, just do:

$ PYTHONPATH=.:$PYTHONPATH py.test --cov-report html --btcroot=/path/to/bitcoin/bin/ --btcconf=~/.bitcoin/bitcoin.conf --btcpwd=123456abcdef --nirc=2 test/test_mytest.py

or

$ python -m pytest --cov-report html --nirc=2 --ignore test/test_tumbler.py --btcroot=/Users/username/python_projects/bitcoin/src/ --btcpwd=123456abcdef --btcconf=/Users/username/.bitcoin/bitcoin.conf test/test_mytest.py

Note the pytest convention: test modules should start with test_* and test functions in test modules, likewise.

The tests are based on two key tools: miniircd and Bitcoin regtest. The former is an ultra-lightweight IRC server which is run locally. The latter is Bitcoin's feature to allow testing end-to-end bitcoin transactions and blockchain access again, locally on your own machine, with control over block mining. The current list of tests covers:

  • test_enc_wrapper.py - the libnacl e2e encryption for messages

  • test_bip32.py - the BIP 32 HD wallet key manipulation library

  • test_addresses.py - testing Bitcoin address handling

  • test_regtest.py - an end to end test of sendpayment with yield generators running in the background

  • test_mnemonic.py - testing the old electrum 12 word mnemonic seed generation used in joinmarket

  • test_blockr.py - tests wallet access via the blockr blockchain interface

  • test_wallets.py - testing wallet passwords and recovery

  • test_btc_formatting.py - some tests of data parsing for Bitcoin

  • test_donations.py - testing the special reusable pubkey ("stealth") style payments for tumbler donations

  • test_ecc_signing.py - test ECDSA signatures with RFC6979 against test vectors

  • test_tumbler.py - test of the tumbler bot

  • test_tx_serialize.py - testing bitcoin transaction serialization and deserialization

  • test_tx_creation.py - test correct complete construction of various Bitcoin transactions

  • test_keys.py - tests handling of WIF private keys

  • test_broadcast_method.py - tests broadcast via self. vs other bots

  • test_podle.py - tests creation of PoDLE style commitments used as of 0.2.0

  • conftest.py - special pytest file used for test setup (in particular, startup/shutdown of bitcoin regtest blockchain and miniircd IRC server)

A number of test vectors are stored in *.json files in the test/ directory also.

###Some features of pytest used:

The build test in travis.yml is basically the command listed above, so according to pytest rules it picks up all the test_* modules in the test directory.

pytest fixtures can be used for setup/teardown of individual test modules, as opposed to the global setup in conftest.py. Mainly this is calling things like load_program_config.

Another very useful feature is test parametrization. For example, from test_regtest.py:

@pytest.mark.parametrize(
"num_ygs, wallet_structures, mean_amt, mixdepth, sending_amt",
[
    # basic 1sp 2yg
    (2, [[1, 0, 0, 0, 0]] * 3, 10, 0, 100000000),
    # 1sp 4yg, 2 mixdepths
    (4, [[1, 2, 0, 0, 0]] * 5, 4, 1, 1234500),
    # 1sp 3yg, 2 mixdepths, sweep from depth1
    (3, [[1, 3, 0, 0, 0]] * 4, 4, 1, 0),
])

Each tuple in the list is the set of arguments to re-run the test function against. So here we are trying the sendpayment in a few different modes. We can add tons more, but have to consider how long it takes.

###Test scripts not included in the build test:

test_tumbler.py - even the simplest semi-realistic tumbler test takes 5 minutes+ to complete so is probably not practical as part of a build test, hence not added to the travis build.

###Tests being worked on or still needed:

One area badly needing more tests is the BlockchainInterface implementations. These are especially tricky as they can't be done against a Bitcoin regtest instance. There is currently test_blockr.py which is covering some features against the testnet version. Finding more sophisticated tests that don't require "real" blockchain access is an open question.

We need more thorough testing of IRC functioning (see test_irc_messaging.py), although a lot has been done.

Related (and more important architecturally), we need more testing of the Joinmarket messaging protocol (e.g. feeding garbage messages to bots). There is some of that in the above mentioned test script.

###Clean up.

If you are doing a lot of tests, your regtest blockchain gets a bit crowded; generally this needn't be a problem, although I have occasionally seen delays and glitches. At least periodically, you'll want to wipe your regtest blockchain:

rm -rf /home/joinmarketeer/.bitcoin/regtest

Needless to say, be very careful doing this.

###Some useful/not obvious things for test writers

The file test/commontest.py contains helper functions, at the moment the only important one is make_wallets, which is a convenient way to populate a set of wallets in regtest mode with coins at various mixdepths. See the documentation of the function for details.

A very unobvious point, related to pytest: pytest does provide coverage of subprocesses. This is very useful in our case, because bots like sendpayment and yieldgenerator work on listening loops, so cannot run in the same path of execution. We start up one bot in the main process and other bots with Python's subprocess module; but, when shutdown is needed, it must be done "gracefully", otherwise pytest's coverage post-processing won't be able to operate. See these lines in test_regtest.py:

finally:
    if any(yigen_procs):
        for ygp in yigen_procs:
            #NB *GENTLE* shutdown is essential for
            #test coverage reporting!
            ygp.send_signal(signal.SIGINT)
            ygp.wait()

Troubleshooting

error: couldn't connect to server: unknown (code -1) Among other causes a bound port can cause the above failure, preventing bitcoind from starting. Run a port check on 18444 to verify that another process isn't currently occupying the port bitcoind would be using.

Clone this wiki locally