Istanbul BFT Test Specification

Welcome to the istanbul-tools wiki!

Availability/Stress Testing


# of validators	4	7

Test cases are numbered with the format of TA-B-CC-DD where:

Symbol	Description
`T`	Numbering initial, defaults to `T`.
`A`	Group. `F`: Functional tests. `P`: Performance tests.
`B`	Attribute sub-group. `U`: Unit tests. `S`: Situational tests.
`C`	Sequence.
`D`	Sub-sequence.

Number	Item	Description	Expectation
TFS-01-01	Blockchain initialization	Run `geth init` with proper `gensis.json`	Should run without fails.
TFS-01-02	Run `geth`	Run `geth` after `geth init`.	Should run without fails.
TFS-01-03	Peer connection	Run `geth` with static validators should be able to connect to peers	`admin.peers` should return expected peers.
TFS-01-04	Consensus progress	Validator network should be able to generate blocks	Watch 100 blocks and each block should be created in 3 seconds.
TFS-01-05	Round robin proposer selection	Validator network should pick proposers in round robin fashion	Watch 100 blocks and make sure each validator generates at least 20 blocks in a 4 validator network.

Number	Item	Description	Expectation
TFS-02-01	Add validators	Randomly add several validators	Validator network should grow.
TFS-02-02	New validators consensus participation	Add a new validator in validator network	Watch few block and see if the newly added validator has been picked as proposer at least once.
TFS-02-03	Remove validators	Randomly remove few validators	Validator network should shrink.
TFS-02-04	Reduce validator network size below 2F+1	Reduce validator network size but keep it more than 2F+1.	Blocks should still be created continuously.
TFS-02-05	Reduce validator network size below 2F+1	Reduce validator network size to less than 2F+1.	Blocks should stop being created.

Number	Item	Description	Expectation
TFS-03-01	Add validator to more than 2F+1	Add validators in a network with < 2F+1 validators to > 2F+1.	Blocks should start being created.

Number	Item	Description	Expectation
TFS-04-01	Stop F validators	In a 3F+1 network, stop F validators.	Blocks should keep generating.

Number	Item	Description	Expectation
TFS-05-01	F faulty validators	In a 3F+1 network, setup F faulty validators and 2F+1 normal validators.	Blocks should keep generating.

Number	Item	Description	Expectation
TFS-06-01	Node connection	Run a 3F+1 validator network and start a new node	The node should be able to connect to the validators.
TFS-06-02	Node synchronization	Run a 3F+1 validator network, start a new node, and stop validator mining.	The node and validators should have the same block height.

Number	Item	Description	Expectation
TFS-07-01	Consensus on gossip network	Run a weakly connected 3F+1 validator network, each validator connects to only one other validator.	Block should keep generating.

Number	Item	Description	Expectation
TPS-01-01	Load testing with 4 nodes	Run 4 validators for 150 seconds with 21000*1000 block gas limit and sending 200 transaction per second.	Watch 100 blocks and measure if average block time (3s) and size (?tx).
TPS-01-02	Load testing with 7 nodes	Run 7 validators for 300 seconds with 21000*2000 block gas limit and sending 400 transaction per second.	Watch 100 blocks and measure if average block time (?s) and size (?tx).
TPS-01-03	Load testing with 10 nodes	Run 10 validators for 600 seconds with 21000*3000 block gas limit and sending 800 transaction per second.	Watch 100 blocks and measure if average block time (?s) and size (?tx).

Notes:

1. Create a new blockchain with 3f+1 validators.
2. Send transactions (Use `eth_sendTransaction`).
3. See if the block creation works.


# of validators	4	7
Block gas limit	21000*1000	21000*2000	21000*3000