By using this software you understand the risks of Web3 and blockchain technologies. This toolkit is intended for testing and learning exciting concepts in the Web3 and blockchain industry with a security-minded self-education approach. The blockchain and DeFi ecosystems do not show mercy in any way to even the smallest mistakes. I recommend that you self-educate yourself first and read MetaMask's security knowledge base before experimenting with this repository. You are responsible for your own personal cryptocurrency funds and Web3 wallet private keys, and it is your personal responsibility to educate yourself to protect them. Any misuse or mishandling of this software on any of the EVM compatible blockchain mainnets or testnets is at your own risk.
Also just a friendly warning, DO NOT ATTEMPT TO SEND FUNDS to any of the 20 test accounts provided by hardhat.
The capabilities of the Solidity programming language and the Ethereum platform in general are vast and growing every day. One of the common usecases for Ethereum has been to approach it as a gigantic Web3 backend system. The very simple OpenZeppelin ERC20 TestToken and other smart contract examples included in this toolkit are only tiny slices of the bigger picture in comparison to what is being used today in Web3 and DeFi production environments. In general when you approach people on the street and they hear the word Ethereum, the reactions you would probably get from them is "oh that's the thing I can buy on Coinbase or Binance and send to my parents on the other side of the world" or "that's the blockchain network that has all those worthless poo-coins right?". In reality, Ethereum was built for much immensely grander designs than the simple monetary usecases that most people are familiar with. Vitalik Buterin and the other Ethereum co-founders originally sought to create a "World Computer". As you browse through this documentation you will notice that I am not attempting to try to create "generic keywords" for the Solidity language or the "World Computer" that smart contracts are deployed to, and you will see a lot of mostly non-generic transparent strategies in this repository. As a busy person who is trying to actively learn blockchain technologies and is worried about the security of his own Web3 wallet (also concerned for the safety of all of yours as well), I will leave the generic blockchain automation keyword development for others to pursue. An important trend in the Ethereum ecosystem to pay more attention to is the rising popularity of bridges into many other types of blockchains, and the current Web3 protocols that are evolving towards cross-chain communication. I have usecases of my own for cross-chain EVM (Ethereum Virtual Machine) projects and prefer to focus on my ideas for this toolkit, but I wanted to share my high-level strategies here (also discussed with TestGuild's Joe Colantonio on YouTube) and possibly hear your thoughts too if you want to reach out to me on Robot Framework's Slack Group.
This multichain toolkit contains keyword-driven automation that helps locally test EVM compatible Solidity smart contracts, deploys them using a multichain approach, and has the basic building blocks for creating blockchain event monitoring bots. One of the primary components is a custom-made robotframework-hardhat-remote-library.js that was built using hardhat and ethers.js libraries for Web3 development, and combined together using comick's node-robotremoteserver. This toolkit has been designed to create local clean-room environment mainnet forks through Hardhat Network for deploying smart contracts on multiple types of blockchains that support the EVM. I have included small checks and basic RPA automation scripts that work for Ethereum, Fantom, Avalanche, Polygon, but they can be adapted to cover many others as well by simply adjusting minor hardhat CLI options and altering the provided config files. Also all of the automated checks in this repository are spending ZERO actual gas because they are mostly reading or interacting with simulated blockchains running on a local machine, and not sending testnet transactions (scroll down to see risks of frequently doing that) or touching anything on mainnets.
You may be wondering. What's so special about Hardhat Network, Trufflesuite's Ganache, and "clean-room environment testing"? Isn't testing on any of the Ethereum Testnets enough since ETH on a testnet has zero real-world value? Technically all blockchain dApps and complex integrations with multiple existing smart contracts should always be deeply tested with testnets first before deploying them to a mainnet, but relying ONLY on testnets for testing is not always a good idea. For example, here are some disadvantages and possibly severe risks for testing on both Ethereum Testnets and Ethereum Mainnet.
Some Testnet testing disadvantages and risks:
- Most of the time a testnet will be slow
- Even though it's a testnet you still need to worry about paying gas fees for every Metamask wallet transaction
- If you're using
hardhatortrufflethere is a risk that you can expose your Metamask wallet private key in your deployment configuration files by accidentally pushing them into GitHub - Deploying smart contracts to a blockchain testnet is immutable or permanent. Which is exactly how mainnets behave as well
- If you are in the security sector of the blockchain industry, testing on an Ethereum Testnet can have other possible serious risks. Testnets are constantly being attacked on a daily basis. Very bad actors are good at tracking all transactions on testnets and mainnets, which could lead to various security risks. One of those risks includes indirectly exposing your Metamask address to these bad actors
Some Mainnet testing disadvantages and risks:
- Very expensive to test on a mainnet
- You need to worry about paying real gas fees for every Metamask wallet transaction
- If you're using
hardhatortruffle, similar to testnets, there is the same risk of exposing your Metamask wallet private key - Deploying smart contracts to a blockchain mainnet is immutable or permanent. It will stay there on the blockchain forever
- Same possible security risks found on testnet that I mentioned above, but on mainnet it can be much worse. Click here to read an interesting real-life Ethereum Mainnet "horror story"
Stay vigilant and protect yourself:
There will be special cases and complex projects where frequent testing and smart contract deployments to Ethereum Testnet are unavoidable. I recommend setting up a "testing-only Metamask wallet" on a different browser (or maybe on a completely different laptop if possible), and also look into investing in a high-quality and well tested VPN service. Always assume you are exposed and try your best to protect your internet connection and your personal workstation from exposure to scammers, malicious hackers, and other bad actors that are frequently found in the blockchain industry. Also don't let this dark side of the industry scare you away from these exciting technologies. Once you set up proper protections and establish good practices you should be ok.
MochaJS is a good and flexible test framework, but it's not easy to figure out how it can handle the following types of tests...
- Model-based Testing
- Chaos Testing running in parallel with other tests (Click here to watch my RoboCon 2023 YouTube Video on this subject)
- Parallel tests that require the tester to measure system resources while running various types of tests at the same time
A lot of the automation in this repository is influenced by my Robot-Framework-Lone-Tester-Strategies-RoboCon-2019, Tool-Strategies-Lone-Testers-Test-Leadership-Congress-2019, and Getting-Started-Robotframework-AppiumLibrary-RoboCon-2021-And-2022 workshops. More information about these RoboCon workshops can be found on robocon.io or watch this YouTube video (click here) to see an example. Also just a general side note, I created this toolkit to demonstrate that there is more than one way to approach Web3, dApp, or blockchain-related testing and to encourage QA professionals who are interested in RPA tools. I also intended this to help suggest ideas to bridge Robot Framework into Web3 and blockchain, as a means to experiment with new or cutting edge testing technologies focused on the security of EVM compatible smart contracts.
In the future I plan to expand this toolkit into the following areas.
- Create a
robotframework-truffle-remote-library.jsusing the Truffle toolkit, and design similar features that are found inrobotframework-hardhat-remote-library.js- Progress:
- (February 12, 2023 - Ready To Use) Created a working Truffle Suite library using a similar design as the Hardhat library, but utilizes web3.js and added a few different tests too.
- Progress:
- Create more static analysis, security, and vulnerability scanning RPA automation integrated with popular tools in the Web3 and blockchain industry
- Create RPA and
robotframework-hardhat-remote-library.jspowered multichain bots utilizing Infura RPC nodes, Hardhat toolkit, and ethers.js - Create RPA and Alchemy powered bots utilizing their SDK and their RPC nodes.
- Attempt to create Robot Framework automation utilizing the Foundry toolkit
- Progress:
- (March 4, 2024 - Ready To Use) Created working parallel running Foundry and Playwright tests that runs checks on a mock wallet dApp connected to a local Anvil node.
- Progress:
- Create Metamask automation using SeleniumLibrary and robotframework-browser (a Playwright library)
- Create a Robot Framework Model-based Testing example utilizing similar concepts found in my PaBot-Android-Device-Graphwalker-Examples.robot
- Play around with some Chaos Testing ideas for dApps
The following are the basic technical requirements needed to run these automated checks. Please note that this entire toolkit was developed on a MacOS machine, but should also work for most Linux users (preferably most current Ubuntu LTS versions etc.) and I would keep an eye on whatever the hardhat or truffle (source: https://github.com/trufflesuite/ganache-ui/releases) communities recommend for Linux distributions.
- Python 3 -> https://www.python.org/downloads/
- NodeJS 18 -> https://nodejs.org/en/ or use https://github.com/nvm-sh/nvm
- If you're using
nvm, you can runnvm install 18to install version 18 - If you are using a MacOS or Linux machine, the
Install-Tools-On-MacOS-Or-Linuxcommand below is a quick install
- If you're using
- Robot Framework -> https://robotframework.org
- Create your own free Infura account or free Alchemy account, then set up your own API keys
- Replace all instances of
<your-infura-api-key-goes-here>in theAvalanche.config.jsandEthereum.config.jsconfiguration files. Also replace<your-alchemy-api-key-goes-here>in thePolygon.config.jsconfiguration file.
Before running any of the automation first make sure that all of the basic technical requirements are installed and working, then you can run bash ./start-solidity-qa-workflows.sh Install-Tools-On-MacOS-Or-Linux. After the installation script successfully finishes you should be able to run any of the following commands without problems.
You can view just this help menu again (without triggering any automation) by running 'bash ./start-solidity-qa-workflows.sh -h' or 'bash ./start-solidity-qa-workflows.sh --help'.
---->>>> Local Solidity Test Environment Setup Commands <<<<----
bash ./start-solidity-qa-workflows.sh Install-Tools-On-MacOS-Or-Linux
bash ./start-solidity-qa-workflows.sh Install-Solidity-Security-Testing-Tools-On-MacOS
bash ./start-solidity-qa-workflows.sh Start-Default-Hardhat-Network-And-Robotremoteserver
bash ./start-solidity-qa-workflows.sh Start-Multichain-Hardhat-Network-And-Robotremoteserver
bash ./start-solidity-qa-workflows.sh Start-Default-Truffle-Develop-And-Robotremoteserver
bash ./start-solidity-qa-workflows.sh Start-Default-Foundry-Anvil-Node-And-Wagmi-dApp
bash ./start-solidity-qa-workflows.sh Start-Forked-Foundry-Anvil-Node-And-Wagmi-dApp
bash ./start-solidity-qa-workflows.sh Stop-Local-Blockchain-Nodes-And-Delete-Logs
---->>>> Solidity Static Analysis And Security Testing Commands <<<<----
bash ./start-solidity-qa-workflows.sh Run-Solidity-Static-Analysis
---->>>> Hardhat Deployment Commands Powered by Robot Framework RPA (NOTE: These scripts are capable of deploying to Mainnets or Testnets!) <<<<----
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Hardhat-Ethereum-RPA-Deployment
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Hardhat-Fantom-RPA-Deployment
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Hardhat-Polygon-RPA-Deployment
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Hardhat-Avalanche-RPA-Deployment
---->>>> Truffle Deployment Commands Powered by Robot Framework RPA (NOTE: These scripts are capable of deploying to Mainnets or Testnets!) <<<<----
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Truffle-Ethereum-RPA-Deployment
---->>>> Interactive Tests For Deployed Smart Contracts Using Hardhat (NOTE: These tests can run on multiple EVM Compatible Blockchains!) <<<<----
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Tests-Default-NonForked-Hardhat-Network
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Tests-Forked-Mainnet-Hardhat-Network
---->>>> Interactive Tests For Deployed Smart Contracts Using Truffle Suite <<<<----
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Tests-Default-Truffle-Develop-Instance
---->>>> Interactive Tests For Deployed Smart Contracts Using Foundry <<<<----
bash ./start-solidity-qa-workflows.sh Run-Parallel-Smart-Contract-Foundry-And-Playwright-Browser-Tests
[Example Workflow 1] -> Deploy then test a smart contract using the built-in Hardhat Network Ethereum Node and node-robotremoteserver (on MacOS or Linux)
From the commands posted above, here is an example workflow (with screenshots) below that will work reliably as long as both the basic technical requirements and the bash ./start-solidity-qa-workflows.sh Install-Tools-On-MacOS-Or-Linux command have been completed without issues.
- First, open two terminals and keep both open the entire time
- In one of the terminals, run the
bash ./start-solidity-qa-workflows.sh Start-Default-Hardhat-Network-And-Robotremoteservercommand then check that the output matches what is shown in the following screenshot
- In the other terminal, run the
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Tests-Default-NonForked-Hardhat-Networkcommand then check that the output matches what is shown in the following screenshot
[Example Workflow 2] -> Fork Fantom Mainnet, deploy, then test an existing smart contract using the Hardhat Network Fantom fork and node-robotremoteserver (on MacOS or Linux)
From the commands posted above, here is an example workflow (with screenshots) below that will work reliably as long as both the basic technical requirements and the bash ./start-solidity-qa-workflows.sh Install-Tools-On-MacOS-Or-Linux command have been completed without issues.
- First, open two terminals and keep both open the entire time
- In one of the terminals, run the
bash ./start-solidity-qa-workflows.sh Start-Multichain-Hardhat-Network-And-Robotremoteservercommand, follow the instructions by enteringFantom, then check that the output matches what is circled in the following screenshot
- In the other terminal, run the
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Tests-Forked-Mainnet-Hardhat-Networkcommand then check that the output matches what is circled in the following screenshot
- This test will locally interact with someone else's existing token project (contract address:
0x5FbDB2315678afecb367f032d93F642f64180aa3) on Fantom Mainnet called "PumpNoDump (PND)". FYI, the PND token is NOT mine and I am not associated with it at all
[Example Workflow 3] -> Use Robot Framework RPA to do only the deployment using Hardhat (on MacOS or Linux)
From the commands posted above, here is an example workflow below that will work reliably as long as both the basic technical requirements and the bash ./start-solidity-qa-workflows.sh Install-Tools-On-MacOS-Or-Linux command have been completed without issues.
- Only one terminal is needed
- Run the
bash ./start-solidity-qa-workflows.sh Run-Smart-Contract-Hardhat-Ethereum-RPA-Deploymentcommand then check that the output matches what is shown in the following screenshot
Results for the example workflows above can be found in the ...-log.html files stored in the logs folder.
All Windows users should please read my NOTES.md file for more details.