Conversation
radupopa2010
added
B0-silent
|
Please remove simnet_tests/configs/local_testnet_with_logs.toml It was my debug for connection problems. Should be cleaned on Gitlab as well |
| forward_port | ||
| run_test | ||
|
|
||
|
|
There was a problem hiding this comment.
Is this script a minimum that's required to launch simnet? In order to lower the maintenance, is it possible to move its' parts to simnet repo?
There was a problem hiding this comment.
This script sometimes is used for the setup of the tests, when we have no config options in the config files.
Each type of test, substrate or parachains has it's own version of run_test.sh.
Fedor was working on another version of this script to spawn a huge parachains network with different setup then the tests we already run
There was a problem hiding this comment.
it combines three functions then: configure, launch and add some testing features. While it should only pass the payload to simnet. Launching mechanism should be on simnet side, testing should happen in test files.
There was a problem hiding this comment.
@TriplEight can you please elaborate? Right now this script is the entry point for the set of tests. Generally it has the following sections:
- Collect required variables
- Spawn the chain, using config files
- Perform additional setup for the chain (things, that cannot be done inside configuration)
- Run set of tests
So this script binds configuration and testing logic into one "entity", and CI runs this "entity". What parts do you suggest to move?
There was a problem hiding this comment.
As a tool, Simnet should have an entrypoint, the user should pass the requirements to the entrypoint, not triggering it in three steps. I suggest on the side of CI we store only the configs and tests, collect them and send to simnet.
And Simnet will
- accept tests and configs
- spawn the chain (setup)
- initiate the tests
- teardown
"Perform additional setup for the chain (things, that cannot be done inside configuration)" looks like another (5th or 6th) config entity, we should have less of them, can it go into the Simnet code with certain API points or feature flags?
I'm not suggesting doing everything in this PR, ofc.
There was a problem hiding this comment.
I agree with you Denis, this is the ideal case. In order to do this we need to do a bit more development and testing.
This will delay a the move of tests into the substrate repo. We need to think about the trade-offs.
For now we can live with the current state and do further iterations to improve the configs.
There was a problem hiding this comment.
yeah, just note this somewhere, please. This PR can be merged.
3 participants
This PR adds directory which will hold high level integration tests that will run with the simnet set of tools.
At the moment those kind of tests are maintained by the simnet team in simnet repo which is hosted just on gitlab.
This was done in this manner while we developed working version of tools to spawn a simulation network based on a docker image and run tests against this network.
There will be a follow up PR to edit the Ci file of this project, but I need those files hosted in this repo before.
We are still developing simnet and we are thinking how to make it simpler to use.
By moving simnet tests into this repository, the expectations are that developers would be able to maintain and create new tests.
Off course simnet team will help explaining how things work and how to use simnet tools