Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fatxpool: add heavy load testsuits #5497

Closed
1 of 10 tasks
Tracked by #5472
michalkucharczyk opened this issue Aug 27, 2024 · 2 comments · Fixed by #7257
Closed
1 of 10 tasks
Tracked by #5472

fatxpool: add heavy load testsuits #5497

michalkucharczyk opened this issue Aug 27, 2024 · 2 comments · Fixed by #7257
Assignees
Labels
T0-node This PR/Issue is related to the topic “node”.

Comments

@michalkucharczyk
Copy link
Contributor

michalkucharczyk commented Aug 27, 2024

There should be some mid/long term tests that are not intended to be executed during CI, but could be executed locally by a developer or by the QA team during the release process.
Some ideas for testing:

  • send 1M transactions to the parachain zombienet network and check if all transactions are getting finalized
  • check the limits on long term run,
  • check how blocks are filled,
  • check if invalid transactions are rejected,
  • check time-to-event (e.g. in block, finalized, invalid), to see if we have some regression,
  • send different transactions - heavy and light,
  • send future and then ready, see if all gets finalized (fatxpool: add heavy load tests based on zombienet #7257),
  • check mortal transactions (shoud be done in fatxpool: cross check (and improve) support for mortal transactions #5626),
  • check submit and submit_and_watch,
  • make sure nodes are not getting disconnected.

Those should be executed in local zombienet network, possibly in different configurations (e.g. one collator, rpc + collator, rpc + 3x collators).

@iulianbarbu
Copy link
Contributor

iulianbarbu commented Jan 20, 2025

Small update on this - I transformed the zn DSL files from #7100 into zombienet-sdk files: #7257. Most of them are using the same zombienet network: two relaychain validators with one parachain collator/validator, or three collators (where two of them are validators), or four collators (where three of them are validators).

The distinctive parts for these networks are the chain specs used for the parachains, and the arguments passed to each collator node, which are the same for the collators of the same network, but vary slightly between collators of different networks, as expected. My thinking though is that some args might need to be modified/removed/added to the base argument list, if these base networks will be used for further local testing, manually (possibly the chainspecs can be slightly updated/modified too). The ways this can be achieved is:

@iulianbarbu
Copy link
Contributor

iulianbarbu commented Feb 10, 2025

Another update (hoping these are useful, considering we're working on this for some time):

  1. ttxt is shaped into a lib here: Enabled ttxt as a lib michalkucharczyk/tx-test-tool#22
  2. ttxt is being used as a lib in here: fatxpool: add heavy load tests based on zombienet #7257 - work in progress, until we settle on ttxt usage as a lib, but I'll ask @michalkucharczyk for a review over this as soon as I am confident the work with the testing scenario is done.
  3. Working on the last bits of the scenario described here: fatxpool: add heavy load tests based on zombienet #7257 (comment). The scenario runs fine, it is just that I would like to assert on expected account nonces for the accounts under test, after the test finishes. There might be an easier route by checking how many txs have been finalized (ttxt reports a summary where it gives this number, and we can reuse it - but I need to see how I can expose that via API)

github-merge-queue bot pushed a commit that referenced this issue Mar 7, 2025
# Description

Builds up towards addressing #5497 by creating some zombienet-sdk code
infra that can be used to spin regular networks, as described in the
fork aware transaction pool testing setup added here #7100. It will be
used for developing tests against such networks, and to also spawn them
on demand locally through tooling that will be developed in follow ups.

## Integration

Node/runtime developers can run tests based on the zombienet-sdk infra
that spins frequently used networks which can be used for analyzing
behavior of various node related components, like fork aware transaction
pool.

## Review Notes

- Uses ttxt API implemented here:
https://github.com/michalkucharczyk/tx-test-tool/pull/22/files
- currently, only two test scenarios are considered: 10k future & 10k
ready txs are sent to two separate networks - one parachain and one
relaychain, asserting at the end on the finalization of all 20k txs on
both networks.

---------

Signed-off-by: Iulian Barbu <[email protected]>
Co-authored-by: Javier Viola <[email protected]>
Co-authored-by: Michal Kucharczyk <[email protected]>
@github-project-automation github-project-automation bot moved this from In Progress to Done in fork-aware-txpool Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T0-node This PR/Issue is related to the topic “node”.
Projects
Status: Done
2 participants