Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create stress test #3

Open
gavv opened this issue Sep 30, 2020 · 0 comments
Open

Create stress test #3

gavv opened this issue Sep 30, 2020 · 0 comments
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@gavv
Copy link
Member

gavv commented Sep 30, 2020

Task

We have created this repo for various real-time integration tests for public API. See #1 for details.

This issue is for adding a stress test to that repo.

The test should randomly create senders and receivers, perform some exchange, and destroy them. This should be done repeatedly from multiple threads.

We can implement this using a simple scheme. Start several threads. Each thread runs a loop: randomly choose some config parameters, create sender (or receiver), randomly choose to which currently running receiver to connect (for sender only), write (or read) samples during random time, destroy sender (or receiver), repeat the loop.

During the test, this algorithm should hit different combinations

  • sender without receiver
  • one sender attached to one receiver
  • multiple senders attached to one receiver

Config parameters that should vary (randomly): FEC scheme, resampler profile, frame / packet / FEC block sizes, FEC redundancy level, packet interleaving, latency parameters. Also, we can bind receiver to: one port (bare RTP), two source & repair ports (reed-solomon or ldpc), or any other number of ports (e.g. multiple source & repair pairs for different or identical FEC schemes). Also we can have a separate context for each sender or receiver, or share context between a random number of senders and receivers.

The test should also perform some minimal check of the received data. E.g. we can send a monotonic sequence (1 2 3 4 ....) and check that we still receiver a monotonic sequence, probably with some amount of loss (lost samples will be zeros).

The user should be able to configure how long the test runs and how much threads to start.

We want to test quite a lot of combinations, but we can implement them incrementally: first implement and merge some minimal version, than iteratively add more randomness.

Info

rt-tests repo currently has a skeleton of a test using public API:
https://github.com/roc-streaming/rt-tests/blob/1212a60b0357c851282be3ec1cb1887e652ab22e/tests/test_service_quality.cpp

Public API is documented here: https://roc-streaming.org/toolkit/docs/api.html

FEC is documented here: https://roc-streaming.org/toolkit/docs/internals/fec.html

Another integration test using public API can be found here:
https://github.com/roc-streaming/roc-toolkit/blob/master/src/tests/public_api/test_sender_receiver.cpp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant