You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have created this repo for various real-time integration tests for public API. See #1 for details.
This issue is for adding a stress test to that repo.
The test should randomly create senders and receivers, perform some exchange, and destroy them. This should be done repeatedly from multiple threads.
We can implement this using a simple scheme. Start several threads. Each thread runs a loop: randomly choose some config parameters, create sender (or receiver), randomly choose to which currently running receiver to connect (for sender only), write (or read) samples during random time, destroy sender (or receiver), repeat the loop.
During the test, this algorithm should hit different combinations
sender without receiver
one sender attached to one receiver
multiple senders attached to one receiver
Config parameters that should vary (randomly): FEC scheme, resampler profile, frame / packet / FEC block sizes, FEC redundancy level, packet interleaving, latency parameters. Also, we can bind receiver to: one port (bare RTP), two source & repair ports (reed-solomon or ldpc), or any other number of ports (e.g. multiple source & repair pairs for different or identical FEC schemes). Also we can have a separate context for each sender or receiver, or share context between a random number of senders and receivers.
The test should also perform some minimal check of the received data. E.g. we can send a monotonic sequence (1 2 3 4 ....) and check that we still receiver a monotonic sequence, probably with some amount of loss (lost samples will be zeros).
The user should be able to configure how long the test runs and how much threads to start.
We want to test quite a lot of combinations, but we can implement them incrementally: first implement and merge some minimal version, than iteratively add more randomness.
Task
We have created this repo for various real-time integration tests for public API. See #1 for details.
This issue is for adding a stress test to that repo.
The test should randomly create senders and receivers, perform some exchange, and destroy them. This should be done repeatedly from multiple threads.
We can implement this using a simple scheme. Start several threads. Each thread runs a loop: randomly choose some config parameters, create sender (or receiver), randomly choose to which currently running receiver to connect (for sender only), write (or read) samples during random time, destroy sender (or receiver), repeat the loop.
During the test, this algorithm should hit different combinations
Config parameters that should vary (randomly): FEC scheme, resampler profile, frame / packet / FEC block sizes, FEC redundancy level, packet interleaving, latency parameters. Also, we can bind receiver to: one port (bare RTP), two source & repair ports (reed-solomon or ldpc), or any other number of ports (e.g. multiple source & repair pairs for different or identical FEC schemes). Also we can have a separate context for each sender or receiver, or share context between a random number of senders and receivers.
The test should also perform some minimal check of the received data. E.g. we can send a monotonic sequence (1 2 3 4 ....) and check that we still receiver a monotonic sequence, probably with some amount of loss (lost samples will be zeros).
The user should be able to configure how long the test runs and how much threads to start.
We want to test quite a lot of combinations, but we can implement them incrementally: first implement and merge some minimal version, than iteratively add more randomness.
Info
rt-tests repo currently has a skeleton of a test using public API:
https://github.com/roc-streaming/rt-tests/blob/1212a60b0357c851282be3ec1cb1887e652ab22e/tests/test_service_quality.cpp
Public API is documented here: https://roc-streaming.org/toolkit/docs/api.html
FEC is documented here: https://roc-streaming.org/toolkit/docs/internals/fec.html
Another integration test using public API can be found here:
https://github.com/roc-streaming/roc-toolkit/blob/master/src/tests/public_api/test_sender_receiver.cpp
The text was updated successfully, but these errors were encountered: