Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create benchmarks #2

Open
gavv opened this issue Sep 30, 2020 · 0 comments
Open

Create benchmarks #2

gavv opened this issue Sep 30, 2020 · 0 comments
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@gavv
Copy link
Member

gavv commented Sep 30, 2020

Task

We have created this repo for various real-time integration tests for public API. See #1 for details.

This issue is for adding API benchmarks.

Currently we have two time-sensitive operations in API: roc_sender_write() and roc_receiver_read(). It would be useful to cover them with benchmarks and periodically check them for regressions. This is especially important for weaker CPUs like ARM.

The performance heavily depends on sender or receiver configuration, so we should check different combinations of all important parameters: resampler profile, FEC scheme, frame, packet, and FEC block sizes, packet interleaving.

Benchmarks should be done with automatic timing disabled (see API) because otherwise we will measure performance of sleep.

Benchmarks can print a table with results (measured performance for different combinations of parameters) to stdout.

We likely should employ Google Benchmark library. Then we'll also need to add it to CMake. Google Test is already present there.

Info

rt-tests repo currently has a skeleton of a test using public API:
https://github.com/roc-streaming/rt-tests/blob/1212a60b0357c851282be3ec1cb1887e652ab22e/tests/test_service_quality.cpp

Public API is documented here: https://roc-streaming.org/toolkit/docs/api.html

FEC is documented here: https://roc-streaming.org/toolkit/docs/internals/fec.html

Another integration test using public API can be found here:
https://github.com/roc-streaming/roc-toolkit/blob/master/src/tests/public_api/test_sender_receiver.cpp

@gavv gavv added good first issue Good for newcomers help wanted Extra attention is needed hacktoberfest labels Sep 30, 2020
@gavv gavv removed the hacktoberfest label Dec 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant