A set of tools to do performance tests on Apache Flink (with multiple applications in parallel), and for reporting.
This repository contains four main components:
-
Injector
- Uses gateling.io to inject data from a file datasource into Apache Kafka, in order to be processed by the applications.
- The injection rate per application can be configured to change over time.
- Simulates applications being deployed and canceled in Apache Flink.
-
Reporter
- Starts/Stops all the machines/services in your performance environment.
- Clean up the state of the environment (logs from old executions).
- Collects logs from the machines available in your performance environment.
- Launches Apache Zeppelin using Docker and allows you to analyse all the information gathered during the execution.
-
runtest
- Coordinates the components above in order to do the full performance test cicle at once.
First create a distribution file by running:
sbt distFlinkPerfTests
This will create a zip file in the /target folder containing all you need to run the performance tests.
- To run the injector, simply modify the configurations file (conf/injector.conf) and run bin/injector.
- This will run the performance tests described in the configuration files, using Gatling.io.
- In the end, it will generate an html report in the ./reports folder.
- Setup the hosts configuration files as described in scala-ssh.
- Modify the configurations file (conf/reporter.conf) as you need and run bin/reporter with the required arguments.