A Ruby performance benchmarker based on Zed Shaw's Programmers Need To Learn Statistics Or I Will Kill Them All and Pat Eyler's Benchmarking, Lies, and Statistics.
Check examples_tests/ for examples of how to setup and run a benchmark comparing two blocks of code (or two webapps).
You can configure:
sample_size: The number of samples of each block to measure.rampup_size: The number of initial samples to ignore to allow the system to reach steady state.
You can generate a graph of the samples (results.png, created using gruff) and statistics tables (using command_line_reporter).
Wynmark will:
- calculate a winner (the faster block of code) based on statistical significance (
+/-2 * standard_deviation) - identify
outliers(which are probably an indication of a garbage collection delay)
gruff depends on rmagick, which depends on ImageMagick v6, which isn't the latest. So you may need to do this to bundle successfully.
# Terminal
brew install imagemagick@6
PKG_CONFIG_PATH="/usr/local/opt/imagemagick@6/lib/pkgconfig" bundle