Add RedisTxn storage adapter.

[arris-ray]

Problem

The RedisNG storage adapter exhibits significant performance degradation as the number of registered metrics increases. This behavior is most notable with Summary metrics but the statement applies for all metric types. The screenshots below shows {p50, p75, p90, p99} timings for collecting {1000, 2000, 5000, 10000} sample Summary metrics across 10 runs in each case. Generally, we can see it takes roughly 4 seconds for a RedisNG adapter to collect 10000 summary metrics.

This behavior seems to stem from the fact that the collection process for each type of metric involves:

• pulling a set of keys names from Redis
• looping over each Redis key in PHP
• and issuing one or more subsequent Redis commands per metric Redis key

This results in network activity that scales with the number of metrics and becomes prohibitively expensive with thousands or tens-of-thousands of metrics.

Reproducing the Problem

This PR adds a benchmark test which was used to generate the data behind the screenshot above. After running docker-compose up to fire up all the configured services, shell into the phpunit container and run the following command:

vendor/bin/phpunit tests/Test/Benchmark --group Benchmark

Once the test completes, it should generate a benchmark.csv file in the /var/www/html directory that you can inspect or import into your favorite spreadsheet for analysis. I've imported a sample run into Google Sheets here for convenience.

Proposed Solution

Optimize network communication with Redis such that all update and collect operations occur within a single transaction.

Changelist

• Adds RedisTxn storage adapter and supporting classes which:
  • Optimizes network communication with Redis
  • Implements TTL support for all metric types
• Adds a Benchmark test framework
• Adds/updates test coverage for new RedisTxn storage adapter
• Updates php-fpm Dockerfile to vendor Composer dependencies into the Docker image

Benchmark Timings

Screenshot taken from this spreadsheet which also contains the source timing data.

[arris-ray]

This adapter is based on RedisNg and currently, the only changes are in the following methods:

• updateSummary()
• collectSummaries()

If there is interest from the repo maintainers, I'd like to apply the same style of refactoring for the remaining metric types:

• counters
• gauges
• histograms

[arris-ray]

All metric types have been optimized. Based on benchmark timing, there may be further opportunity to optimize histograms but that will probably be left as an exercise for a future contributor.

[arris-ray]

@LKaemmerling You seem like an active maintainer on this project and I was curious to gauge level of interest in a change like this PR. No rush but would love feedback on whether y'all think this is worth pursuing or general advice on making this worth your consideration to merge.

[LKaemmerling]

Hey @arris-ray,

this sounds interesting and of course, we (or better said I, as at the moment I'm the only maintainer) are open to every contribution!

[arris-ray]

Thanks for the encouragement, @LKaemmerling! This is probably ready for review when you're available.

Apologies in advance for the massive changeset. If there's anything I can do to help make the review more manageable, please don't hesitate to ask!

[pluk77]

@LKaemmerling Is there a reason this PR is not being merged other than time?