Allow filtering params based on parent keys

[gmalette]

Some parameters on APIs have ambiguous names
that cannot be filtered altogether.

For example, it makes sense not to filter :code in

{ file: { code: '<% source %>' }}

It makes sense however to filter it in

{ credit_card: { code: '424242424242' }}

This PR allows doing so by using

config.filter_parameters += ["credit_card.code"]

[gmalette]

This is an alternative take on #13670

[gmalette]

@tenderlove, you're the last one to have worked on this. Any thoughts?

[arthurnn]

👍

[tenderlove]

param_filter is a hotspot for perf which is why I took care to remove as much logic from call as possible. Unless there is an extremely good reason for this change, I'm 👎.

[gmalette]

I'll benchmark it.

[gmalette]

You were right that it was much slower. The updated patch is a bit larger, but the performances are roughly the same as master.

The params have synthetic depth of N.

2.1.0
1x10 (50000)
                      user     system      total        real
vanilla           4.650000   0.010000   4.660000 (  4.671173)
modified no .     4.620000   0.000000   4.620000 (  4.626721)
modified with .   5.940000   0.000000   5.940000 (  5.947298)


3x10 (5000)
                      user     system      total        real
vanilla          47.190000   0.010000  47.200000 ( 47.302018)
modified no .    48.530000   0.010000  48.540000 ( 48.643803)
modified with .  60.900000   0.010000  60.910000 ( 61.044795)


5x10 (1000)
                      user     system      total        real
vanilla          989.440000   0.710000  990.150000 ( 992.263921)
modified no .   1002.170000   0.050000 1002.220000 (1004.351774)
modified with . 1305.360000   0.910000 1306.270000 (1309.029364)

here's the script for the benchmark: https://gist.github.com/gmalette/9367909

[gmalette]

This has been in production for a while. It slowed the log filtering time by 20-30% depending on where.

Our average is now sitting at 0.12ms when it was previously taking 0.10ms

[gmalette]

Has any thought been given to this? We've been running it for a while now

[rafaelfranca]

cc @tenderlove @jeremy

[arthurnn]

What is the slowest part of the new code? is this going to effect all apps, or only the ones that use this new feature?

[gmalette]

What is the slowest part of the new code?

The slowest part of the new code is the line that assembles the new key and runs the regex. People not using the feature will not be impacted. The slowdown for people not using the feature comes from the management of parent keys for each new level of nesting. As shown in the current benchmarks, the impact is minimal.

Here are benchmarks for Ruby 2.2.1 (same code, but using IPS)

SMALL
Calculating -------------------------------------
     vanilla           844.000  i/100ms
     modified no .     725.000  i/100ms
     modified with .   658.000  i/100ms
-------------------------------------------------
     vanilla              8.838k (± 2.6%) i/s -     44.732k
     modified no .        8.705k (± 3.0%) i/s -     44.225k
     modified with .      7.016k (± 2.9%) i/s -     35.532k


MEDIUM
Calculating -------------------------------------
     vanilla             8.000  i/100ms
     modified no .       8.000  i/100ms
     modified with .     7.000  i/100ms
-------------------------------------------------
     vanilla             86.996  (± 3.4%) i/s -    440.000
     modified no .       85.721  (± 3.5%) i/s -    432.000
     modified with .     70.258  (± 4.3%) i/s -    357.000

LARGE
Calculating -------------------------------------
     vanilla             1.000  i/100ms
     modified no .       1.000  i/100ms
     modified with .     1.000  i/100ms
-------------------------------------------------
     vanilla              0.863  (± 0.0%) i/s -      5.000  in   5.795388s
     modified no .        0.849  (± 0.0%) i/s -      5.000  in   5.888606s
     modified with .      0.651  (± 0.0%) i/s -      4.000  in   6.169669s

[arthurnn]

@gmalette Sounds good to me. Can you add a changelog before I merge it?
thanks

[gmalette]

updated

[arthurnn]

Merci beaucoup .