Skip to content

LG-14216: Assign reCAPTCHA A/B test bucket for missing user#11210

Merged
aduth merged 1 commit intomainfrom
aduth-recaptcha-anonymous-bucket
Sep 6, 2024
Merged

LG-14216: Assign reCAPTCHA A/B test bucket for missing user#11210
aduth merged 1 commit intomainfrom
aduth-recaptcha-anonymous-bucket

Conversation

@aduth
Copy link
Contributor

@aduth aduth commented Sep 6, 2024

🎫 Ticket

Addendum for LG-14216 (implemented in #11148)

🛠 Summary of changes

Updates the discriminator evaluation for reCAPTCHA at sign-in to randomly assign requests which aren't associated with a user. This provides some resiliency against user existence probing, though there are other mitigations which already provide protection (ping me on Slack if interested).

This test is not currently live in production.

This would likely be very incompatible with #11202 since it would result in persisting records for many random values (although only within the domain of 1 bytes worth of random data), so I'd plan to avoid opting-in reCAPTCHA for persistence if moving forward with these changes.

📜 Testing Plan

Verify that reCAPTCHA is performed based on percent tested when trying to authenticate with a user that doesn't exist:

  1. Configure in config/application.yml:
    development:
      sign_in_recaptcha_percent_tested: 50
    
  2. Restart server, if running
  3. In a fresh private browsing window, go to http://localhost:3000
  4. Sign in to an account that doesn't exist, e.g. abc@example.com, assigning a failing reCAPTCHA score (e.g. 0.1)
  5. Repeat Steps 3 & 4 until you encounter the "Security check failed" screen
  6. Observe that you see the screen on about 50% of attempts

changelog: Upcoming Features, Fraud Prevention, Assign reCAPTCHA A/B test bucket for missing user
@aduth aduth requested a review from a team September 6, 2024 15:55
user.uuid
else
user&.uuid
SecureRandom.gen_random(1)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought process with the selection of method here:

  1. We do want this to be resilient against timing attacks (SecureRandom)
  2. We don't care much about the actual value. SecureRandom.hex does a lot more work than we need

Benchmarks, since I was curious:

require 'benchmark/ips'
require 'securerandom'

Benchmark.ips do |x|
  x.report('gen_random') do
    SecureRandom.gen_random(1)
  end

  x.report('hex') do
    SecureRandom.hex
  end

  x.compare!
end
ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [arm64-darwin21]
Warming up --------------------------------------
          gen_random   486.807k i/100ms
                 hex   169.039k i/100ms
Calculating -------------------------------------
          gen_random      4.867M (± 0.9%) i/s -     24.340M in   5.001117s
                 hex      1.679M (± 1.9%) i/s -      8.452M in   5.035533s

Comparison:
          gen_random:  4867370.6 i/s
                 hex:  1679161.7 i/s - 2.90x  slower

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not exactly a fair test, they'll pulling different numbers of bytes from random? we can close the gap a bit by using SecureRandom.hex(1)

irb(local):005:0> require 'benchmark/ips'
irb(local):006:0> require 'securerandom'
irb(local):007:0> 
irb(local):008:1* Benchmark.ips do |x|
irb(local):009:2*   x.report('gen_random') do
irb(local):010:2*     SecureRandom.gen_random(1)
irb(local):011:1*   end
irb(local):012:1* 
irb(local):013:2*   x.report('hex') do
irb(local):014:2*     SecureRandom.hex(1)
irb(local):015:1*   end
irb(local):016:1* 
irb(local):017:1*   x.compare!
irb(local):018:0> end
irb(local):019:0> 
Warming up --------------------------------------
          gen_random   692.696k i/100ms
                 hex   415.078k i/100ms
Calculating -------------------------------------
          gen_random      6.852M (± 1.7%) i/s -     34.635M in   5.055886s
                 hex      4.109M (± 3.7%) i/s -     20.754M in   5.059781s

Comparison:
          gen_random:  6852411.5 i/s
                 hex:  4109274.2 i/s - 1.67x  slower

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. I think I opted to use it without arguments since the expected pushback I expected for going for something like gen_random(1) is that it's very common and "simple" for us to use SecureRandom.hex (unparameterized) elsewhere in code.

@aduth aduth merged commit cf3e2e9 into main Sep 6, 2024
@aduth aduth deleted the aduth-recaptcha-anonymous-bucket branch September 6, 2024 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants