LG-4305: Log new event in case of lockout from proofing

[aduth]

Why: As a login.gov developer, I want to see an event in the event log that indicates that a user was locked out for 6 hours from proofing along with the relevant data points that directly resulted in that lockout, so that I can troubleshoot any issues reported by end users and pinpoint exactly what caused the lock out.

[zachmargolis]

all the other events have lowercase rate limited, can we do that here too? Since cloudwatch regexes are case-sensitive by default it would make it easier for us to find all the rate limits with one query if we wanted to

Suggested change

	IAL2_RECOVERY_REQUEST_RATE_LIMIT_TRIGGERED = 'IAL2 Recovery Request Rate Limited'.freeze
	IAL2_RECOVERY_REQUEST_RATE_LIMIT_TRIGGERED = 'IAL2 Recovery Request rate limited'.freeze

[aduth]

Yeah, this was an unfortunate choice between consistency amongst the peer of "IAL2 Recovery Request" events vs. "rate limited" events. You raise a good point about being able to query rate limited events.

I was actually beginning to explore an alternative approach where the logging happens as part of the throttling behaviors (e.g. in or as a result of Throttler::IsThrottledElseIncrement and Throttler::Increment). To your point, this would have the added benefit of there being only one event associated with rate limits being triggered.

[aduth]

Exploring a couple alternatives:

• afdc8d3 at least consolidates to a single event, per point above, to optimize for discoverability of throttle trigger events
• b09153f moves logging behavior into the Throttler service module
  • This feels better in the sense that the throttler is responsible for its own logging, but doesn't feel so great to be passing around analytics like this. I wonder if there's an argument to be made to allow the service to create its own Analytics instance, regardless if we can carry through all the "base" analytics values like those derived from sp or request.

cc also @amathews-fs , re: our chat yesterday.

Still tinkering here, and finalizing details around tests.

[aduth]

I created a follow-up ticket LG-4404 to log events for non-proofing throttle types.