Log Face/Touch setup A/B test from session value#11549
Merged
Conversation
changelog: Internal, A/B Tests, Fix logging for A/B test to recommend platform authenticator to SMS users
kevinsmaster5
approved these changes
Nov 25, 2024
Contributor
kevinsmaster5
left a comment
There was a problem hiding this comment.
Local testing behaves as promised.
Looks good!
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎫 Ticket
Related to LG-14191
🛠 Summary of changes
Updates logging to include an additional property in the "Multi-Factor Authentication Setup" event corresponding to whether the user was presented the recommendation for SMS users to set up Face or Touch Unlock.
A planned CloudWatch query was proposed in the content of #11496, but it did not account for all of the additional conditions that apply for whether the user was part of the test group. The approach here adds a session value to accurately compute the denominators.
Revised query:
Recommendation in account creation:
Recommendation in authentication:
📜 Testing Plan
Verify that A/B test value is included in "Multi-Factor Authentication Setup" if and only if you were actually presented recommendation:
Configure A/B test:
make watch_eventsin a separate terminal processmake watch_eventsterminal process. There should be awebauthn_platform_recommended: 'account_creation'value only if you saw the Face/Touch recommendation at Step 7