-
Notifications
You must be signed in to change notification settings - Fork 166
A/B testing infrastructure updates #11026
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
4307b22
Rename AbTestBucket to AbTest
matthinz 3ef5128
Refactor ab_test_spec.rb
matthinz 7f0fbb9
Move discriminator calculation into AbTest
matthinz d6f397b
Automatically log AB tests with analytics events.
matthinz 80734a1
Add AbTestingConcern
matthinz 846b611
Update ACUANT_SDK AB test to use new system
matthinz 64a03b7
Update DOC_AUTH_VENDOR A/B test to use new system
matthinz 1031578
Allow more control over what events log A/B tests
matthinz 09c24e8
Limit existing A/B tests to IdV events
matthinz 6bcb83b
Improve use of document_capture_session_uuid as a discriminator
matthinz f1ec176
Limit should_log to RegExp only
matthinz 145caef
Pass acuant_sdk_upgrade_ab_test_bucket into ApiImageUploadForm
matthinz 7578083
Remove stray method accidentally added to Idv::Session
matthinz acb26f9
Fix lint issues in api_image_upload_form_spec.rb
matthinz 2c4f6fd
Remove stray _test_ for method accidentally committed
matthinz f5f1d18
Add test coverage for A/B test initializers
matthinz 7e1d027
Merge remote-tracking branch 'origin/main' into matthinz/ab-test-updates
matthinz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module AbTestingConcern | ||
| # @param [Symbol] test Name of the test, which should correspond to an A/B test defined in | ||
| # # config/initializer/ab_tests.rb. | ||
| # @return [Symbol,nil] Bucket to use for the given test, or nil if the test is not active. | ||
| def ab_test_bucket(test_name) | ||
| test = AbTests.all[test_name] | ||
| raise "Unknown A/B test: #{test_name}" unless test | ||
|
|
||
| test.bucket( | ||
| request:, | ||
| service_provider: current_sp&.issuer, | ||
| session:, | ||
| user: current_user, | ||
| user_session:, | ||
| ) | ||
| end | ||
| end | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,18 +2,12 @@ | |
|
|
||
| module Idv | ||
| module AcuantConcern | ||
| def acuant_sdk_ab_test_analytics_args | ||
| return {} if document_capture_session_uuid.blank? | ||
|
|
||
| { | ||
| acuant_sdk_upgrade_ab_test_bucket: | ||
| AbTests::ACUANT_SDK.bucket(document_capture_session_uuid), | ||
| } | ||
| end | ||
|
Comment on lines
-5
to
-12
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👋🏿 @matthinz ... assuming you're getting rid of this b/c it's noisy?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, bucket will now be present on |
||
| include AbTestingConcern | ||
|
|
||
| def acuant_sdk_upgrade_a_b_testing_variables | ||
| bucket = AbTests::ACUANT_SDK.bucket(document_capture_session_uuid) | ||
| testing_enabled = IdentityConfig.store.idv_acuant_sdk_upgrade_a_b_testing_enabled | ||
| bucket = ab_test_bucket(:ACUANT_SDK) | ||
| testing_enabled = IdentityConfig.store.idv_acuant_sdk_upgrade_a_b_testing_enabled && | ||
| bucket.present? | ||
| use_alternate_sdk = (bucket == :use_alternate_sdk) | ||
|
|
||
| if use_alternate_sdk | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module Idv | ||
| module DocAuthVendorConcern | ||
| include AbTestingConcern | ||
|
|
||
| # @returns[String] String identifying the vendor to use for doc auth. | ||
| def doc_auth_vendor | ||
| bucket = ab_test_bucket(:DOC_AUTH_VENDOR) | ||
| DocAuthRouter.doc_auth_vendor_for_bucket(bucket) | ||
| end | ||
| end | ||
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.