Skip to content

LG-16117 Create A/B test toggle to support manual upload removal #12139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
theabrad merged 4 commits into from
May 6, 2025
Merged

LG-16117 Create A/B test toggle to support manual upload removal #12139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
caf9e3d
add a/b test for disabling manual document upload
theabrad May 5, 2025
4b088f2
add changelog
theabrad May 5, 2025
fd708de
change ab testing name to add enabled at the end
theabrad May 6, 2025
3cda194
default percent to 0
theabrad May 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions config/application.yml.default
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ doc_auth_client_sharpness_threshold: 50
doc_auth_error_dpi_threshold: 290
doc_auth_error_glare_threshold: 40
doc_auth_error_sharpness_threshold: 40
doc_auth_manual_upload_disabled_a_b_testing_enabled: false
doc_auth_manual_upload_disabled_a_b_testing_percent: 0
doc_auth_max_attempts: 5
doc_auth_max_capture_attempts_before_native_camera: 3
doc_auth_max_submission_attempts_before_native_camera: 3
Expand Down
15 changes: 15 additions & 0 deletions config/initializers/ab_tests.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,19 @@ def self.all
) do |service_provider:, session:, user:, user_session:, **|
user&.uuid
end.freeze

DOC_AUTH_MANUAL_UPLOAD_DISABLED = AbTest.new(
Copy link
Copy Markdown
Contributor

@amirbey amirbey May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
DOC_AUTH_MANUAL_UPLOAD_DISABLED = AbTest.new(
DOC_AUTH_MANUAL_UPLOAD_ENABLED = AbTest.new(

for readability, i prefer affirmative as opposed to having to later use/read double negatives( ie: !disabled)... could choose to rename throughout PR, but your call

Copy link
Copy Markdown
Contributor Author

@theabrad theabrad May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Would we have to change the percentage around for the affirmative if we change to enabled? For example with the disabled name it would be doc_auth_manual_upload_disabled_percent: 15. Would we use doc_auth_manual_upload_enabled_percent: 85 instead?

Copy link
Copy Markdown
Contributor

@amirbey amirbey May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i tend to default percentages to 0 .... and i see your point

ultimately... this is temporary ... i think it's fine to leave as is 👍🏿

experiment_name: 'Doc Auth Manual Upload Disabled',
should_log: [
'Idv: doc auth hybrid handoff visited',
'IdV: doc auth document_capture visited',
],
buckets: {
manual_upload_disabled:
IdentityConfig.store.doc_auth_manual_upload_disabled_a_b_testing_enabled ?
IdentityConfig.store.doc_auth_manual_upload_disabled_a_b_testing_percent : 0,
},
) do |service_provider:, session:, user:, user_session:, **|
user&.uuid
end.freeze
end
2 changes: 2 additions & 0 deletions lib/identity_config.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,8 @@ def self.store
config.add(:doc_auth_error_glare_threshold, type: :integer)
config.add(:doc_auth_error_sharpness_threshold, type: :integer)
config.add(:doc_auth_max_attempts, type: :integer)
config.add(:doc_auth_manual_upload_disabled_a_b_testing_enabled, type: :boolean)
config.add(:doc_auth_manual_upload_disabled_a_b_testing_percent, type: :integer)
config.add(:doc_auth_max_capture_attempts_before_native_camera, type: :integer)
config.add(:doc_auth_max_submission_attempts_before_native_camera, type: :integer)
config.add(:doc_auth_passports_enabled, type: :boolean)
Expand Down
24 changes: 24 additions & 0 deletions spec/config/initializers/ab_tests_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -433,4 +433,28 @@

it_behaves_like 'an A/B test that uses user_uuid as a discriminator'
end

describe 'DOC_AUTH_MANUAL_UPLOAD_DISABLED' do
let(:ab_test) { :DOC_AUTH_MANUAL_UPLOAD_DISABLED }

let(:enable_ab_test) do
-> {
allow(IdentityConfig.store)
.to receive(:doc_auth_manual_upload_disabled_a_b_testing_enabled)
.and_return(true)
allow(IdentityConfig.store)
.to receive(:doc_auth_manual_upload_disabled_a_b_testing_percent)
.and_return(50)
}
end

let(:disable_ab_test) do
-> {
allow(IdentityConfig.store).to receive(:doc_auth_manual_upload_disabled_a_b_testing_enabled)
.and_return(false)
}
end

it_behaves_like 'an A/B test that uses user_uuid as a discriminator'
end
end