LG-11671: Lexisnexis Instant Verify workflow A/B test#9743
Merged
Conversation
…ify workflow changelog: Internal, Identity Verification vendors, enable AB test for verification workflow Co-authored-by: Doug Price <douglas.price@gsa.gov>
…erform The job does not otherwise have access to the session uuid.
matthinz
reviewed
Dec 12, 2023
Add new A/B test bucket to welcome and agreement step analytics
…capture_session_uuid This can be removed after the deploy since this is a 50/50 state spec
…g_job for 50/50 state
matthinz
approved these changes
Dec 14, 2023
Contributor
matthinz
left a comment
matthinz
left a comment
There was a problem hiding this comment.
LGTM, just a couple minor naming concerns that shouldn't block
| config.add(:lexisnexis_instant_verify_timeout, type: :float) | ||
| config.add(:lexisnexis_instant_verify_workflow, type: :string) | ||
| config.add(:lexisnexis_instant_verify_workflow_ab_testing_enabled, type: :boolean) | ||
| config.add(:lexisnexis_instant_verify_workflow_ab_testing_percent, type: :integer) |
Contributor
There was a problem hiding this comment.
I always have to 🤔 about whether _percent means the number that gets the "A" variant or the "B" variant. Not something we have to fix in this PR though!
Contributor
There was a problem hiding this comment.
It appears to be the percentage on the alternate, but I always wonder too. Especially since there are comments sprinkled around the Acuant code saying it has no effect? We'll definitely have to check on that in the next PR.
…ator So that people won't get confused about why it has the document_capture_session_uuid
Merged
jmhooper
added a commit
that referenced
this pull request
Dec 14, 2023
…ProofingJob` This commit adds a arg to the `ResolutionProofingJob` which was used in the changes in #9743. That commit was reverted. Old workers did not know about this arg so they raised an ArgumentError when they encountered it. We reverted that change so we could introduce the arg in this one and then revert the revert. [skip changelog]
jmhooper
added a commit
that referenced
this pull request
Dec 14, 2023
When we deployed the changes in #9763 and #9743 a new argument was introduced for the resolution proofing job. This argument was not recognized by old hosts performing the job which led to an argument error. This commit reverts both of those and adds a commit to introduce the argument. Once this commit is merged and deployed we will make another change to use the argument by reverting the reverts. This commit adds a arg to the `ResolutionProofingJob` which was used in the changes in #9743. That commit was reverted. Old workers did not know about this arg so they raised an ArgumentError when they encountered it. We reverted that change so we could introduce the arg in this one and then revert the revert. [skip changelog]
jmhooper
added a commit
that referenced
this pull request
Dec 14, 2023
I reverted 2 changes in #9764 to address a bug in the way the arguments were passed to the ResolutionProofingJob. This commit reverts those 2 reverts to put the A/B test implementation back in place. * Revert "Revert "LG-11671: Lexisnexis Instant Verify workflow A/B test (#9743)"" * Revert "Revert "Capitalization: Rename LexisnexisInstantVerify to LexisNexisInstantVerify (#9763)"" [skip changelog]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎫 Ticket
LG-11671
🛠 Summary of changes
Add infrastructure for LexisNexis Instant Verify alternative workflow A/B test. This will allow us to test a new workflow that does not reject users if their address is not deliverable, which is an issue for some rural addresses and many Puerto Rico and other territory addresses.
The discriminator for the A/B test is the document_capture_session_uuid, which means the same user could get different workflows in different IdV sessions. This would not be visible to them.
Add new class app/services/idv/lexisnexis_instant_verify.rb instead of a concern that's used for the acuant sdk A/B test, because this is called from the proofing job, not from a controller.
Include the new test bucket in IdV analytics.
Existing config value:
lexisnexis_instant_verify_workflow: [valid workflow]
New config values:
lexisnexis_instant_verify_workflow_ab_testing_enabled: false
lexisnexis_instant_verify_workflow_ab_testing_percent: 5
lexisnexis_instant_verify_workflow_alternate: [valid workflow]
📜 Testing Plan