Skip to content

LG-16417 Remove KYC Shadowmode #12550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mawar2 merged 5 commits into from
Oct 7, 2025
Merged

LG-16417 Remove KYC Shadowmode #12550

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1782e52
LG-16417 Remove KYC Shadowmode
Mawar2 Oct 2, 2025
3279269
changelog: Internal, Code cleanup, Remove KYC Shadowmode evaluation code
Mawar2 Oct 2, 2025
e3831fa
update for failing spec
Mawar2 Oct 2, 2025
f5f17a4
address pr feedback
Mawar2 Oct 7, 2025
5e1da41
Remove proofing_components from resolution_proofing_job_spec
Mawar2 Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion app/controllers/concerns/idv/verify_info_concern.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ def shared_update
threatmetrix_session_id: idv_session.threatmetrix_session_id,
request_ip: request.remote_ip,
ipp_enrollment_in_progress: ipp_enrollment_in_progress?,
proofing_components: ProofingComponents.new(idv_session:),
proofing_vendor:,
)

Expand Down
41 changes: 1 addition & 40 deletions app/jobs/resolution_proofing_job.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ def perform(
user_id: nil,
service_provider_issuer: nil,
threatmetrix_session_id: nil,
request_ip: nil,
proofing_components: nil
request_ip: nil
)
timer = JobHelpers::Timer.new

Expand Down Expand Up @@ -77,34 +76,6 @@ def perform(
timing: timer.results,
user_id: user.uuid,
)

if use_shadow_mode?(user:, proofing_components:)
SocureShadowModeProofingJob.perform_later(
document_capture_session_result_id: document_capture_session&.result_id,
encrypted_arguments:,
service_provider_issuer:,
user_email: user_email_for_proofing(user),
user_uuid: user.uuid,
)
end
end

# @param user [User]
# @param proofing_components [Hash,nil]
def use_shadow_mode?(user:, proofing_components:)
# Let idv_socure_shadow_mode_enabled setting control shadow mode globally
disabled_globally = !IdentityConfig.store.idv_socure_shadow_mode_enabled
return false if disabled_globally

# If the user went through Socure docv, they are already a Socure user and
# are thus eligible for shadow mode.
enabled_for_docv_users =
IdentityConfig.store.idv_socure_shadow_mode_enabled_for_docv_users
is_docv_user = proofing_components&.dig(:document_check) == Idp::Constants::Vendors::SOCURE
return true if enabled_for_docv_users && is_docv_user

# Otherwise fall back to A/B test
shadow_mode_ab_test_bucket(user:) == :socure_shadow_mode_for_non_docv_users
end

private
Expand Down Expand Up @@ -163,14 +134,4 @@ def progressive_proofer(user:, proofing_vendor:)
user_uuid: user.uuid, proofing_vendor:, user_email: user_email_for_proofing(user),
)
end

def shadow_mode_ab_test_bucket(user:)
AbTests::SOCURE_IDV_SHADOW_MODE_FOR_NON_DOCV_USERS.bucket(
request: nil,
service_provider: nil,
session: nil,
user:,
user_session: nil,
)
end
end
129 changes: 0 additions & 129 deletions app/jobs/socure_shadow_mode_proofing_job.rb
View file
Open in desktop

This file was deleted.

25 changes: 0 additions & 25 deletions app/services/analytics_events.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5567,31 +5567,6 @@ def idv_socure_reason_code_download(
)
end

# Logs a Socure KYC result alongside a resolution result for later comparison.
# @param [Hash] socure_result Result from Socure KYC API call
# @param [Hash] resolution_result Result from resolution proofing
# @param [String,nil] phone_source Whether the phone number is from MFA or hybrid handoff
def idv_socure_shadow_mode_proofing_result(
socure_result:,
resolution_result:,
phone_source:,
**extra
)
track_event(
:idv_socure_shadow_mode_proofing_result,
resolution_result: resolution_result.to_h,
phone_source:,
socure_result: socure_result.to_h,
**extra,
)
end

# Indicates that no proofing result was found when SocureShadowModeProofingJob
# attempted to look for one.
def idv_socure_shadow_mode_proofing_result_missing(**extra)
track_event(:idv_socure_shadow_mode_proofing_result_missing, **extra)
end

# @param [Boolean] success Whether form validation was successful
# @param [Hash] errors Errors resulting from form validation
# @param [String] exception
Expand Down
4 changes: 0 additions & 4 deletions app/services/idv/agent.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,13 @@ def initialize(applicant)
@applicant = applicant.symbolize_keys
end

# @param document_capture_session [DocumentCaptureSession]
# @param proofing_components [Idv::ProofingComponents]
def proof_resolution(
document_capture_session,
trace_id:,
user_id:,
threatmetrix_session_id:,
request_ip:,
ipp_enrollment_in_progress:,
proofing_components:,
proofing_vendor:
)
document_capture_session.create_proofing_session
Expand All @@ -33,7 +30,6 @@ def proof_resolution(
threatmetrix_session_id:,
request_ip:,
ipp_enrollment_in_progress:,
proofing_components: proofing_components.to_h,
proofing_vendor:,
}

Expand Down
2 changes: 0 additions & 2 deletions app/services/idv/analytics_events_enhancer.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,6 @@ module AnalyticsEventsEnhancer
idv_in_person_usps_proofing_results_job_user_sent_to_fraud_review
idv_ipp_deactivated_for_never_visiting_post_office
idv_socure_reason_code_download
idv_socure_shadow_mode_proofing_result
idv_socure_shadow_mode_proofing_result_missing
idv_socure_verification_data_requested
idv_usps_auth_token_refresh_job_completed
idv_usps_auth_token_refresh_job_network_error
Expand Down
3 changes: 0 additions & 3 deletions config/application.yml.default
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,8 +211,6 @@ idv_socure_reason_code_download_enabled: false
idv_socure_reason_codes_docv_selfie_fail: '[]'
idv_socure_reason_codes_docv_selfie_not_processed: '[]'
idv_socure_reason_codes_docv_selfie_pass: '[]'
idv_socure_shadow_mode_enabled: false
idv_socure_shadow_mode_enabled_for_docv_users: true
idv_sp_required: false
in_person_completion_survey_delivery_enabled: false
in_person_completion_survey_url: 'https://login.gov'
Expand Down Expand Up @@ -457,7 +455,6 @@ socure_docv_webhook_secret_key: ''
socure_docv_webhook_secret_key_queue: '[]'
socure_idplus_api_key: ''
socure_idplus_base_url: ''
socure_idplus_shadow_mode_percent: 0
socure_idplus_timeout_in_seconds: 5
socure_reason_code_api_key: ''
socure_reason_code_base_url: ''
Expand Down
8 changes: 0 additions & 8 deletions config/initializers/ab_tests.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,14 +159,6 @@ def self.all
user&.uuid
end.freeze

SOCURE_IDV_SHADOW_MODE_FOR_NON_DOCV_USERS = AbTest.new(
experiment_name: 'Socure shadow mode',
should_log: ['IdV: doc auth verify proofing results'].to_set,
buckets: {
socure_shadow_mode_for_non_docv_users: IdentityConfig.store.socure_idplus_shadow_mode_percent,
},
).freeze

PROOFING_VENDOR = AbTest.new(
experiment_name: 'Proofing Vendor',
should_log: /^idv/i,
Expand Down
3 changes: 0 additions & 3 deletions lib/identity_config.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,8 +238,6 @@ def self.store
config.add(:idv_socure_reason_codes_docv_selfie_fail, type: :json)
config.add(:idv_socure_reason_codes_docv_selfie_not_processed, type: :json)
config.add(:idv_socure_reason_codes_docv_selfie_pass, type: :json)
config.add(:idv_socure_shadow_mode_enabled, type: :boolean)
config.add(:idv_socure_shadow_mode_enabled_for_docv_users, type: :boolean)
config.add(:idv_sp_required, type: :boolean)
config.add(:idv_aamva_split_last_name_states, type: :json)
config.add(:in_person_completion_survey_delivery_enabled, type: :boolean)
Expand Down Expand Up @@ -481,7 +479,6 @@ def self.store
config.add(:socure_docv_webhook_secret_key, type: :string)
config.add(:socure_idplus_api_key, type: :string)
config.add(:socure_idplus_base_url, type: :string)
config.add(:socure_idplus_shadow_mode_percent, type: :integer)
config.add(:socure_idplus_timeout_in_seconds, type: :integer)
config.add(:socure_reason_code_api_key, type: :string)
config.add(:socure_reason_code_base_url, type: :string)
Expand Down
34 changes: 1 addition & 33 deletions lib/reporting/api_transaction_count_report.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,7 @@ def api_transaction_count
'Instant verify',
'Phone Finder',
'Socure (DocV)',
'Socure (KYC) - Shadow',
'Socure (KYC) - Non-Shadow',
'Socure (KYC)',
'Fraud Score and Attribute',
'Threat Metrix (IDV)',
'Threat Metrix (Auth Only)',
Expand All @@ -85,7 +84,6 @@ def api_transaction_count
instant_verify_table.first,
phone_finder_table.first,
socure_table.first,
socure_kyc_shadow_table.first,
socure_kyc_non_shadow_table.first,
fraud_score_and_attribute_table.first,
threat_metrix_idv_table.first,
Expand Down Expand Up @@ -128,12 +126,6 @@ def socure_kyc_non_shadow_table
[socure_table_count, result]
end

def socure_kyc_shadow_table
result = fetch_results(query: socure_kyc_shadow_query)
socure_table_count = result.count
[socure_table_count, result]
end

def instant_verify_table
result = fetch_results(query: instant_verify_query)
instant_verify_table_count = result.count
Expand Down Expand Up @@ -319,30 +311,6 @@ def fraud_score_and_attribute_query
QUERY
end

def socure_kyc_shadow_query
<<~QUERY
fields
properties.event_properties.socure_result.success as success,
properties.event_properties.socure_result.timed_out as timed_out,
properties.event_properties.socure_result.transaction_id as transaction_id,
properties.event_properties.socure_result.vendor_name as vendor_name,
properties.event_properties.socure_result.verified_attributes.0 as v0,
properties.event_properties.socure_result.verified_attributes.1 as v1,
properties.event_properties.socure_result.verified_attributes.2 as v2,
properties.event_properties.socure_result.verified_attributes.3 as v3,
properties.event_properties.socure_result.verified_attributes.4 as v4,
properties.event_properties.socure_result.verified_attributes.5 as v5,
properties.event_properties.socure_result.errors.I352 as I352,
properties.event_properties.socure_result.errors.I900 as I900,
properties.event_properties.socure_result.errors.I901 as I901,
properties.event_properties.socure_result.errors.I902 as I902,
properties.event_properties.socure_result.errors.I919 as I919,
properties.event_properties.socure_result.errors.R354 as R354
| filter name = "idv_socure_shadow_mode_proofing_result"
| limit 10000
QUERY
end

def socure_kyc_non_shadow_query
<<~QUERY
fields @timestamp, @message, @logStream, @log
Expand Down
Loading