Team Data 1105(feature) Add Fraud Ops Tracking for data warehouse and FCMS by MrNagoo · Pull Request #12503 · 18F/identity-idp

MrNagoo · 2025-09-16T18:15:45Z

🎫 Ticket

Link to the relevant ticket:
https://gitlab.login.gov/lg-teams/Team-Data/data-warehouse-ag/-/issues/1105

🛠 Summary of changes

Adds the FraudOps::Tracker, custom RedisClient and duplicates most IdV events to be consumed by the FCMS

MrNagoo · 2025-09-22T18:09:39Z

app/controllers/concerns/idv/phone_otp_rate_limitable.rb

@@ -31,6 +31,9 @@ def handle_too_many_otp_sends
      attempts_api_tracker.idv_rate_limited(
        limiter_type: :phone_otp,
      )
+      fraud_ops_tracker.idv_rate_limited(


FYI: I'm only tracking IdV events.

MrNagoo · 2025-09-22T18:15:23Z

app/services/attempts_api/tracker.rb

@@ -62,12 +62,12 @@ def jwe(event)
      )
    end

-    def extra_attributes
+    def extra_attributes(event_type: nil) # rubocop:disable Lint/UnusedMethodArgument


In order to pass event type to the child class to override we have to add all these hoops. I'm fine removing the code and just transforming in the child if need be.

i think this is okay -- i know what you mean about not wanting to alter the parent class, but i sort of like making the changes between the original tracker and the fraud_tracker really explicit (which the extra_attributes method exposes, imo)

MrNagoo · 2025-09-22T18:16:14Z

app/services/fraud_ops/redis_client.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module FraudOps


custom client to give only the feature specific api

Sgtpluck · 2025-09-22T18:29:00Z

app/controllers/application_controller.rb

@@ -91,6 +91,17 @@ def attempts_api_tracker
    )
  end

+  def fraud_ops_tracker
+    @fraud_ops_tracker ||= FraudOps::Tracker.new(
+      session_id: attempts_api_session_id,


i wonder if this will be useful for fraud_tracker? it will only be passed in for requests from attempts api consumers, and is more for them to be able to associate with transactions on their end. on our end, we'll have the hashed session id to associate events for the same user, along with the uuid.

ah, ok, i'll remove that.

Sgtpluck · 2025-09-22T19:11:37Z

app/controllers/sign_up/passwords_controller.rb

      attempts_api_tracker.user_registration_password_submitted(
        success: result.success?,
-        failure_reason:,
+        failure_reason: attempts_api_tracker.parse_failure_reason(result),


[request] can we remove these kinds of style-based changes that aren't necessary in files that wouldn't otherwise be touched? it's making this PR a bit noisier

yes, this might be a remnant of my initial go. I'll take a look. Thanks

Sgtpluck · 2025-09-22T19:12:05Z

app/controllers/sign_up/registrations_controller.rb

@@ -11,12 +11,16 @@ class RegistrationsController < ApplicationController
    CREATE_ACCOUNT = 'create_account'

    def new
-      @register_user_email_form = RegisterUserEmailForm.new(analytics:, attempts_api_tracker:)
+      @register_user_email_form = RegisterUserEmailForm.new(


[request] can we remove these kinds of style-based changes that aren't necessary in files that wouldn't otherwise be touched? it's making this PR a bit noisier

yeah i think it's some over zealous format on save happening. 😬

there's just a couple of them, so i suppose it's not a big deal (i was just noting them as i went through, so wasn't sure how much it was adding to the PR)

Sgtpluck

i think this is generally fine! i wish there was some way to not just duplicate the event calls, but i think any options would get complicated in a hurry, and probably be riskier long-term.

i do think you could probably axe the SP-provided session_id, but it's not blocking imo.

MrNagoo · 2025-09-22T21:15:32Z

i think this is generally fine! i wish there was some way to not just duplicate the event calls, but i think any options would get complicated in a hurry, and probably be riskier long-term.

I agree, there is probably a good use case for pub/sub having multiple trackers. Each entity could respond to broadcasted event's on their own.
https://guides.rubyonrails.org/active_support_instrumentation.html

update specs undo many events update specs (feature) Add Fraud Ops Tracking for data warehouse and FCMS

* Remove unused config setting for approving IPP enrollments (#12477) * Remove unused config setting for approving IPP enrollments I had originally submitted PR #12014 to allow immediate approval of IPP enrollments in `int`, to make it easier for partners to test IPP in the sandbox environment. At the time, I thought this was the only way to do it, after seeing PR #11653 that introduced the ability to manually approve IPP enrollments in local development only. However, it turns out the route that displays the IPP enrollments is behind another config setting that requires enabling test routes. We didn't think it was a good idea to enable test routes in `int`, and we then realized that solving our problem was as easy as changing these two settings that were causing the 1.5 hour delay between completing IPP and receiving the confirmation email: ``` get_usps_proofing_results_job_cron: '0/1 * * * *' in_person_results_delay_in_hours: 0 ``` whereas the default values in our lower environments were: ``` get_usps_proofing_results_job_cron: '0/30 * * * *' in_person_results_delay_in_hours: 1 ``` which resulted in having to wait 1.5 hours to complete IPP testing. changelog: Internal, IPP, Remove unused config setting * Team Data 1105(feature) Add Fraud Ops Tracking for data warehouse and FCMS (#12503) (feature) Add Fraud Ops Tracking for data warehouse and FCMS * changelog: Internal, Reporting, Tracking for Fraud Ops (Team Data-1105) * Handle SAML request errors from saml_idp gem (#12520) Before, the saml_idp gem would return a 403 response if it found specific errors with the SAML request, such as a missing issuer, or a missing AuthnRequest. The problem is that this did not provide any helpful information to partners while testing their integration. To improve this, we updated the `saml_idp` gem to populate the `saml_request` object with errors and not do anything else. This allows the IdP to consume those errors and handle them appropriately, by rendering them in the existing `/saml/auth/error` template. This allows partners to quickly understand what part of their SAML Request is invalid, and makes it easier for them to fix the problem. If they don't know how to fix it themselves, they can let us know the error message they see, which makes it easier and faster for us to help our partners. changelog: User-Facing Improvements, SAML Request Handling, Display SAML request errors so partners can understand what they did wrong * Update cron schedules for reports to improve timing and avoid overlaps (#12531) * changelog: internal, reporting, issue-#1113 stagger jobs Update cron schedules for reports to improve timing and avoid overlaps * Prevent users from skipping choose_id_type (#12502) * Prevent users from skipping ID type selection when accessing document capture directly changelog: Bug Fixes, Identity Verification, Prevent users from skipping ID type selection when accessing document capture directly * Fix failing specs after choose_id_type enforcement * Use session state for choose_id_type enforcement * Prevent hybrid mobile users from skipping choose_id_type step * Update hybrid mobile flow test * Refactor ID type enforcement based on PR feedback * move logic from step_info preconditions to a dedicated before_action * fix failing test * patch failing test * fix issue in mobile flow * prevent users from skipping choose ID type step * Simplify document_capture preconditions and update test helper * address pr feedback * Rewrite the IRS Monthly Credential Metrics Report (#12527) * changelog: Bug Fixes, Reporting, Simplifies execution and logic of the monthly credential report by using a single-partner/single-issuer invocation of the CombinedInvoiceSupplementReportV2 Report Source Changes * Add partner strings config * Refactor data fetching and generation * Add MAU and other fields Email Report Changes * Update corresponding mailer preview invocation * Remove reference to key metrics report in email text * Transpose Data table Spec Updates * Add csv file as a fixture for testing row and column logic * Remove bucket testing since we are relying on default one from BaseReport See https://gitlab.login.gov/lg-teams/Team-Data/reporting/-/issues/222 * GL_Data_reporting: remove cred tenure from Fraud metric (#12534) * changelog: Internal, Reporting, remove credential tenure metric from fraud * changelog: Internal, Reporting, fix lint issue * Lg-16684 HowToverify mobile is always required (#12536) * changelog: Internal, Doc Auth, removing mobile required and updating logic to always have mobile required on how to verify presenter * Updating context title and resolving PR comments * Update pinpoint supported countries (#12539) * Update to reflect drift in footnote numbering changelog: Internal, Pinpoint, Update script to reflect drift in footnote numbering Looks like https://docs.aws.amazon.com/sms-voice/latest/userguide/phone-numbers-sms-by-country.html#sms-support-note-9 became https://docs.aws.amazon.com/sms-voice/latest/userguide/phone-numbers-sms-by-country.html#sms-support-note-8. * Remove country that ceases to exist See https://en.wikipedia.org/wiki/Netherlands_Antilles * Fix flaky feature test (#12526) changelog: Internal, Testing, Fix flaky feature test * LG-16724 Add methods to pii passport and state_id structs (#12533) Added methods to pii passport and state_id structs for determining if residential address is needed and constructing pii address from internal object data. changelog: Internal, Remote IdV, Add methods to pii passport and state_id structs * Enable freezing string literals when running tests in CI (#12541) changelog: Internal, Continuous Integration, Enable freezing string literals when running tests in CI * Bump libphonenumber-js from 1.12.22 to 1.12.23 (#12538) Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.12.22 to 1.12.23. - [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md) - [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.12.22...v1.12.23) --- updated-dependencies: - dependency-name: libphonenumber-js dependency-version: 1.12.23 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * One Account Feature Tests (#12514) * feature tests * sign in spec * changelog: Internal, One Account, Basic Feature tests for flow * spec * update sign in spec * make sure setting is at 100 * make sure to reload ab test --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Moncef Belyamani <moncef.belyamani@gsa.gov> Co-authored-by: Aaron <aaron.nagucki@gmail.com> Co-authored-by: Sree Appachi <guruparan18@users.noreply.github.com> Co-authored-by: Malik Warren <33402370+Mawar2@users.noreply.github.com> Co-authored-by: Gerardo E. Cruz-Ortiz <59618057+astrogeco@users.noreply.github.com> Co-authored-by: shilen <shilen.patel@gsa.gov> Co-authored-by: A Shukla <abir.shukla@gsa.gov> Co-authored-by: Vraj Mohan <vraj.mohan@gsa.gov> Co-authored-by: Shane Chesnutt <shane.chesnutt@gsa.gov> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Malick Diarra <malick.diarra@gsa.gov>

MrNagoo force-pushed the 1105/fraud-ops-replication branch 2 times, most recently from 5c34031 to 683d881 Compare September 18, 2025 18:07

MrNagoo marked this pull request as ready for review September 19, 2025 16:05

MrNagoo requested review from Sgtpluck and mitchellhenke September 19, 2025 16:27

MrNagoo changed the title ~~1105/fraud ops replication~~ Team Data 1105(feature) Add Fraud Ops Tracking for data warehouse and FCMS Sep 19, 2025

MrNagoo commented Sep 22, 2025

View reviewed changes

Sgtpluck reviewed Sep 22, 2025

View reviewed changes

Sgtpluck approved these changes Sep 22, 2025

View reviewed changes

MrNagoo added 5 commits September 24, 2025 12:28

(feature) Add Fraud Ops Tracking for data warehouse and FCMS

55007cb

update specs undo many events update specs (feature) Add Fraud Ops Tracking for data warehouse and FCMS

changelog: Internal, Reporting, Tracking for Fraud Ops (Team Data-1105)

d5d7f4f

test redis url

9945058

update specs

0ca1a40

last test

50550a6

MrNagoo force-pushed the 1105/fraud-ops-replication branch from c22f39d to 50550a6 Compare September 24, 2025 17:28

MrNagoo added 3 commits September 25, 2025 11:05

updates against attemtps_api needs

58ba2f1

update PR comments and attempts session

7a5b203

update lint

dcf73c8

MrNagoo merged commit 0654da7 into main Sep 25, 2025
1 check passed

MrNagoo deleted the 1105/fraud-ops-replication branch September 25, 2025 19:58

mitchellhenke mentioned this pull request Sep 30, 2025

Deploy RC 517 to Production #12542

Merged

astrogeco mentioned this pull request Oct 31, 2025

Further refactor of Attempts API tracker - FCMS Prework #12490

Closed

		@@ -0,0 +1,23 @@
		# frozen_string_literal: true

		module FraudOps

Conversation

MrNagoo commented Sep 16, 2025

🎫 Ticket

🛠 Summary of changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sgtpluck left a comment

Choose a reason for hiding this comment

Uh oh!

MrNagoo commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MrNagoo commented Sep 22, 2025 •

edited

Loading