Skip to content

LG-7663 email analytics for success and failure emails#7156

Merged
svalexander merged 10 commits intomainfrom
shannon/lg-7663-email-sent-analytics
Oct 18, 2022
Merged

LG-7663 email analytics for success and failure emails#7156
svalexander merged 10 commits intomainfrom
shannon/lg-7663-email-sent-analytics

Conversation

@svalexander
Copy link
Contributor

@svalexander svalexander commented Oct 17, 2022

🎫 Ticket

Lg-7663

🛠 Summary of changes

This pr is to add analytics to Cloudwatch around the success and fail emails in the GetUspsProofingResultsJob. Because we are using a service to send the emails we cannot know for sure that they are sent, but we can know when we've initiated/queued the job. When we handle a failed or successful status we call the functions that send the appropriate emails, this pr adds an analytic's event for emails when we handle those statuses. The analytics event is called idv_in_person_usps_proofing_results_job_email_initiated and it accepts email_analytics_attributes. These attributes include:
timestamp: time email was initiated,
user_id: user id attached to the enrollment,
service_provider: found in enrollment's issuer field,
delay_time_seconds: wait time for email to be sent,

📜 Testing Plan

Provide a checklist of steps to confirm the changes.

  • Comment out enrollment_outcomes[:enrollments_passed] += 1 on line 221 and line 247
  • Open a rails console
  • Save user: u = User.last
  • Save enrollment: e = u.in_person_enrollments.last
  • Save json using a mock response: json =
    UspsInPersonProofing::Mock::Fixtures.request_passed_proofing_results_response
  • Parse json: res = JSON.parse(json)
  • Trigger successful update: GetUspsProofingResultsJob.new.send(:handle_successful_status_update, e, res)
  • Go to local events.log file in repo
  • Confirm that event named "GetUspsProofingResultsJob: Success or failure email initiated" was triggered with email version "Success email version".
  • Repeat steps from json to confirmation using mocked failed status and failed fraud status for the json, and handle_failed_status in GetUspsProofingResultsJob.new.send() . Confirm appropriate email versions are shown.

👀 Screenshots

If relevant, include a screenshot or screen capture of the changes.

Success email analytics: Screen Shot 2022-10-17 at 5 24 13 PM Screen Shot 2022-10-17 at 12 58 55 PM
Failed fraud suspected email analytics: Screen Shot 2022-10-17 at 5 19 54 PM Screen Shot 2022-10-17 at 2 19 52 PM
Failed email analytics: Screen Shot 2022-10-17 at 5 14 01 PM Screen Shot 2022-10-17 at 2 24 26 PM

@svalexander svalexander changed the title Shannon/lg 7663 email sent analytics lg-7663 email analytics for success and failure emails Oct 17, 2022
Comment on lines +22 to +27
def email_analytics_attributes(enrollment, delay_time)
{
timestamp: Time.zone.now,
user_id: enrollment.user_id,
service_provider: enrollment.issuer,
delay_time_seconds: delay_time,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the delivery parameter method is in-scope from this method, what would you think about calling it directly rather than expecting it to be passed as a parameter, optionally refactoring out a method to just get the delay value?

Suggested change
def email_analytics_attributes(enrollment, delay_time)
{
timestamp: Time.zone.now,
user_id: enrollment.user_id,
service_provider: enrollment.issuer,
delay_time_seconds: delay_time,
def email_analytics_attributes(enrollment)
{
timestamp: Time.zone.now,
user_id: enrollment.user_id,
service_provider: enrollment.issuer,
delay_time_seconds: mail_delivery_delay_hours.in_seconds,

# Tracks emails that are initiated during GetUspsProofingResultsJob
# @param [String] email_version success, failed or failed fraud
def idv_in_person_usps_proofing_results_job_email_initiated(
email_version:,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure if it was prescribed in the ticket, but "version" here had me thinking something incremented like "v1", "v2". What would you think about a "email name", "email type", "email variant", or "email template" ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, i think email type is clearer than version

Copy link
Contributor

@aduth aduth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple comments, but LGTM 👍

Comment on lines +41 to +58
context 'email_analytics_attributes' do
before(:each) do
stub_request_passed_proofing_results
end
it 'logs message with email analytics attributes' do
freeze_time do
job.perform(Time.zone.now)
expect(job_analytics).to have_logged_event(
'GetUspsProofingResultsJob: Success or failure email initiated',
timestamp: Time.zone.now,
user_id: pending_enrollment.user_id,
service_provider: pending_enrollment.issuer,
delay_time_seconds: 3600,
)
end
end
end

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally email_analytics_attributes seems to be a good candidate for a private method, since we're primarily concerned with each of the specific email types, and the helper method is an implementation detail that we could choose to refactor or remove altogether. As a private method, I don't think we'd need to test it at all in isolation like this, but I'd suggest enhancing the assertions below for the specific email types to ensure they include all the details we're looking for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i separated out the email attributes from the earlier test for job analytics so I could use a completed enrollment rather then the pending enrollment used for the test for the other job analytics.

send_failed_fraud_email(enrollment.user, enrollment)
analytics(user: enrollment.user).idv_in_person_usps_proofing_results_job_email_initiated(
**email_analytics_attributes(enrollment),
email_type: 'Failed fraud suspected email type',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trailing " email type" in these strings feels unnecessary since it's already part of the property name.

Suggested change
email_type: 'Failed fraud suspected email type',
email_type: 'Failed fraud suspected',

@svalexander svalexander merged commit 8ef5079 into main Oct 18, 2022
@svalexander svalexander deleted the shannon/lg-7663-email-sent-analytics branch October 18, 2022 15:37
@svalexander svalexander changed the title lg-7663 email analytics for success and failure emails LG-7663 email analytics for success and failure emails May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants