Skip to content

[LG-588] Rake task populating email addresses#2516

Merged
jgsmith-usds merged 1 commit intomasterfrom
jgs/lg-588-rake-task-populate-email-address-table
Sep 13, 2018
Merged

[LG-588] Rake task populating email addresses#2516
jgsmith-usds merged 1 commit intomasterfrom
jgs/lg-588-rake-task-populate-email-address-table

Conversation

@jgsmith-usds
Copy link
Contributor

Why:
We want to support multiple email addresses per account. This
requires that we store email addresses in their own table.

How:
A rake task to copy existing email information to the new table.

Hi! Before submitting your PR for review, and/or before merging it, please
go through the checklists below. These represent the more critical elements
of our code quality guidelines. The rest of the list can be found in
CONTRIBUTING.md

Controllers

  • When adding a new controller that requires the user to be fully
    authenticated, make sure to add before_action :confirm_two_factor_authenticated
    as the first callback.

Database

  • Unsafe migrations are implemented over several PRs and over several
    deploys to avoid production errors. The strong_migrations gem
    will warn you about unsafe migrations and has great step-by-step instructions
    for various scenarios.

  • Indexes were added if necessary. This article provides a good overview
    of indexes in Rails.

  • Verified that the changes don't affect other apps (such as the dashboard)

  • When relevant, a rake task is created to populate the necessary DB columns
    in the various environments right before deploying, taking into account the users
    who might not have interacted with this column yet (such as users who have not
    set a password yet)

  • Migrations against existing tables have been tested against a copy of the
    production database. See LG-228 Make migrations safer and more resilient #2127 for an example when a migration caused deployment
    issues. In that case, all the migration did was add a new column and an index to
    the Users table, which might seem innocuous.

Encryption

  • The changes are compatible with data that was encrypted with the old code.

Routes

  • GET requests are not vulnerable to CSRF attacks (i.e. they don't change
    state or result in destructive behavior).

Session

  • When adding user data to the session, use the user_session helper
    instead of the session helper so the data does not persist beyond the user's
    session.

Testing

  • Tests added for this feature/bug
  • Prefer feature/integration specs over controller specs
  • When adding code that reads data, write tests for nil values, empty strings,
    and invalid inputs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 second seems like a long time to sleep here. That adds a little of 2 hours of idle time to your job (assuming 8M users). With a batch size of 1000 you could probably do 0.25.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this looks like it isn't making any KMS calls. What is the reason for the limit here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit is to make sure we aren't driving the database too hard. This is the same process we went through when populating the phone configurations database. It does take about 6 hours to do everything, but I just let it run for the day.

**Why**:
We want to support multiple email addresses per account. This
requires that we store email addresses in their own table.

**How**:
A rake task to copy existing email information to the new table.
@jgsmith-usds jgsmith-usds force-pushed the jgs/lg-588-rake-task-populate-email-address-table branch from dcf6c42 to bc05ec1 Compare September 13, 2018 14:55
@jgsmith-usds jgsmith-usds merged commit d2e1d55 into master Sep 13, 2018
@jgsmith-usds jgsmith-usds deleted the jgs/lg-588-rake-task-populate-email-address-table branch September 13, 2018 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants