Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions app/services/populate_email_addresses_table.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
class PopulateEmailAddressesTable
def initialize
@count = 0
@total = 0
end

# :reek:DuplicateMethodCall
def call
User.in_batches(of: 1000) do |relation|
sleep(1)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 second seems like a long time to sleep here. That adds a little of 2 hours of idle time to your job (assuming 8M users). With a batch size of 1000 you could probably do 0.25.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this looks like it isn't making any KMS calls. What is the reason for the limit here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit is to make sure we aren't driving the database too hard. This is the same process we went through when populating the phone configurations database. It does take about 6 hours to do everything, but I just let it run for the day.

process_batch(relation)
Rails.logger.info "#{@count} / #{@total}"
end
Rails.logger.info "Processed #{@count} user email addresses"
end

private

def process_batch(relation)
User.transaction do
relation.each do |user|
@total += 1
next if user.email_address.present? || user.encrypted_email.blank?
user.create_email_address(email_info_for_user(user))
@count += 1
end
end
end

def email_info_for_user(user)
{
encrypted_email: user.encrypted_email,
email_fingerprint: user.email_fingerprint,
confirmed_at: user.confirmed_at,
confirmation_sent_at: user.confirmation_sent_at,
confirmation_token: user.confirmation_token,
}
end
end
7 changes: 7 additions & 0 deletions lib/tasks/migrate_email_addresses.rake
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
namespace :adhoc do
desc 'Copy email addresses to the new table'
task populate_email_addresses: :environment do
Rails.logger = Logger.new(STDOUT)
PopulateEmailAddressesTable.new.call
end
end
49 changes: 49 additions & 0 deletions spec/services/populate_email_addresses_table_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
require 'rails_helper'

describe PopulateEmailAddressesTable do
let(:subject) { described_class.new }

describe '#call' do
context 'a user with no email' do
let!(:user) { create(:user, email: '', confirmed_at: nil) }

it 'migrates nothing' do
expect(user.email_address).to be_nil

expect { subject.call }.to change { EmailAddress.count }.by(0)
end
end

context 'a user with an email' do
let!(:user) { create(:user) }

context 'and no email_address entry' do
before(:each) do
user.email_address.delete
user.reload
end

it 'migrates without decrypting and re-encrypting' do
expect(EncryptedAttribute).to_not receive(:new)
subject.call
end

it 'migrates the email' do
expect { subject.call }.to change { EmailAddress.count }.by(1)

address = user.reload.email_address
expect(user.email).to eq address.email
expect(user.confirmed_at).to eq user.confirmed_at
expect(user.confirmation_sent_at).to eq user.confirmation_sent_at
expect(user.confirmation_token).to eq user.confirmation_token
end
end

context 'and an existing email_address entry' do
it 'adds no new rows' do
expect { subject.call }.to change { EmailAddress.count }.by(0)
end
end
end
end
end