Conversation
Adjust query and add index
| partner_account_status: Agreements::PartnerAccountStatus.find_by(name: 'active'), | ||
| became_partner: ..report_date, | ||
| }). | ||
| distinct |
There was a problem hiding this comment.
this has a 2x speed improvement
| @facial_match_issuers ||= Profile.active.verified.facial_match. | ||
| where('verified_at <= ?', report_date.end_of_day). | ||
| distinct. | ||
| pluck(:initiating_service_provider_issuer) |
There was a problem hiding this comment.
I think this is now 6x faster
| def change | ||
| add_index :profiles, %i[active idv_level verified_at], algorithm: :concurrently, name: 'index_profiles_on_active_and_idv_level_and_verified_at' | ||
| end | ||
| end |
There was a problem hiding this comment.
This shouldn't have too many writes. It should significantly speed up the query above.
There was a problem hiding this comment.
Does the report run fast enough without the index?
There was a problem hiding this comment.
I think so. Are you concerned about the writes for this?
There was a problem hiding this comment.
I think in general we should avoid creating indexes if possible. The current query that would use this runs quickly enough, and we'll hopefully be able to avoid using Postgres for it soon.
Co-authored-by: Mitchell Henke <mitchell.henke@gsa.gov>
Co-authored-by: Zach Margolis <zachmargolis@users.noreply.github.com>
mitchellhenke
force-pushed
the
LG-15119/query-optimization
branch
from
ef9d53c to
49038c6
Compare
| require "rubygems" | ||
| require "bundler/setup" | ||
|
|
||
| load Gem.bin_path("rspec-core", "rspec") |
There was a problem hiding this comment.
helpful to not have to type 'bundle exec'
| Profile.where(active: true).where( | ||
| 'verified_at <= ?', | ||
| end_date, | ||
| Profile.active.where( |
There was a problem hiding this comment.
Updated most of this file to use the available class methods.
n1zyy
left a comment
n1zyy
left a comment
There was a problem hiding this comment.
Nit to update the PR description prior to merge since it says it adds an index but no longer does.
🤜🤛 for both speeding up queries and making the code easier to read.
| pluck(:initiating_service_provider_issuer).uniq | ||
| @facial_match_issuers ||= Reports::BaseReport.transaction_with_timeout do | ||
| Profile.active.verified.facial_match_opt_in. | ||
| where('verified_at <= ?', report_date.end_of_day). |
There was a problem hiding this comment.
Is the verified scope on the prior line redundant with this?
There was a problem hiding this comment.
yeah, it can't be active without verified so idk why I am the way that I am.
4 participants
Adjust queries and add index
Link to the relevant ticket:
LG-15119
🛠 Summary of changes
Noticed a timeout in the PG query. We were pulling all profiles and using ruby .uniq instead of .distinct. This needed wrapped with the read_replica time_out method.
I've also added a new IDV Facial Match level to be IAL2 opt-in and updated the queries with it.