Background job to log psql table bloat stats

[jgrevich]

The title sums it up, this is in support of https://github.com/18F/identity-devops/issues/4175 to help us log psql table bloat stats. This will help us better calculate how often we need to run table vacuums and if the current configuration is able to keep up with the churn.

Enqueued PsqlStatsJob (Job ID: 18be88e9-1a97-41d7-985e-63d75d6e2bc8) to GoodJob(default)
{"duration_ms":0.034745999073493294,"timestamp":"2022-04-14T03:00:00.108Z","name":"perform_start.active_job","job_class":"PsqlStatsJob","trace_id":null,"queue_name":"GoodJob(default)","job_id":"18be88e9-1a97-41d7-985e-63d75d6e2bc8","enqueued_at":"2022-04-14T03:00:00Z","queued_duration_ms":108.100209}
Performing PsqlStatsJob (Job ID: 18be88e9-1a97-41d7-985e-63d75d6e2bc8) from GoodJob(default) enqueued at 2022-04-14T03:00:00Z
{"name":"psql_bloat","table_data":"[[\"idp\",\"public\",\"account_recovery_requests\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"account_reset_requests\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"agencies\",\"16384.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"agency_identities\",\"8380416.0\",7536640.0,89.93157380254155,100,7536640.0,89.93157380254155,false],[\"idp\",\"public\",\"ar_internal_metadata\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"auth_app_configurations\",\"8192.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"backup_code_configurations\",\"1049600000.0\",802873344.0,76.49326829268293,100,802873344.0,76.49326829268293,false],[\"idp\",\"public\",\"deleted_users\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"devices\",\"7695368192.0\",536797184.0,6.975588049939534,100,536797184.0,6.975588049939534,false],[\"idp\",\"public\",\"doc_auth_logs\",\"1876049920.0\",261357568.0,13.931269376883106,100,261357568.0,13.931269376883106,false],[\"idp\",\"public\",\"doc_auths\",\"1245184.0\",163840.0,13.157894736842104,100,163840.0,13.157894736842104,false],[\"idp\",\"public\",\"document_capture_sessions\",\"1651269632.0\",83845120.0,5.077615331570514,100,83845120.0,5.077615331570514,false],[\"idp\",\"public\",\"email_addresses\",\"62934196224.0\",8913010688.0,14.162428731553446,100,8913010688.0,14.162428731553446,false],[\"idp\",\"public\",\"events\",\"16499449856.0\",1051746304.0,6.374432560959201,100,1051746304.0,6.374432560959201,false],[\"idp\",\"public\",\"good_jobs\",\"401408.0\",196608.0,48.97959183673469,100,196608.0,48.97959183673469,false],[\"idp\",\"public\",\"iaa_gtcs\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"iaa_orders\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"identities\",\"1863450624.0\",146702336.0,7.872617289160864,100,146702336.0,7.872617289160864,false],[\"idp\",\"public\",\"integration_statuses\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"integration_usages\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"integrations\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"letter_requests_to_usps_ftp_logs\",\"16384.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"monthly_auth_counts\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"monthly_sp_auth_counts\",\"546791424.0\",29638656.0,5.420468335655535,100,29638656.0,5.420468335655535,false],[\"idp\",\"public\",\"otp_requests_trackers\",\"913522688.0\",591986688.0,64.8026256792869,100,591986688.0,64.8026256792869,false],[\"idp\",\"public\",\"partner_account_statuses\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"partner_accounts\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"phone_configurations\",\"38549307392.0\",1397071872.0,3.6241166612760707,100,1397071872.0,3.6241166612760707,false],[\"idp\",\"public\",\"phone_number_opt_outs\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"piv_cac_configurations\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"profiles\",\"6280855552.0\",-520347648.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"proofing_components\",\"454672384.0\",57901056.0,12.73467622788368,100,57901056.0,12.73467622788368,false],[\"idp\",\"public\",\"proofing_costs\",\"433963008.0\",21274624.0,4.902404953373353,100,21274624.0,4.902404953373353,false],[\"idp\",\"public\",\"registration_logs\",\"714211328.0\",28344320.0,3.9686180950633143,100,28344320.0,3.9686180950633143,false],[\"idp\",\"public\",\"schema_migrations\",\"16384.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"security_events\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"service_provider_quota_limits\",\"8192.0\",0.0,0.0,100,0.0,0.0,false],[\"idp\",\"public\",\"service_providers\",\"24576.0\",8192.0,33.333333333333336,100,8192.0,33.333333333333336,false],[\"idp\",\"public\",\"sign_in_restrictions\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"sp_costs\",\"10681393152.0\",960241664.0,8.989854135461748,100,960241664.0,8.989854135461748,false],[\"idp\",\"public\",\"sp_return_logs\",\"3583762432.0\",438181888.0,12.226867609510116,100,438181888.0,12.226867609510116,false],[\"idp\",\"public\",\"throttles\",\"536510464.0\",536502272.0,99.99847309595064,100,536502272.0,99.99847309595064,false],[\"idp\",\"public\",\"users\",\"154301120512.0\",17678073856.0,11.456866805205848,100,17678073856.0,11.456866805205848,false],[\"idp\",\"public\",\"usps_confirmation_codes\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"usps_confirmations\",\"0.0\",0.0,0.0,100,0.0,0.0,true],[\"idp\",\"public\",\"webauthn_configurations\",\"8192.0\",0.0,0.0,100,0.0,0.0,false]]"}
{"duration_ms":18.182562998845242,"timestamp":"2022-04-14T03:00:00.126Z","name":"perform.active_job","job_class":"PsqlStatsJob","trace_id":null,"queue_name":"GoodJob(default)","job_id":"18be88e9-1a97-41d7-985e-63d75d6e2bc8","enqueued_at":"2022-04-14T03:00:00Z"}
Performed PsqlStatsJob (Job ID: 18be88e9-1a97-41d7-985e-63d75d6e2bc8) from GoodJob(default) in 18.5ms

[zachmargolis]

Two last changes we need and then it's good to go

[jgrevich]

Thanks @mitchellhenke I also noticed that cron_1m was unexpectedly firing hourly

Changing it to * * * * * seemed to do the trick

[jgrevich]

🚢