Reduce limit for RemoveOldThrottlesJob#6199
Conversation
app/jobs/remove_old_throttles_job.rb
Outdated
There was a problem hiding this comment.
another option would be to decrease the limit, right?
There was a problem hiding this comment.
that's a much better idea, I will drop all of these changes and halve (?) both limits
app/jobs/remove_old_throttles_job.rb
Outdated
There was a problem hiding this comment.
do we need to ensure and set the timeout back to the default value? I forget the scope of these
mitchellhenke
force-pushed
the
mitchellhenke/longer-timeout-old-throttles-job
branch
from
baba368 to
132b0dd
Compare
changelog: Internal, Maintenance, Reduce limit for background job that cleans up old Throttle database rows
mitchellhenke
force-pushed
the
mitchellhenke/longer-timeout-old-throttles-job
branch
from
132b0dd to
8c385b0
Compare
mitchellhenke
changed the title
| discard_on GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError | ||
|
|
||
| def perform(now, limit: 1000, total_limit: 100_000) | ||
| def perform(now, limit: 500, total_limit: 50_000) |
There was a problem hiding this comment.
maybe even drop this more? since it may be a sparse table, the limit might be expensive to search for
Another idea: we could use a find_in_batches kind of approach that uses id key to search + page (because LIMIT / OFFSET is expensive)
There was a problem hiding this comment.
The index should help here, so I'm ok with trying this for now
There was a problem hiding this comment.
Oh, never mind, I understand now
There was a problem hiding this comment.
Actually, we don't use offset here, do we?
Going to roll forward with this for now.
mitchellhenke
deleted the
mitchellhenke/longer-timeout-old-throttles-job
branch
2 participants
We get some failures every so often because the statement timeout is reached, this increases it to 30 seconds