Compatibility with gitlab-sidekiq-fetcher

[fernandomm]

I'm using https://gitlab.com/gitlab-org/sidekiq-reliable-fetch in order to increase reliability and avoid losing jobs.

When a worker is killed ( even with kill -9 ), it will enqueue the job again ( relevant code https://gitlab.com/gitlab-org/sidekiq-reliable-fetch/-/blob/master/lib/sidekiq/base_reliable_fetch.rb#L166 ):

2021-06-19T17:02:07.378Z 6310 TID-ouz3v51om INFO: Cleaning working queues
2021-06-19T17:02:07.583Z 6310 TID-ouz3v51om INFO: {:message=>"Pushed job bcfcbe156e9e63e35eb1fd7f back to queue queue:default", :jid=>"bcfcbe156e9e63e35eb1fd7f", :queue=>"queue:default"}

But depending on the used lock it will be discarded since the lock still exists:

2021-06-19T17:02:07.607Z 17605 TID-os4s1jhi1 JID-bcfcbe156e9e63e35eb1fd7f INFO: start
2021-06-19T17:02:07.611Z 17605 TID-os4s1jhi1 JID-bcfcbe156e9e63e35eb1fd7f uniquejobs-server DIG-uniquejobs:a562be31dac91f44c859be69521ffff4 WARN: Couldn't unlock digest: uniquejobs:a562be31dac91f44c859be69521ffff4, jid: bcfcbe156e9e63e35eb1fd7f
2021-06-19T17:02:07.612Z 17605 TID-os4s1jhi1 JID-bcfcbe156e9e63e35eb1fd7f INFO: done: 0.004 sec

In this case, the lock options are:

class MyWorker
  include Sidekiq::Worker
  sidekiq_options lock: :until_and_while_executing, on_conflict: { client: :log, server: :reschedule }, retry: 0
  
  def perform(args)
     sleep 30
  end
end

So, i'm asking here just in case someone is using a similar setup and has fixed this problem. Is there an easy method of removing the lock when worker dies and before gitlab-sidekiq-fetcher enqueues the job?

The only thing that I could think of is to monkey-patch gitlab-sidekiq-fetcher.

[mhenrixon]

I've been thinking about writing my own fetcher because of these types of problems.

I'll have a look at their fetcher sometime next week. Always did seem like a more reliable fetcher should be part of this gem but I never had time to look into it.