Skip to content

Commit

Permalink
Sidekiq 7: Replace redis by redis-client
Browse files Browse the repository at this point in the history
As discussed in redis/redis-rb#1070 (comment)

Current shortcomings (redis-rb/redis-client#6):

  - No `namespace` support. This one is debatable.
  - No connection `url` support (this will be added very soon)
  - No `reconnect_attempts` support. I'd like to add it, but in a smarter way.
  • Loading branch information
byroot committed Mar 28, 2022
1 parent 9baf907 commit d2a272f
Show file tree
Hide file tree
Showing 30 changed files with 278 additions and 341 deletions.
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ source "https://rubygems.org"
gemspec

gem "rake"
gem "redis-namespace"
gem "redis-client", github: "redis-rb/redis-client"
gem "rails", "~> 7.0"

# Required for Ruby 3.1
Expand Down
15 changes: 9 additions & 6 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
GIT
remote: https://github.com/redis-rb/redis-client.git
revision: 36e3e26be9bcee2a18b6cf735b0fa04d6cff4205
specs:
redis-client (0.0.0)

PATH
remote: .
specs:
sidekiq (7.0.0)
connection_pool (>= 2.2.5)
rack (~> 2.2)
redis (>= 4.5.1)
redis-client

GEM
remote: https://rubygems.org/
Expand Down Expand Up @@ -154,9 +160,6 @@ GEM
zeitwerk (~> 2.5)
rainbow (3.1.1)
rake (13.0.6)
redis (4.6.0)
redis-namespace (1.8.1)
redis (>= 3.0.4)
regexp_parser (2.2.1)
rexml (3.2.5)
rubocop (1.25.1)
Expand Down Expand Up @@ -208,7 +211,7 @@ DEPENDENCIES
net-smtp
rails (~> 7.0)
rake
redis-namespace
redis-client!
sidekiq!
simplecov
sqlite3
Expand All @@ -217,4 +220,4 @@ DEPENDENCIES
yalphabetize

BUNDLED WITH
2.2.32
2.3.9
4 changes: 2 additions & 2 deletions bin/sidekiqload
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ rescue ArgumentError
puts "Signal #{sig} not supported"
end

Sidekiq.redis { |c| c.flushdb }
Sidekiq.redis { |c| c.call("FLUSHDB") }
def handle_signal(launcher, sig)
Sidekiq.logger.debug "Got #{sig} signal"
case sig
Expand Down Expand Up @@ -100,7 +100,7 @@ Monitoring = Thread.new do
while true
sleep 0.2
qsize = Sidekiq.redis do |conn|
conn.llen "queue:default"
conn.call "LLEN", "queue:default"
end
total = qsize
# Sidekiq.logger.error("RSS: #{Process.rss} Pending: #{total}")
Expand Down
18 changes: 6 additions & 12 deletions lib/sidekiq.rb
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def self.options=(opts)
# Configuration for Sidekiq server, use like:
#
# Sidekiq.configure_server do |config|
# config.redis = { :namespace => 'myapp', :size => 25, :url => 'redis://myhost:8877/0' }
# config.redis = { :size => 25, :url => 'redis://myhost:8877/0' }
# config.server_middleware do |chain|
# chain.add MyServerHook
# end
Expand All @@ -76,7 +76,7 @@ def self.configure_server
# Configuration for Sidekiq client, use like:
#
# Sidekiq.configure_client do |config|
# config.redis = { :namespace => 'myapp', :size => 1, :url => 'redis://myhost:8877/0' }
# config.redis = { ::size => 1, :url => 'redis://myhost:8877/0' }
# end
def self.configure_client
yield self unless server?
Expand All @@ -92,14 +92,14 @@ def self.redis
retryable = true
begin
yield conn
rescue Redis::BaseError => ex
rescue RedisClient::Error => ex
# 2550 Failover can cause the server to become a replica, need
# to disconnect and reopen the socket to get back to the primary.
# 4495 Use the same logic if we have a "Not enough replicas" error from the primary
# 4985 Use the same logic when a blocking command is force-unblocked
# The same retry logic is also used in client.rb
if retryable && ex.message =~ /READONLY|NOREPLICAS|UNBLOCKED/
conn.disconnect!
conn.close
retryable = false
retry
end
Expand All @@ -110,14 +110,8 @@ def self.redis

def self.redis_info
redis do |conn|
# admin commands can't go through redis-namespace starting
# in redis-namespace 2.0
if conn.respond_to?(:namespace)
conn.redis.info
else
conn.info
end
rescue Redis::CommandError => ex
conn.call("INFO").lines(chomp: true).map { |l| l.split(":", 2) }.select { |l| l.size == 2 }.to_h
rescue RedisClient::CommandError => ex
# 2850 return fake version when INFO command has (probably) been renamed
raise unless /unknown command/.match?(ex.message)
FAKE_INFO
Expand Down
Loading

0 comments on commit d2a272f

Please sign in to comment.