Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Sharded pub/sub support via dedicated subscribers #1956

Merged
merged 23 commits into from
Mar 5, 2025

Conversation

dmaier-redislabs
Copy link
Contributor

The previous implementation used one subscriber connection per cluster instance, but the command SSUBSCRIBE <channel> needs to be executed against a node that is responsible for the hash slot that is derived from the channel name.

This PR adds a ClusterSubscriberGroup whereby each subscriber in the group is responsible for one shard.

@bobymicroby
Copy link
Member

There are some tests that need attention as well:

2) cluster:spub/ssub
       should receive messages:
     Uncaught TypeError: Cannot read properties of undefined (reading 'writable')
      at MockServer.write (test/helpers/mock_server.ts:139:11)
      at /home/runner/work/ioredis/ioredis/test/functional/cluster/spub_ssub.ts:26:13
      at tryCatcher (node_modules/standard-as-callback/built/utils.js:12:23)
      at /home/runner/work/ioredis/ioredis/node_modules/standard-as-callback/built/index.js:21:53
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
  3) cluster:spub/ssub
       supports password:
     Uncaught AssertionError: expected undefined to deeply equal 'abc'
      at MockServer.handler (test/functional/cluster/spub_ssub.ts:66:31)
      at JavascriptRedisParser.returnReply (test/helpers/mock_server.ts:96:54)
      at JavascriptRedisParser.execute (node_modules/redis-parser/lib/parser.js:544:14)
      at Socket.<anonymous> (test/helpers/mock_server.ts:113:16)
      at Socket.emit (node:events:517:28)
      at Socket.emit (node:domain:489:12)
      at addChunk (node:internal/streams/readable:368:12)
      at readableAddChunk (node:internal/streams/readable:341:9)
      at Socket.Readable.push (node:internal/streams/readable:278:10)
      at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
      
     ``` 

tishun and others added 6 commits February 26, 2025 12:09
* Resubscribe shard channels

* Polishing
* Rewrote the resubscribe logic for sharded PubSub by making the sharded subscriber group aware of the channels.
* Added and extended the integration test cases in regards to sharded pubsub.
* Skipping the Coveralls build step if the action run is not coming via a PR.
@dmaier-redislabs dmaier-redislabs changed the title Sharded pub/sub support via dedicated subscribers feat: Sharded pub/sub support via dedicated subscribers Mar 5, 2025
@dmaier-redislabs dmaier-redislabs merged commit 963a395 into main Mar 5, 2025
6 checks passed
@dmaier-redislabs dmaier-redislabs deleted the dmaier-sharded-pubsub branch March 5, 2025 21:20
github-actions bot pushed a commit that referenced this pull request Mar 5, 2025
# [5.6.0](v5.5.0...v5.6.0) (2025-03-05)

### Features

* Sharded pub/sub support via dedicated subscribers ([#1956](#1956)) ([963a395](963a395))
Copy link

github-actions bot commented Mar 5, 2025

🎉 This PR is included in version 5.6.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants