Skip to content

feat: Sharded pub/sub support via dedicated subscribers#1956

Merged
dmaier-redislabs merged 23 commits intomainfrom
dmaier-sharded-pubsub
Mar 5, 2025
Merged

feat: Sharded pub/sub support via dedicated subscribers#1956
dmaier-redislabs merged 23 commits intomainfrom
dmaier-sharded-pubsub

Conversation

@dmaier-redislabs
Copy link
Copy Markdown
Contributor

The previous implementation used one subscriber connection per cluster instance, but the command SSUBSCRIBE <channel> needs to be executed against a node that is responsible for the hash slot that is derived from the channel name.

This PR adds a ClusterSubscriberGroup whereby each subscriber in the group is responsible for one shard.

Comment thread lib/cluster/ConnectionPool.ts
Comment thread lib/cluster/ClusterSubscriberGroup.ts Outdated
@bobymicroby
Copy link
Copy Markdown
Member

There are some tests that need attention as well:

2) cluster:spub/ssub
       should receive messages:
     Uncaught TypeError: Cannot read properties of undefined (reading 'writable')
      at MockServer.write (test/helpers/mock_server.ts:139:11)
      at /home/runner/work/ioredis/ioredis/test/functional/cluster/spub_ssub.ts:26:13
      at tryCatcher (node_modules/standard-as-callback/built/utils.js:12:23)
      at /home/runner/work/ioredis/ioredis/node_modules/standard-as-callback/built/index.js:21:53
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
  3) cluster:spub/ssub
       supports password:
     Uncaught AssertionError: expected undefined to deeply equal 'abc'
      at MockServer.handler (test/functional/cluster/spub_ssub.ts:66:31)
      at JavascriptRedisParser.returnReply (test/helpers/mock_server.ts:96:54)
      at JavascriptRedisParser.execute (node_modules/redis-parser/lib/parser.js:544:14)
      at Socket.<anonymous> (test/helpers/mock_server.ts:113:16)
      at Socket.emit (node:events:517:28)
      at Socket.emit (node:domain:489:12)
      at addChunk (node:internal/streams/readable:368:12)
      at readableAddChunk (node:internal/streams/readable:341:9)
      at Socket.Readable.push (node:internal/streams/readable:278:10)
      at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
      
     ``` 

tishun and others added 6 commits February 26, 2025 12:09
* Resubscribe shard channels

* Polishing
* Rewrote the resubscribe logic for sharded PubSub by making the sharded subscriber group aware of the channels.
* Added and extended the integration test cases in regards to sharded pubsub.
* Skipping the Coveralls build step if the action run is not coming via a PR.
@dmaier-redislabs dmaier-redislabs changed the title Sharded pub/sub support via dedicated subscribers feat: Sharded pub/sub support via dedicated subscribers Mar 5, 2025
@dmaier-redislabs dmaier-redislabs merged commit 963a395 into main Mar 5, 2025
@dmaier-redislabs dmaier-redislabs deleted the dmaier-sharded-pubsub branch March 5, 2025 21:20
github-actions Bot pushed a commit that referenced this pull request Mar 5, 2025
# [5.6.0](v5.5.0...v5.6.0) (2025-03-05)

### Features

* Sharded pub/sub support via dedicated subscribers ([#1956](#1956)) ([963a395](963a395))
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 5, 2025

🎉 This PR is included in version 5.6.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

seta-chuang-0909 added a commit to seta-chuang-0909/socket.io-redis-adapter that referenced this pull request Jan 12, 2026
- 在 util.ts 中添加 isIoRedisCluster 和 hasShardedSubscribers 輔助函數
- 在 sharded-adapter.ts 中添加 ioredis Cluster 配置驗證
- 啟用 [sharded] ioredis cluster 測試(需要 shardedSubscribers: true)
- 更新 package.json 中 ioredis 版本要求到 ^5.9.0
- 更新 README.md 添加 ioredis Cluster 與 sharded pub/sub 的使用說明
- 添加 ioredis Cluster 的 natMap 配置以解決 Docker 環境問題

參考: redis/ioredis#1956
darrachequesne pushed a commit to socketio/socket.io-redis-adapter that referenced this pull request Jan 20, 2026
Support for sharded pub/sub was added in `ioredis@5.9.0`.

Related:

- redis/ioredis#1759
- redis/ioredis#1956
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants