本文学习redis中集群模式，通过Redis Cluster部署拓扑进行水平扩展

1. Redis Cluster相关配置

# 设置redis实例能够加入集群
cluster-enabled yes
# 集群配置文件，由redis节点创建和更新
cluster-config-file nodes-6379.conf
# 节点超时时间15s
cluster-node-timeout 15000
# 集群内部通信的端口，即集群总线端口
cluster-port 16379
# 为了获得最大可用性，设置为0，这意味着副本将始终尝试故障转移
# 否则(node-timeout * cluster-replica-validity-factor) + repl-ping-replica-period时间后才会故障转移
cluster-replica-validity-factor 0
# 主节点至少拥有1个副本则宕机后副本才会故障迁移
cluster-migration-barrier 1
# 允许自动故障迁移
cluster-allow-replica-migration yes
# 即使当部分插槽不可用集群仍可用
cluster-require-full-coverage no
# 副本可以自动故障迁移
cluster-replica-no-failover no
# 集群挂掉不可读
cluster-allow-reads-when-down no# 以下为docker部署特别指定
cluster-announce-ip 192.168.1.5
cluster-announce-port 6379
cluster-announce-bus-port 16379

以上配置我们假定为redis6379.conf、对于redis6380.conf、redis6381.conf、redis6382.conf、redis6383.conf、redis6384.conf，我们修改为对应端口即可，如下所示

cluster-announce-ip 192.168.1.5
cluster-announce-port 6380
cluster-announce-bus-port 16380

关注公众号 算法小生，回复 redis集群 即可获取对应代码详细配置

2. docker-compose配置及集群启动

version: '3'
services:redis6379:container_name: redis6379image: redis:7.0.5-alpine3.16ports:- 6379:6379- 16379:16379command: redis-server /usr/local/redis/conf/redis.confvolumes:- ./conf/redis6379.conf:/usr/local/redis/conf/redis.confsysctls:- net.core.somaxconn=1024privileged: trueenvironment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8redis6380:container_name: redis6380image: redis:7.0.5-alpine3.16ports:- 6380:6379- 16380:16379command: redis-server /usr/local/redis/conf/redis.confvolumes:- ./conf/redis6380.conf:/usr/local/redis/conf/redis.confsysctls:- net.core.somaxconn=1024privileged: trueenvironment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8redis6381:container_name: redis6381image: redis:7.0.5-alpine3.16ports:- 6381:6379- 16381:16379command: redis-server /usr/local/redis/conf/redis.confvolumes:- ./conf/redis6381.conf:/usr/local/redis/conf/redis.confsysctls:- net.core.somaxconn=1024privileged: trueenvironment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8redis6382:container_name: redis6382image: redis:7.0.5-alpine3.16ports:- 6382:6379- 16382:16379command: redis-server /usr/local/redis/conf/redis.confvolumes:- ./conf/redis6382.conf:/usr/local/redis/conf/redis.confsysctls:- net.core.somaxconn=1024privileged: trueenvironment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8redis6383:container_name: redis6383image: redis:7.0.5-alpine3.16ports:- 6383:6379- 16383:16379command: redis-server /usr/local/redis/conf/redis.confvolumes:- ./conf/redis6383.conf:/usr/local/redis/conf/redis.confsysctls:- net.core.somaxconn=1024privileged: trueenvironment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8redis6384:container_name: redis6384image: redis:7.0.5-alpine3.16ports:- 6384:6379- 16384:16379command: redis-server /usr/local/redis/conf/redis.confvolumes:- ./conf/redis6384.conf:/usr/local/redis/conf/redis.confsysctls:- net.core.somaxconn=1024privileged: trueenvironment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8

执行docker-compose up -d启动redis集群

shenjian\redis\cluster>docker-compose up -d
WARNING: Found orphan containers (es3, kibana, es1) for this project. If you removed or renamed this service in your compose file, you can run this command with th
e --remove-orphans flag to clean it up.
Creating redis6382 ... done
Creating redis6379 ... done
Creating redis6384 ... done
Creating redis6380 ... done
Creating redis6383 ... done
Creating redis6381 ... done

我们连接6379节点，执行以下redis-cli --askpass -p 6379 cluster nodes命令查看节点集群信息

/data # redis-cli --askpass -p 6379 cluster nodes
Please input password: ******
e8392606e948e6e33183119a93b7b3011f5b9407 192.168.1.5:6379@16379 myself,master - 0 0 0 connected

执行以下命令进行集群创建

/data #  redis-cli --cluster create --cluster-replicas 1 192.168.1.5:6379 192.168.1.5:6380 192.168.1.5:6381 192.168.1.5:
6382 192.168.1.5:6383 192.168.1.5:6384 --askpass
Please input password: ******
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.1.5:6383 to 192.168.1.5:6379
Adding replica 192.168.1.5:6384 to 192.168.1.5:6380
Adding replica 192.168.1.5:6382 to 192.168.1.5:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: e8392606e948e6e33183119a93b7b3011f5b9407 192.168.1.5:6379slots:[0-5460] (5461 slots) master
M: 3e55473bdff571c26761998a499da50dd743082b 192.168.1.5:6380slots:[5461-10922] (5462 slots) master
M: 76d7a639bc4849c60fbb0986389c8cf0a335e064 192.168.1.5:6381slots:[10923-16383] (5461 slots) master
S: 896b2f4d563abb071758506544b0db41365ffb3e 192.168.1.5:6382replicates e8392606e948e6e33183119a93b7b3011f5b9407
S: e89bbe523c5854a056356400960034c1715c11ee 192.168.1.5:6383replicates 3e55473bdff571c26761998a499da50dd743082b
S: 42a6b0e0f2df8a95fafa3d52d01a41e1b9310581 192.168.1.5:6384replicates 76d7a639bc4849c60fbb0986389c8cf0a335e064
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 192.168.1.5:6379)
M: e8392606e948e6e33183119a93b7b3011f5b9407 192.168.1.5:6379slots:[0-5460] (5461 slots) master1 additional replica(s)
S: e89bbe523c5854a056356400960034c1715c11ee 192.168.1.5:6383slots: (0 slots) slavereplicates 3e55473bdff571c26761998a499da50dd743082b
S: 42a6b0e0f2df8a95fafa3d52d01a41e1b9310581 192.168.1.5:6384slots: (0 slots) slavereplicates 76d7a639bc4849c60fbb0986389c8cf0a335e064
S: 896b2f4d563abb071758506544b0db41365ffb3e 192.168.1.5:6382slots: (0 slots) slavereplicates e8392606e948e6e33183119a93b7b3011f5b9407
M: 76d7a639bc4849c60fbb0986389c8cf0a335e064 192.168.1.5:6381slots:[10923-16383] (5461 slots) master1 additional replica(s)
M: 3e55473bdff571c26761998a499da50dd743082b 192.168.1.5:6380slots:[5461-10922] (5462 slots) master1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

再次执行集群查看命令

/data # redis-cli --askpass -p 6379 cluster nodes
Please input password: ******
e8392606e948e6e33183119a93b7b3011f5b9407 192.168.1.5:6379@16379 myself,master - 0 1669455837000 1 connected 0-5460
e89bbe523c5854a056356400960034c1715c11ee 192.168.1.5:6383@16383 slave 3e55473bdff571c26761998a499da50dd743082b 0 1669455833031 2 connected
42a6b0e0f2df8a95fafa3d52d01a41e1b9310581 192.168.1.5:6384@16384 slave 76d7a639bc4849c60fbb0986389c8cf0a335e064 0 1669455838063 3 connected
896b2f4d563abb071758506544b0db41365ffb3e 192.168.1.5:6382@16382 slave e8392606e948e6e33183119a93b7b3011f5b9407 0 1669455837056 1 connected
76d7a639bc4849c60fbb0986389c8cf0a335e064 192.168.1.5:6381@16381 master - 0 1669455835000 3 connected 10923-16383
3e55473bdff571c26761998a499da50dd743082b 192.168.1.5:6380@16380 master - 0 1669455836051 2 connected 5461-10922

我们可以看到3主3从集群已经创建成功，0-16383总共16384个插槽已经分配完毕，并且可以看到

• 0-5460插槽分配到主节点192.168.1.5:6379@16379
• 5461-10922插槽分配到主节点192.168.1.5:6380@16380
• 10923-16383插槽分配到主节点 192.168.1.5:6381@16381

3. 数据写入插槽

# 连接到集群
/data # redis-cli -c -p 6379 --askpass
Please input password: ******
# 设置键值，可以看到被分配到14315插槽位于192.168.1.5:6381
127.0.0.1:6379> set username shenjian
-> Redirected to slot [14315] located at 192.168.1.5:6381
OK
192.168.1.5:6381> set age 26
-> Redirected to slot [741] located at 192.168.1.5:6379
OK
# 多值设置key不在同一个插槽，报错，原因CRC16（key）% 16384得出插槽位置不唯一
192.168.1.5:6380> mset username shenjian age 20
(error) CROSSSLOT Keys in request don't hash to the same slot
# 我们通过哈希标签{}标识指定CRC16（key）% 16384中的key为user,故可设置成功
192.168.1.5:6380> mset username:{user} shenjian age:{user} 20
OK
# 获取值
192.168.1.5:6380> mget username:{user} age:{user}
1) "shenjian"
2) "20"

4. 自动故障转移

我们把6379shutdown,连接其他节点查看集群状态可以看到6379主节点的丛节点192.168.1.5:6382@16382成为了主节点

/data # redis-cli --askpass -p 6379 cluster nodes
Please input password: ******
e89bbe523c5854a056356400960034c1715c11ee 192.168.1.5:6383@16383 slave 3e55473bdff571c26761998a499da50dd743082b 0 1669457824960 2 connected
3e55473bdff571c26761998a499da50dd743082b 192.168.1.5:6380@16380 myself,master - 0 1669457820000 2 connected 5461-10922
76d7a639bc4849c60fbb0986389c8cf0a335e064 192.168.1.5:6381@16381 master - 0 1669457823954 3 connected 10923-16383
42a6b0e0f2df8a95fafa3d52d01a41e1b9310581 192.168.1.5:6384@16384 slave 76d7a639bc4849c60fbb0986389c8cf0a335e064 0 1669457823000 3 connected
e8392606e948e6e33183119a93b7b3011f5b9407 192.168.1.5:6379@16379 master,fail - 1669457796163 1669457795760 1 connected
896b2f4d563abb071758506544b0db41365ffb3e 192.168.1.5:6382@16382 master - 0 1669457822948 7 connected 0-5460

由于我们是单机模拟伪集群，没有进行数据目录挂载，在实际的集群中，请勿忘记数据目录挂载备份