Redis 集群

10.0.0.101,10.0.0.102,10.0.0.103 - 集群master
10.0.0.104,10.0.0.105,10.0.0.106 - 集群slave

# 修改集群配置
sed -i.bak -e '/masterauth/a masterauth 123456' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file nodes-6379.conf/a cluster-config-file nodes-6379.conf' -e '/cluster-requirefull-coverage yes/c cluster-require-full-coverage no' /apps/redis/etc/redis.conf

# 配置集群
redis-cli -a 123456 --cluster create 10.0.0.101:6379 10.0.0.102:6379 10.0.0.103:6379 10.0.0.104:6379 10.0.0.105:6379 10.0.0.106:6379 --cluster-replicas 1

故障自愈之前，105是101的从
[20:33:22 root@localhost ~]#redis-cli -a 123456 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379@16379 master - 0 1633610227401 2 connected 5461-10922
9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379@16379 slave 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 0 1633610227000 1 connected
0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379@16379 slave 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 0 1633610226371 2 connected
fe7ea30427e3a02b838958a27f415dba7dd72da3 10.0.0.104:6379@16379 slave 98c0b7a483cddb733f40df7fdb02cee30766b52c 0 1633610227000 3 connected
98c0b7a483cddb733f40df7fdb02cee30766b52c 10.0.0.103:6379@16379 myself,master - 0 1633610226000 3 connected 10923-16383
8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379@16379 master - 0 1633610228408 1 connected 0-5460


故障自愈以后，105变主，101变成105的从
[20:57:48 root@localhost ~]#redis-cli -a 123456 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379@16379 master - 0 1633611468000 7 connected 0-5460
2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379@16379 master - 0 1633611466000 2 connected 5461-10922
0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379@16379 myself,slave 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 0 1633611467000 2 connected
8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379@16379 slave 9f222d9116edd65a1850aac65c1cdceb15159bfa 0 1633611468521 7 connected
98c0b7a483cddb733f40df7fdb02cee30766b52c 10.0.0.103:6379@16379 master - 0 1633611469533 3 connected 10923-16383
fe7ea30427e3a02b838958a27f415dba7dd72da3 10.0.0.104:6379@16379 slave 98c0b7a483cddb733f40df7fdb02cee30766b52c 0 1633611467511 3 connected

[20:37:48 root@localhost ~]#redis-cli -a 123456 --cluster check 10.0.0.103:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.103:6379 (98c0b7a4...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.102:6379 (2be4bfee...) -> 1 keys | 5462 slots | 1 slaves.
10.0.0.105:6379 (9f222d91...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.103:6379)
M: 98c0b7a483cddb733f40df7fdb02cee30766b52c 10.0.0.103:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
S: fe7ea30427e3a02b838958a27f415dba7dd72da3 10.0.0.104:6379
   slots: (0 slots) slave
   replicates 98c0b7a483cddb733f40df7fdb02cee30766b52c
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa

### 缩容
# 把103的slot分配给其他主节点机器
[21:18:34 root@localhost ~]#redis-cli -a 123456 --cluster reshard 10.0.0.103:6379

# 分配所有的slot以后，删除当前集群的103节点
[21:18:34 root@localhost ~]#redis-cli -a 123456 --cluster del-node 10.0.0.102:6379 98c0b7a483cddb733f40df7fdb02cee30766b52c

# 删除一行，它的slave变成其他主的从节点
[21:12:27 root@localhost ~]#redis-cli -a 123456 --cluster check 10.0.0.102:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.102:6379 (2be4bfee...) -> 1 keys | 7282 slots | 1 slaves.
10.0.0.105:6379 (9f222d91...) -> 0 keys | 9102 slots | 2 slaves.
[OK] 1 keys in 2 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.102:6379)
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[5461-12742] (7282 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[0-5460],[12743-16383] (9102 slots) master
   2 additional replica(s)
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa
S: fe7ea30427e3a02b838958a27f415dba7dd72da3 10.0.0.104:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa

# 删除从节点
[20:37:48 root@localhost ~]#redis-cli -a 123456 --cluster del-node 10.0.0.102:6379 fe7ea30427e3a02b838958a27f415dba7dd72da3

# 删除主节点和从节点的node.conf文件
[21:18:34 root@localhost ~]#rm -f /apps/redis/data/nodes-6379.conf

# 到这里，所有缩容完成
[21:12:38 root@localhost ~]#redis-cli -a 123456 --cluster check 10.0.0.102:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.102:6379 (2be4bfee...) -> 1 keys | 7282 slots | 1 slaves.
10.0.0.105:6379 (9f222d91...) -> 0 keys | 9102 slots | 1 slaves.
[OK] 1 keys in 2 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.102:6379)
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[5461-12742] (7282 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[0-5460],[12743-16383] (9102 slots) master
   1 additional replica(s)
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa


## 扩容

[21:23:40 root@localhost ~]#redis-cli -a 123456 --cluster add-node 10.0.0.103:6379 10.0.0.102:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 10.0.0.103:6379 to cluster 10.0.0.102:6379
>>> Performing Cluster Check (using node 10.0.0.102:6379)
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[5461-12742] (7282 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[0-5460],[12743-16383] (9102 slots) master
   1 additional replica(s)
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.0.0.103:6379 to make it join the cluster.
[OK] New node added correctly.

# 此时已经加入到集群，但是无slave和slots，仅仅是一个master
[21:25:34 root@localhost ~]#redis-cli -a 123456 --cluster info 10.0.0.103:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.103:6379 (98c0b7a4...) -> 0 keys | 0 slots | 0 slaves.
10.0.0.102:6379 (2be4bfee...) -> 1 keys | 7282 slots | 1 slaves.
10.0.0.105:6379 (9f222d91...) -> 0 keys | 9102 slots | 1 slaves.

# 分配slot

[21:31:01 root@localhost ~]#redis-cli -a 123456 --cluster reshard 10.0.0.103:6379

[21:30:58 root@localhost ~]#redis-cli -a 123456 --cluster check 10.0.0.103:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.103:6379 (98c0b7a4...) -> 0 keys | 5462 slots | 0 slaves.
10.0.0.102:6379 (2be4bfee...) -> 1 keys | 5462 slots | 1 slaves.
10.0.0.105:6379 (9f222d91...) -> 0 keys | 5460 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.103:6379)
M: 98c0b7a483cddb733f40df7fdb02cee30766b52c 10.0.0.103:6379
   slots:[0-3641],[5461-7280] (5462 slots) master
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[7281-12742] (5462 slots) master
   1 additional replica(s)
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[3642-5460],[12743-16383] (5460 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


# 加入从节点
[21:31:42 root@localhost ~]#redis-cli -a 123456 --cluster add-node 10.0.0.104:6379 10.0.0.102:6379 --cluster-slave --cluster-master-id 98c0b7a483cddb733f40df7fdb02cee30766b52c
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 10.0.0.104:6379 to cluster 10.0.0.102:6379
>>> Performing Cluster Check (using node 10.0.0.102:6379)
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[7281-12742] (5462 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
M: 98c0b7a483cddb733f40df7fdb02cee30766b52c 10.0.0.103:6379
   slots:[0-3641],[5461-7280] (5462 slots) master
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[3642-5460],[12743-16383] (5460 slots) master
   1 additional replica(s)
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.0.0.104:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 10.0.0.103:6379.
[OK] New node added correctly.


# 查看集群
[21:34:10 root@localhost ~]#redis-cli -a 123456 --cluster check 10.0.0.103:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.0.103:6379 (98c0b7a4...) -> 0 keys | 5462 slots | 1 slaves.
10.0.0.102:6379 (2be4bfee...) -> 1 keys | 5462 slots | 1 slaves.
10.0.0.105:6379 (9f222d91...) -> 0 keys | 5460 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.103:6379)
M: 98c0b7a483cddb733f40df7fdb02cee30766b52c 10.0.0.103:6379
   slots:[0-3641],[5461-7280] (5462 slots) master
   1 additional replica(s)
M: 2be4bfee4bbcff66ed4678b364e1acc0a0de689b 10.0.0.102:6379
   slots:[7281-12742] (5462 slots) master
   1 additional replica(s)
M: 9f222d9116edd65a1850aac65c1cdceb15159bfa 10.0.0.105:6379
   slots:[3642-5460],[12743-16383] (5460 slots) master
   1 additional replica(s)
S: 0ea52b759530416a9d3a05c9bf3e7445c298eda6 10.0.0.106:6379
   slots: (0 slots) slave
   replicates 2be4bfee4bbcff66ed4678b364e1acc0a0de689b
S: fe7ea30427e3a02b838958a27f415dba7dd72da3 10.0.0.104:6379
   slots: (0 slots) slave
   replicates 98c0b7a483cddb733f40df7fdb02cee30766b52c
S: 8216763e1d73c191f6ec1e7e4f6fae090a72abbc 10.0.0.101:6379
   slots: (0 slots) slave
   replicates 9f222d9116edd65a1850aac65c1cdceb15159bfa
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[21:34:13 root@localhost ~]#redis-cli -a 123456 --no-auth-warning cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:10
cluster_my_epoch:10
cluster_stats_messages_ping_sent:2969
cluster_stats_messages_pong_sent:24699
cluster_stats_messages_meet_sent:2
cluster_stats_messages_auth-ack_sent:1
cluster_stats_messages_update_sent:9
cluster_stats_messages_sent:27680
cluster_stats_messages_ping_received:2851
cluster_stats_messages_pong_received:8432
cluster_stats_messages_fail_received:1
cluster_stats_messages_auth-req_received:1
cluster_stats_messages_received:11285