Redis cluster

一、使用docker 实现 Redis cluster（6节点）

1、创建docker自定义网络

docker network create -d bridge --subnet 172.18.0.0/16 --gateway 172.18.0.1 net-redis

root@docker:~# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
d1db90ab7390        bridge              bridge              local
3100b091b036        host                host                local
ed55e56b4694        net-redis           bridge              local
a225d07fa51f        none                null                local

2、准备6个容器redis配置文件

#!/bin/bash
for port in {1..6};do
        mkdir -p /data/redis/node-${port}/conf
        cat >> /data/redis/node-${port}/conf/redis.conf << EOF
port 6379
bind 0.0.0.0
masterauth 123456
requirepass 123456
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.18.0.1${port}
cluster-announce-port 6379
cluster-announce-bus-port 16379
appendonly yes
EOF
done

root@docker:/data# tree redis/
redis/
├── node-1
│   └── conf
│       └── redis.conf
├── node-2
│   └── conf
│       └── redis.conf
├── node-3
│   └── conf
│       └── redis.conf
├── node-4
│   └── conf
│       └── redis.conf
├── node-5
│   └── conf
│       └── redis.conf
└── node-6
    └── conf
        └── redis.conf

3、使用上面的配置文件创建6个redis容器

#!/bin/bash
for port in {1..6};do
    docker run -p 637${port}:6379 -p 1667${port}:16379 --name redis-${port} \
        -v /data/redis/node-${port}/data:/data \
        -v /data/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
        -d --net net-redis --ip 172.18.0.1${port} redis:5.0.12-alpine3.13 redis-server /etc/redis/redis.conf
done
image.png

4、创建redis cluster
连接到其中一个容器 docker exec -it redis-1 sh 创建redis cluster

redis-cli -a 123456 --cluster create 172.18.0.11:6379 172.18.0.12:6379 172.18.0.13:6379 \
                                     172.18.0.14:6379 172.18.0.15:6379 172.18.0.16:6379 --cluster-replicas 1
image.png
image.png

观察以上结果，可以看到集群创建成功，并分配好主从节点（三主三从）3组master/slave
172.18.0.11 --172.18.0.15
172.18.0.12 -- 172.18.0.16
172.18.0.13 -- 172.18.0.14

查看集群信息

127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:495
cluster_stats_messages_pong_sent:510
cluster_stats_messages_sent:1005
cluster_stats_messages_ping_received:505
cluster_stats_messages_pong_received:495
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1005

添加key，键值已插入至集群不同的节点中

127.0.0.1:6379> set name ken
-> Redirected to slot [5798] located at 172.18.0.12:6379
OK
172.18.0.12:6379> set type xfs
-> Redirected to slot [14919] located at 172.18.0.13:6379
OK

关掉redis-1容器docker stop redis-1，连接进redis-5docker exec -it sh，可以看到redis-5已经接替redis-1成为主节点

redis-cli -a 123456 -c
replication info

# Replication
role:master
connected_slaves:0
master_replid:3fbba25f5a195bf8781568e9e51468dd146f5506
master_replid2:85ef53986b24a64db70d77f3df7ff4b7c306a41d
master_repl_offset:742
second_repl_offset:743
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:742

重新启动redis-1docker start redis-1，并连接进redis-1docker exec -it sh，可以看到redis-1重新连接进集群并成为redis-5的从节点

# Replication
role:slave
master_host:172.18.0.15
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:994
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:3fbba25f5a195bf8781568e9e51468dd146f5506
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:994
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:743
repl_backlog_histlen:252

二、Redis Cluster slot 重新分配

1、查看插槽节点分配redis-cli -a 123456 --cluster check 172.18.0.15:6379

/data # redis-cli -a 123456 --cluster check 172.18.0.15:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.18.0.15:6379 (5c9d6dbd...) -> 0 keys | 5461 slots | 1 slaves.
172.18.0.13:6379 (401b06e7...) -> 1 keys | 5461 slots | 1 slaves.
172.18.0.12:6379 (f89cd2c7...) -> 1 keys | 5462 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.18.0.15:6379)
M: 5c9d6dbd8ace2b8a558f8c78fe41d7b2d4f57846 172.18.0.15:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: f22b4985596209da20effedc368efe6f0d5e5694 172.18.0.14:6379
   slots: (0 slots) slave
   replicates 401b06e79cf06627c4aa70f84e859125f84e835c
S: 5d05f0537288ae6f61e3243f92b71e08896867ca 172.18.0.16:6379
   slots: (0 slots) slave
   replicates f89cd2c7bae63da3bb59add2812a487ec76b8d28
S: 5f1c159a28c641279eb9c571ad857c270507abd0 172.18.0.11:6379
   slots: (0 slots) slave
   replicates 5c9d6dbd8ace2b8a558f8c78fe41d7b2d4f57846
M: 401b06e79cf06627c4aa70f84e859125f84e835c 172.18.0.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: f89cd2c7bae63da3bb59add2812a487ec76b8d28 172.18.0.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

2、准备两个新redis节点的配置文件

#!/bin/bash
for port in {7..8};do
    mkdir -p /data/redis/node-${port}/conf
    cat >> /data/redis/node-${port}/conf/redis.conf << EOF
port 6379
bind 0.0.0.0
masterauth 123456
requirepass 123456
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.18.0.1${port}
cluster-announce-port 6379
cluster-announce-bus-port 16379
appendonly yes
EOF
done

3、创建两个新的redis容器

for port in {7..8};do
    docker run -p 637${port}:6379 -p 1667${port}:16379 --name redis-${port} \
        -v /data/redis/node-${port}/data:/data \
        -v /data/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
        -d --net net-redis --ip 172.18.0.1${port} redis:5.0.12-alpine3.13 redis-server /etc/redis/redis.conf
done
image.png

4、把一台新的节点加入到集群redis-cli -a 123456 --cluster add-node 172.18.0.17:6379 172.18.0.15:6379

image.png

观察到该节点已经加入成功，但此节点上没有slot位,也无从节点，而且新的节点是master


image.png

5、在新的master节点上重新分配槽位

新的node节点加到集群之后,默认是master节点，但是没有slots，需要重新分配添加主机之后需要对添加至集群种的新主机重新分片,否则其没有分片也就无法写入数据。
注：重新分配槽位需要清空数据，所以需要先备份数据，扩展后再恢复数据

redis-cli -a 123456 --cluster reshard 172.18.0.15:6379

image.png

查看槽位重新分配结果，可以看到从原来的三个master节点各抽取了部分划分给新的master

/data # redis-cli -a 123456 --cluster check 172.18.0.11:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.18.0.15:6379 (5c9d6dbd...) -> 0 keys | 4096 slots | 1 slaves.
172.18.0.13:6379 (401b06e7...) -> 1 keys | 4096 slots | 1 slaves.
172.18.0.12:6379 (f89cd2c7...) -> 0 keys | 4096 slots | 1 slaves.
172.18.0.17:6379 (7f90d5fc...) -> 1 keys | 4096 slots | 0 slaves.
[OK] 2 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.18.0.11:6379)
S: 5f1c159a28c641279eb9c571ad857c270507abd0 172.18.0.11:6379
   slots: (0 slots) slave
   replicates 5c9d6dbd8ace2b8a558f8c78fe41d7b2d4f57846
M: 5c9d6dbd8ace2b8a558f8c78fe41d7b2d4f57846 172.18.0.15:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 401b06e79cf06627c4aa70f84e859125f84e835c 172.18.0.13:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 5d05f0537288ae6f61e3243f92b71e08896867ca 172.18.0.16:6379
   slots: (0 slots) slave
   replicates f89cd2c7bae63da3bb59add2812a487ec76b8d28
S: f22b4985596209da20effedc368efe6f0d5e5694 172.18.0.14:6379
   slots: (0 slots) slave
   replicates 401b06e79cf06627c4aa70f84e859125f84e835c
M: f89cd2c7bae63da3bb59add2812a487ec76b8d28 172.18.0.12:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
M: 7f90d5fc14d7c5af4b1ddf9b479abb58e39ad4f6 172.18.0.17:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

5、为新的master添加slave节点，用于解决172.18.0.17节点潜在的宕机问题

redis-cli -a 123456 --cluster add-node 172.18.0.18:6379 172.18.0.15:6379  --cluster-slave --cluster-master-id 7f90d5fc14d7c5af4b1ddf9b479abb58e39ad4f6

查看集群状态，slave节点加入成功

/data # redis-cli -a 123456 --cluster check 172.18.0.11:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.18.0.15:6379 (5c9d6dbd...) -> 0 keys | 4096 slots | 1 slaves.
172.18.0.13:6379 (401b06e7...) -> 1 keys | 4096 slots | 1 slaves.
172.18.0.12:6379 (f89cd2c7...) -> 0 keys | 4096 slots | 1 slaves.
172.18.0.17:6379 (7f90d5fc...) -> 1 keys | 4096 slots | 1 slaves.
[OK] 2 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.18.0.11:6379)
S: 5f1c159a28c641279eb9c571ad857c270507abd0 172.18.0.11:6379
   slots: (0 slots) slave
   replicates 5c9d6dbd8ace2b8a558f8c78fe41d7b2d4f57846
M: 5c9d6dbd8ace2b8a558f8c78fe41d7b2d4f57846 172.18.0.15:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 401b06e79cf06627c4aa70f84e859125f84e835c 172.18.0.13:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 5d05f0537288ae6f61e3243f92b71e08896867ca 172.18.0.16:6379
   slots: (0 slots) slave
   replicates f89cd2c7bae63da3bb59add2812a487ec76b8d28
S: 1eeea1853d8a1f6843e48021f23bd40a513430b3 172.18.0.18:6379
   slots: (0 slots) slave
   replicates 7f90d5fc14d7c5af4b1ddf9b479abb58e39ad4f6
S: f22b4985596209da20effedc368efe6f0d5e5694 172.18.0.14:6379
   slots: (0 slots) slave
   replicates 401b06e79cf06627c4aa70f84e859125f84e835c
M: f89cd2c7bae63da3bb59add2812a487ec76b8d28 172.18.0.12:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
M: 7f90d5fc14d7c5af4b1ddf9b479abb58e39ad4f6 172.18.0.17:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

连接进redis-7，可以看到集群信息，节点已扩容至8个，并且其slave节点为172.18.0.18

127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:8
cluster_size:4
cluster_current_epoch:8
cluster_my_epoch:8
cluster_stats_messages_ping_sent:6211
cluster_stats_messages_pong_sent:6259
cluster_stats_messages_meet_sent:1
cluster_stats_messages_update_sent:12
cluster_stats_messages_sent:12483
cluster_stats_messages_ping_received:6259
cluster_stats_messages_pong_received:6212
cluster_stats_messages_received:12471

# Replication
role:master
connected_slaves:1
slave0:ip=172.18.0.18,port=6379,state=online,offset=1316,lag=0
master_replid:1248438ca0873b04acf7452b6c7704242acf1524
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1316
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:1316

删除集群节点，注：如加入集群失败，可以先删除节点，再删除/data目录下的文件，再重新加入

redis-cli -a 123456 --cluster del-node 172.18.0.18:6379 6e4120dc5cc0e99689d8dfd3fb53faefcc46c26c

三、Redis及Memcached对比

  1. 数据持久化：Redis 可以将内存中的数据保持在磁盘中，重启redis服务或者服务器之后可以从备份文件（RDB、AOF）中恢复数据到内存继续使用，memcached把数据全部存在内存之中，断电后会挂掉
  2. Redis 支持更多的数据类型：支持string(字符串)、hash(哈希数据)、list(列表)、set(集合)、zset(有序集合)
  3. 支持数据的备份：可以实现类似于数据的master-slave模式的数据备份，另外也支持使用快照+AOF
  4. 支持更大的value数据：memcache单个key value最大只支持1MB，而redis最大支持512MB(生产不建议超过2M,性能受影响)
  5. 在Redis6版本前,Redis 是单线程，而memcached是多线程，所以单机情况下没有memcached 并发高,性能更好,但redis 支持分布式集群以实现更高的并发，单Redis实例可以实现数万并发
  6. 支持集群横向扩展：基于redis cluster的横向扩展，可以实现分布式集群，大幅提升性能和数据安全性
©著作权归作者所有,转载或内容合作请联系作者
平台声明：文章内容（如有图片或视频亦包括在内）由作者上传并发布，文章内容仅代表作者本人观点，简书系信息发布平台，仅提供信息存储服务。

推荐阅读更多精彩内容