安装环境准备
- CentOS
- Docker
创建redis docker基础镜像
- 下载安装包
[root@izwz962mggaelpqejjtvimz docker_redis_cluster]# wget http://download.redis.io/releases/redis-5.0.4.tar.gz
- 解压
[root@izwz962mggaelpqejjtvimz docker_redis_cluster]# tar zxvf redis-5.0.4.tar.gz
- 修改redis.conf配置文件
- NETWORK配置,修改为0.0.0.0，是为了接收外部连接
################################## NETWORK #####################################
# By default, if no "bind" configuration directive is specified, Redis listens
# for connections from all the network interfaces available on the server.
# It is possible to listen to just one or multiple selected interfaces using
# the "bind" configuration directive, followed by one or more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1 ::1
#
# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 loopback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#bind 127.0.0.1
bind 0.0.0.0
- 默认客户端登录不需要密码，开启后需要使用密码，如果是部署在自己的机器上可以使用默认不开启
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
# commands. This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g. they run their own servers).
#
# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.
#
# requirepass foobared
requirepass 123456
如果主节点开启了客户端密码验证机制，则需要配置，不配置集群情况下，Slave连接不上master节点
# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the replica to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the replica request.
#
# masterauth <master-password>
masterauth 123456
- 关闭安全模式
# By default protected mode is enabled. You should disable it only if
# you are sure you want clients from other hosts to connect to Redis
# even if no authentication is configured, nor a specific set of interfaces
# are explicitly listed using the "bind" directive.
#protected-mode yes
protected-mode no
- 设置日志路径
# Specify the log file name. Also the empty string can be used to force
# Redis to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
#logfile ""
logfile "/var/log/redis/redis-server.log"
- 设置AOF模式开启（持久化）
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
appendonly yes
- 集群相关配置
################################ REDIS CLUSTER ###############################
#
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however
# in order to mark it as "mature" we need to wait for a non trivial percentage
# of users to deploy it in production.
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:
#
# cluster-enabled yes
cluster-enabled yes
# Every cluster node has a cluster configuration file. This file is not
# intended to be edited by hand. It is created and updated by Redis nodes.
# Every Redis Cluster node requires a different cluster configuration file.
# Make sure that instances running in the same system do not have
# overlapping cluster configuration file names.
#
# cluster-config-file nodes-6379.conf
cluster-config-file nodes-6379.conf
# Cluster node timeout is the amount of milliseconds a node must be unreachable
# for it to be considered in failure state.
# Most other internal time limits are multiple of the node timeout.
#
# cluster-node-timeout 15000
cluster-node-timeout 15000
- Dockerfile
FROM centos
MAINTAINER huangyifei justin_yf@sina.com
ENV REDIS_HOME /usr/local
ADD redis-5.0.4.tar.gz /
RUN mkdir -p $REDIS_HOME/redis
ADD redis-5.0.4/redis.conf $REDIS_HOME/redis/
RUN yum -y update
RUN yum -y install vim
RUN yum install -y gcc make
WORKDIR /redis-5.0.4
RUN make
RUN mv /redis-5.0.4/src/redis-server $REDIS_HOME/redis/
WORKDIR /
RUN rm -rf /redis-5.0.4
RUN yum remove -y gcc make
VOLUME ["/var/log/redis"，"/appendonly.aof"]
EXPOSE 6379
- 制作执行镜像
[root@izwz962mggaelpqejjtvimz docker_redis_cluster]# docker build -t cluster-redis .
[root@izwz962mggaelpqejjtvimz docker_redis_cluster]# docker images cluster-redis
REPOSITORY TAG IMAGE ID CREATED SIZE
cluster-redis latest 46e49549692a 51 seconds ago 770 MB
- 设置镜像标签
[root@izwz962mggaelpqejjtvimz docker_redis_cluster]# docker tag cluster-redis cluster-redis:5.0.4
[root@izwz962mggaelpqejjtvimz docker_redis_cluster]# docker images cluster-redis
REPOSITORY TAG IMAGE ID CREATED SIZE
cluster-redis 5.0.4 46e49549692a 11 minutes ago 770 MB
cluster-redis latest 46e49549692a 11 minutes ago 770 MB
制作Reids节点镜像
- 编写Dockerfile
FROM cluster-redis:5.0.4
MAINTAINER huangyifei justin_yf@sina.com
ENTRYPOINT ["/usr/local/redis/redis-server","/usr/local/redis/redis.conf"]
- 构建redis节点镜像
[root@izwz962mggaelpqejjtvimz docker_redsi_node]# docker build -t nodes-redis:5.0.4 .
- 查看镜像
[root@izwz962mggaelpqejjtvimz ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a0eaad0e335a nodes-redis:5.0.4 "/usr/local/redis/..." About an hour ago Up About an hour 6379/tcp redis-node6
d289938a3c8f nodes-redis:5.0.4 "/usr/local/redis/..." About an hour ago Up About an hour 6379/tcp redis-node5
5e6f16a9bbbb nodes-redis:5.0.4 "/usr/local/redis/..." About an hour ago Up About an hour 6379/tcp redis-node4
0cf5311e5d99 nodes-redis:5.0.4 "/usr/local/redis/..." 2 hours ago Up 2 hours 6379/tcp redis-node3
beef6e750e60 nodes-redis:5.0.4 "/usr/local/redis/..." 2 hours ago Up 2 hours 6379/tcp redis-node2
9b21d9a2464b nodes-redis:5.0.4 "/usr/local/redis/..." 2 hours ago Up 2 hours 0.0.0.0:6379->6379/tcp redis-node1
运行redis集群
- 运行redis容器
[root@izwz962mggaelpqejjtvimz docker_redsi_node]# docker run -d --name redis-node1 -p 6379:6379 nodes-redis:5.0.4
cfbfb4d3d5126c6f986021cd3f55837cf9037447440e2c044ba99d7375479d7a
[root@izwz962mggaelpqejjtvimz docker_redsi_node]# docker run -d --name redis-node2 -p nodes-redis:5.0.4
0c4d097017fb54110f82f3fcffd3f0035067086f3a4f1906f39b57cfb457c50e
[root@izwz962mggaelpqejjtvimz docker_redsi_node]# docker run -d --name redis-node3 -p nodes-redis:5.0.4
c055331f76baae52e6753cbdbc8b2024b830a462185b7276ee785ee48f92503e
- 查看容器
[root@izwz962mggaelpqejjtvimz docker_redsi_node]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c055331f76ba nodes-redis:5.0.4 "/usr/local/redis/..." 35 seconds ago Up 34 seconds 0.0.0.0:6381->6379/tcp redis-node3
0c4d097017fb nodes-redis:5.0.4 "/usr/local/redis/..." 52 seconds ago Up 51 seconds 0.0.0.0:6380->6379/tcp redis-node2
cfbfb4d3d512 nodes-redis:5.0.4 "/usr/local/redis/..." About a minute ago Up About a minute 0.0.0.0:6379->6379/tcp redis-node1
e9e382ce5dec sso-service "/bin/sh -c '/usr/..." 3 weeks ago Up 2 weeks 0.0.0.0:8082->8080/tcp sso-service
f66c22c8c0ff docker.io/mariadb:latest "docker-entrypoint..." 3 weeks ago Up 2 weeks 0.0.0.0:3306->3306/tcp mariadb
- 启动集群
[root@izwz962mggaelpqejjtvimz src]# ./redis-cli --cluster create 172.17.0.3:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 172.17.0.8:6379 172.17.0.9:6379 --cluster-replicas 1 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.8:6379 to 172.17.0.3:6379
Adding replica 172.17.0.9:6379 to 172.17.0.5:6379
Adding replica 172.17.0.7:6379 to 172.17.0.6:6379
M: 7567e7e809175552c8c1d91709071767605c337c 172.17.0.3:6379
slots:[0-5460] (5461 slots) master
M: 8a76d2a39d6abdd98097869f6b2c9041205dfe3a 172.17.0.5:6379
slots:[5461-10922] (5462 slots) master
M: 7219456b036427623d210e83131cde537b54fcbd 172.17.0.6:6379
slots:[10923-16383] (5461 slots) master
S: 766ac71ba36e430607622fdf459c57100f81ca46 172.17.0.7:6379
replicates 7219456b036427623d210e83131cde537b54fcbd
S: f05fc4685bc928090c72983ec2cdfe04fc03d8b2 172.17.0.8:6379
replicates 7567e7e809175552c8c1d91709071767605c337c
S: f32b2042406a40d146abbcf9686e8ae82dc56ebf 172.17.0.9:6379
replicates 8a76d2a39d6abdd98097869f6b2c9041205dfe3a
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 172.17.0.3:6379)
M: 7567e7e809175552c8c1d91709071767605c337c 172.17.0.3:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: f05fc4685bc928090c72983ec2cdfe04fc03d8b2 172.17.0.8:6379
slots: (0 slots) slave
replicates 7567e7e809175552c8c1d91709071767605c337c
M: 8a76d2a39d6abdd98097869f6b2c9041205dfe3a 172.17.0.5:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: f32b2042406a40d146abbcf9686e8ae82dc56ebf 172.17.0.9:6379
slots: (0 slots) slave
replicates 8a76d2a39d6abdd98097869f6b2c9041205dfe3a
M: 7219456b036427623d210e83131cde537b54fcbd 172.17.0.6:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 766ac71ba36e430607622fdf459c57100f81ca46 172.17.0.7:6379
slots: (0 slots) slave
replicates 7219456b036427623d210e83131cde537b54fcbd
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 检查集群状态
[root@izwz962mggaelpqejjtvimz src]# ./redis-cli --cluster info localhost:6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
localhost:6379 (7567e7e8...) -> 0 keys | 5461 slots | 1 slaves.
172.17.0.5:6379 (8a76d2a3...) -> 0 keys | 5462 slots | 1 slaves.
172.17.0.6:6379 (7219456b...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
- 客户端检查某一Master集群状态
[root@izwz962mggaelpqejjtvimz redis-5.0.4]# ./src/redis-cli -h localhost -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
localhost:6379> info replication
# Replication
role:master
connected_slaves:0
master_replid:e67fbbdc16211eeda02bacdf242b76145fae8cbb
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
-
集群架构
image.png
a、一个集群里面有M1、M2、M3三个节点，其中节点 M1包含 0 到 5500号哈希槽，节点M2包含5501 到 11000 号哈希槽，节点M3包含11001 到 16384号哈希槽。如果M2宕掉了，就会导致5501 到 11000 号哈希槽不可用，从而使整个集群不可用。
b、一个集群里面有M1-S1、M2-S2、M3-S3六个主从节点，其中节点 M1包含 0 到 5500号哈希槽，节点M2包含5501 到 11000 号哈希槽，节点M3包含11001 到 16384号哈希槽。如果是M2宕掉，集群便会选举S2为新节点继续服务，整个集群还会正常运行。当M2、S2都宕掉了，这时候集群就不可用了。
- 测试集群
[root@izwz962mggaelpqejjtvimz src]# ./redis-cli -c -h localhost -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
localhost:6379> set a 100
-> Redirected to slot [15495] located at 172.17.0.6:6379
OK
172.17.0.6:6379> get a
"100"
172.17.0.6:6379>
