HA Cluster 高可用集群keepalived

一、简述HA Cluster原理

高可用集群，英文原文为High Availability Cluster，简称HA Cluster；集群（cluster）就是一组计算机，它们作为一个整体向用户提供一组网络资源。这些单个的计算机系统 就是集群的节点（node）。高可用性集群（HA cluster）是指如单系统一样地运行并支持（计算机）持续正常运行的一个主机群。

高可用集群的出现是为了使集群的整体服务尽可能可用，从而减少由计算机硬件和软件易错性所带来的损 失。如果某个节点失效，它的备援节点将在几 秒钟的时间内接管它的职责。因此，对于用户而言，集群永远不会停机。高可用集群软件的主要作用就是实现故障检查和业务切换的自动化。

简单说高可用集群就是为了解决集群中的单点故障（SPoF）,保证服务不间断运行的冗余（redundant）手段。

  • SPoF：Single Point of Failure；单点故障
  • 冗余（redundant）： 在两个节点上装一个软件程序，根据判断状态完成资源转移；
高可用集群的衡量标准

通常用平均无故障时间(MTTF)来度量系统的可靠性,用平均故障维修时间（MTTR）来度量系统的可维护性。于是可用性被定义为：HA=MTTF/(MTTF+MTTR)*100%

  • 可用性衡量指标：
    • 基本可用性：2个9；99%；年度停机时间 87.6小时
    • 较高可用性：3个9；99.9%；年度停机时间 8.8小时
    • 具有故障自动恢复能力的可用性：4个9；99.99%；年度停机时间53分钟
    • 极高可用性：5个9；99.999%；年度停机时间5分钟

二、keepalived

  1. KeepAlived主要有两个功能：
  • (1).能够对RealServer进行健康状况检查，支持4层、5层和7层协议进行健康检查；
  • (2).对负载均衡调度器实现高可用，防止Director单点故障。
  1. KeepAlived工作过程：
    keepalived实现故障转移的功能是通过VRRP(virtual router redundancy protocol虚拟路由器冗余协议)协议来实现的。 在keepalived正常工作的时候，主节点(master)会不断的发送心跳信息给备节点(backup)，当备节点不能在一定时间内收到主节点的心跳信息时，备节点会认为主节点宕了，然后会接管主节点上的资源，并继续向外提供服务保证其可用性。当主节点恢复的时候，备节点会自动让出资源并再次自动成为备节点。
  1. keepalived基于vrrp协议的软件实现，原生设计的目的为了高可用ipvs服务；
  • 基于vrrp协议完成地址流动；
  • 为vip地址所在的节点生成ipvs规则（在配置文件中预先定义）；
  • 为ipvs集群的各RS做健康状态检测；
  • 基于脚本调用接口通过执行脚本完成脚本中定义的功能，进而影响集群事务；
  1. HA Cluster的配置前提：
    (1) 各节点时间必须同步； ntp, chrony
    (2) 确保iptables及selinux不会成为阻碍；
    (3) 各节点之间可通过主机名互相通信（对KA并非必须）；
    建议使用/etc/hosts文件实现；
    (4) 确保各节点的用于集群服务的接口支持MULTICAST通信；
    D类：224-239；
  2. keepalived安装配置：
    在CentOS6.4以后，keepalivd随base仓库提供；

  • 程序环境：
    主配置文件：/etc/keepalived/keepalived.conf
    主程序文件：/usr/sbin/keepalived
    nit File：keepalived.service
    Unit File的环境配置文件：/etc/sysconfig/keepalived

  • 配置文件组件部分：
    TOP HIERACHY
    - GLOBAL CONFIGURATION
    - Global definitions
    - Static routes/addresses
    - VRRPD CONFIGURATION
    - VRRP synchronization group(s)：vrrp同步组；
    - VRRP instance(s)：每个vrrp instance即一个vrrp路由器；
    - LVS CONFIGURATION
    - Virtual server group(s)
    - Virtual server(s)：ipvs集群的vs和rs；

  • 配置语法：

    • 配置虚拟路由器：
      vrrp_instance <STRING> { .... }

    • 专用参数：
      state MASTER|BACKUP：当前节点在此虚拟路由器上的初始状态；只能有一个是MASTER，余下的都应该为BACKUP；
      interface IFACE_NAME：绑定为当前虚拟路由器使用的物理接口；
      virtual_router_id VRID：当前虚拟路由器的惟一标识，范围是0-255；
      priority 100：当前主机在此虚拟路径器中的优先级；范围1-254；
      advert_int 1：vrrp通告的时间间隔；

                    authentication {
                        auth_type AH|PASS  
                      auth_pass <PASSWORD>
              }```
              `virtual_ipaddress {
                  <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
                  192.168.200.17/24 dev eth1
                  192.168.200.18/24 dev eth2 label eth2:1
              }
               track_interface {
                  eth0
                  eth1
                  ...
              }`  # 配置要监控的网络接口，一旦接口出现故障，则转为FAULT状态；

              `nopreempt：`定义工作模式为非抢占模式；
              `preempt_delay 300：`抢占式模式下，节点上线后触发新选举操作的延迟时长；

  • 定义通知脚本：

                          notify_master <STRING>|<QUOTED-STRING>：当前节点成为主节点时触发的脚本；
                      notify_backup <STRING>|<QUOTED-STRING>：当前节点转为备节点时触发的脚本；
                      notify_fault <STRING>|<QUOTED-STRING>：当前节点转为“失败”状态时触发的脚本；                           
                      notify <STRING>|<QUOTED-STRING>：通用格式的通知触发机制，一个脚本可完成以上三种状态的转换时的通知；

  • 虚拟服务器：
    配置参数：

        virtual_server IP port |
    virtual_server fwmark int 
    {
        ...
        real_server {
            ...
        }
        ...
    }

常用参数：
delay_loop <INT>：服务轮询的时间间隔；
lb_algo rr|wrr|lc|wlc|lblc|sh|dh：定义调度方法；
lb_kind NAT|DR|TUN：集群的类型；
persistence_timeout <INT>：持久连接时长；
protocol TCP：服务协议，仅支持TCP；
sorry_server <IPADDR> <PORT>：备用服务器地址；

real_server <IPADDR> <PORT>
                {
                     weight <INT>
                     notify_up <STRING>|<QUOTED-STRING>
                     notify_down <STRING>|<QUOTED-STRING>
                     HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK { ... }：定义当前主机的健康状态检测方法；
                }

  • HTTP_GET|SSL_GET：应用层检测

              HTTP_GET|SSL_GET {
              url {
                  path <URL_PATH>：定义要监控的URL；
                  status_code <INT>：判断上述检测机制为健康状态的响应码；
                  digest <STRING>：判断上述检测机制为健康状态的响应的内容的校验码；
              }
              nb_get_retry <INT>：重试次数；
              delay_before_retry <INT>：重试之前的延迟时长；
              connect_ip <IP ADDRESS>：向当前RS的哪个IP地址发起健康状态检测请求
              connect_port <PORT>：向当前RS的哪个PORT发起健康状态检测请求
              bindto <IP ADDRESS>：发出健康状态检测请求时使用的源地址；
              bind_port <PORT>：发出健康状态检测请求时使用的源端口；
              connect_timeout <INTEGER>：连接请求的超时时长；
          }

  • TCP_CHECK：传输层检测

               TCP_CHECK {
              connect_ip <IP ADDRESS>：向当前RS的哪个IP地址发起健康状态检测请求
              connect_port <PORT>：向当前RS的哪个PORT发起健康状态检测请求
              bindto <IP ADDRESS>：发出健康状态检测请求时使用的源地址；
              bind_port <PORT>：发出健康状态检测请求时使用的源端口；
              connect_timeout <INTEGER>：连接请求的超时时长；
          }

三、keepalived实现主从、主主架构

  1. 主从配置：
    准备2个节点：node1:192.168.1.108；node2:192.168.1.109
    同步时间：[root@node1 ~]# ntpdate 192.168.1.10
    安装配置keepalived：
    在node1如下配置

     [root@node1 ~]# yum -y install keepalived         #安装keepalived
 [root@node1 ~]# cd /etc/keepalived/
 [root@node1 keepalived]# cp keepalived.conf{,.bak}    #备份keepalived原始配置文件
 [root@node1 keepalived]# vim keepalived.conf
 #在打开的文件中配置如下内容
     ! Configuration File for keepalived

         global_defs {
                notification_email {
                 root@localhost
            }
             
                notification_email_from keepalived@localhost
                smtp_server 127.0.0.1
                smtp_connect_timeout 30
                router_id node1
                 vrrp_mcast_group4 224.1.105.33
             }
         
         vrrp_instance VI_1 {
                 state MASTER   #当前节点在此虚拟路由器上的初始状态；只能有一个是MASTER，余下的都应该为BACKUP；
                 interface ens33
                 virtual_router_id 33
                 priority 100
                 advert_int 1
                 authentication {
                     auth_type PASS
                     auth_pass 1111
                 }
                 virtual_ipaddress {
                     192.168.1.99 dev ens33 label ens33:0
                 }
             }

在node2节点上如下配置：

    [root@node2 ~]# yum -y install keepalived         #安装keepalived
    [root@node2 ~]# cd /etc/keepalived/
    [root@node2 keepalived]# cp keepalived.conf{,.bak}    #备份keepalived原始配置文件
    [root@node2 keepalived]# vim keepalived.conf
    #在打开的文件中配置如下内容
        ! Configuration File for keepalived

            global_defs {
                   notification_email {
                    root@localhost
               }
                
                   notification_email_from keepalived@localhost
                   smtp_server 127.0.0.1
                   smtp_connect_timeout 30
                   router_id node2
                    vrrp_mcast_group4 224.1.105.33
                }
            
            vrrp_instance VI_1 {
                    state BACKUP
                    interface ens33
                    virtual_router_id 33
                    priority 96
                    advert_int 1
                    authentication {
                        auth_type PASS
                        auth_pass 1111
                    }
                    virtual_ipaddress {
                        192.168.1.99 dev ens33 label ens33:0
                    }
                }

启动node2节点keepalived测试

[root@node2 ~]# systemctl start keepalived
[root@node2 ~]# ifconfig
...
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.99  netmask 255.255.255.255  broadcast 0.0.0.0
    ether 00:0c:29:f0:be:9c  txqueuelen 1000  (Ethernet)
[root@node2 ~]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2018-08-26 09:37:52 CST; 7s ago
  Process: 15928 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 15929 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─15929 /usr/sbin/keepalived -D
           ├─15930 /usr/sbin/keepalived -D
           └─15931 /usr/sbin/keepalived -D

Aug 26 09:37:52 node2 Keepalived_vrrp[15931]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug 26 09:37:56 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ...1.99
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: Sending gratuitous ARP on ens33 for 192.168.1.99

#在node1节点上抓包测试
[root@node1 ~]# tcpdump -i ens33 -nn host 224.1.105.33
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
09:37:56.221751 IP 192.168.1.109 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 96, authtype simple, intvl 1s, length 20
09:37:57.227332 IP 192.168.1.109 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 96, authtype simple, intvl 1s, length 20

启动node1节点keepalived：

[root@node1 ~]# systemctl start keepalived
[root@node1 ~]# ifconfig
...
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.99  netmask 255.255.255.255  broadcast 0.0.0.0
    ether 00:0c:29:21:8d:06  txqueuelen 1000  (Ethernet)
[root@node1 ~]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2018-08-26 09:42:23 CST; 1min 9s ago
  Process: 15076 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 15077 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─15077 /usr/sbin/keepalived -D
           ├─15078 /usr/sbin/keepalived -D
           └─15079 /usr/sbin/keepalived -D

Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: VRRP_Instance(VI_1) Sending/queueing gratuitous ....99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Hint: Some lines were ellipsized, use -l to show in full.

#node1节点抓包测试
[root@node1 ~]# tcpdump -i ens33 -nn host 224.1.105.33
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
09:43:18.304748 IP 192.168.1.108 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 100, authtype simple, intvl 1s, length 20
09:43:19.305917 IP 192.168.1.108 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 100, authtype simple, intvl 1s, length 20

  1. 双主模式配置

     #node1节点上修改keepalived.conf配置文件，在最后添加如下内容
 vrrp_instance VI_ 2{
     state BACKUP
     interface ens33
     virtual_router_id 34
     priority 96
     advert_int 1
     authentication {
         auth_type PASS
         auth_pass XXXX1111
     }
     virtual_ipaddress {
         192.168.1.98 dev ens33 label ens33:0
     }
 }

 #node2节点上修改keepalived.conf配置文件，在最后添加如下内容
 vrrp_instance VI_2 {
     state MASTER
     interface ens33
     virtual_router_id 34
     priority 100
     advert_int 1
     authentication {
         auth_type PASS
         auth_pass XXXX1111
     }
     virtual_ipaddress {
         192.168.1.98 dev ens33 label ens33:0
     }
 }
  #停止keepalived服务，再重新启动
  [root@node2 ~]# systemctl stop keepalived  
 [root@node2 ~]# systemctl start keepalived
 [root@node2 ~]# ip a l
 ...
 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:f0:be:9c brd ff:ff:ff:ff:ff:ff
     inet 192.168.1.109/24 brd 192.168.1.255 scope global ens33
        valid_lft forever preferred_lft forever
     inet 192.168.1.98/32 scope global ens33:0
        valid_lft forever preferred_lft forever
     inet6 fe80::7221:3e5e:6c0:4c1c/64 scope link 
        valid_lft forever preferred_lft forever
 ...
 [root@node2 ~]# systemctl status keepalived
 ● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
    Active: active (running) since Sun 2018-08-26 10:17:01 CST; 1min 8s ago
   Process: 16217 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
  Main PID: 16218 (keepalived)
    CGroup: /system.slice/keepalived.service
            ├─16218 /usr/sbin/keepalived -D
            ├─16219 /usr/sbin/keepalived -D
            └─16220 /usr/sbin/keepalived -D

 Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
 Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
 Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
 Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ...1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Hint: Some lines were ellipsized, use -l to show in full

 #重新启动node1节点的keepalived服务
  [root@node1 keepalived]# systemctl start keepalived
 [root@node1 keepalived]# ip a l
 ...
 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:21:8d:06 brd ff:ff:ff:ff:ff:ff
     inet 192.168.1.108/24 brd 192.168.1.255 scope global dynamic ens33
        valid_lft 2403sec preferred_lft 2403sec
     inet 192.168.1.99/32 scope global ens33:0
        valid_lft forever preferred_lft forever
     inet6 fe80::959:d8ab:dd39:b1b/64 scope link 
        valid_lft forever preferred_lft forever
 ...
 [root@node1 keepalived]# systemctl status keepalived
 ● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
    Active: active (running) since Sun 2018-08-26 10:19:19 CST; 1min 24s ago
   Process: 15146 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
  Main PID: 15147 (keepalived)
    CGroup: /system.slice/keepalived.service
            ├─15147 /usr/sbin/keepalived -D
            ├─15148 /usr/sbin/keepalived -D
            └─15149 /usr/sbin/keepalived -D

 Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: VRRP_Instance(VI_1) Sending/queueing gratuitous ....99
 Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Hint: Some lines were ellipsized, use -l to show in full.

 #   在node2节点上status查看状态
 [root@node2 ~]# systemctl status keepalived
 ● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
    Active: active (running) since Sun 2018-08-26 10:17:01 CST; 4min 59s ago
   Process: 16217 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
  Main PID: 16218 (keepalived)
    CGroup: /system.slice/keepalived.service
            ├─16218 /usr/sbin/keepalived -D
            ├─16219 /usr/sbin/keepalived -D
            └─16220 /usr/sbin/keepalived -D

 Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ...1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
 Aug 26 10:19:21 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Received advert with higher priority...s 96
 Aug 26 10:19:21 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Entering BACKUP STATE
 Aug 26 10:19:21 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) removing protocol VIPs.

  2. 通知脚本使用方式

     #编辑通知脚本
 [root@node1 ~]# cd /etc/keepalived/
 [root@node1 keepalived]# vim notify.sh
         #!/bin/bash
     #keepalived 邮件通知脚本
 #date： 2018-8-26
 contact = 'root@localhost'
 notify () {
     local mailsubject="$(hostname) to be $1 vip floating"
     local mailbody="$(data + '%F %T'): vrrp transition, $(hostname) changed to be $1"
     echo "$mailbody" | mail -s "$mailsubject" $contact
 }
 case $1 in
 master)
     notify master
     ;;
 backup)
     notify backup
     ;;
 fault)
     notify fault
     ;;
 *)
     echo "Usage: $(basename $0) {master|backup|fault}"
     exit 1
     ;;
 esac
 #在keepalived.conf中的vrrp实例中添加如下内容
 vrrp_instance VI_1 {
                 state BACKUP
                 interface ens33
                 virtual_router_id 33
                 priority 96
                 advert_int 1
                 authentication {
                     auth_type PASS
                     auth_pass 1111
                 }
                 virtual_ipaddress {
                     192.168.1.99 dev ens33 label ens33:0
                 }
                 notify_master "/etc/keepalived/notify.sh master"
                 notify_backuo "/etc/keepalived/notify.sh backup"
                 notify_fault "/etc/keepalived/notify.sh fault"
             }

  3. 以dr集群架构配置示例


    dr架构.png
[root@node1 ~]# yum -y install ipvsadm #安装ipvsadm以便查看生成的规则
#编辑keepalived.conf为node1和node2生成规则
[root@node1 keepalived]# vim keepalived.conf
    ! Configuration File for keepalived

        global_defs {
           notification_email {
            root@localhost
           }
            
           notification_email_from keepalived@localhost
           smtp_server 127.0.0.1
           smtp_connect_timeout 30
           router_id node1
            vrrp_mcast_group4 224.1.105.33
        }
        
        vrrp_instance VI_1 {
            state MASTER
            interface ens33
            virtual_router_id 33
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass XXXX1111
            }
            virtual_ipaddress {
                192.168.1.99 dev ens33 label ens33:0
            }
             notify_master "/etc/keepalived/notify.sh master"
             notify_backup "/etc/keepalived/notify.sh backup"
             notify_fault "/etc/keepalived/notify.sh fault"
        }
        virtual_server 192.168.1.99 80 {
            delay_loop 1
            lb_algo wrr
            lb_kind DR
            protocol TCP
            sorry_server 127.0.0.1 80
        
            real_server 192.168.1.111 80 {
                weight 1
                HTTP_GET {
                    url {
                        path /index.html
                        status_code 200
                        }
                    nb_get_retry 3
                    delay_before_retry 2
                    connect_timeout 3
                    }
            }
            real_server 192.168.1.122 80 {
                weight 1
                HTTP_GET {
                    url {
                        path /index.html
                        status_code 200
                        }
                    nb_get_retry 3
                    delay_before_retry 2
                    connect_timeout 3
                    }
            }
                    
        }
#将此配置文件拷贝到node2节点，并修改以下几行
    router_id node2
    state BACKUP
    priority 96
#重新启动node2节点的keepalived服务
[root@node2 ~]# systemctl stop keepalived
[root@node2 ~]# systemctl start keepalived
[root@node2 ~]# ifconfig
...
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.99  netmask 255.255.255.255  broadcast 0.0.0.0
        ether 00:0c:29:f0:be:9c  txqueuelen 1000  (Ethernet)

...

[root@node2 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.99:80 wrr
  -> 192.168.1.111:80             Route   1      0          0         
  -> 192.168.1.122:80             Route   1      0          0     
  
#使用client访问测试正常
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 1</h1>
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 2</h1>

#启动node1的keepalived服务，通过下面查看ip和status后看到node1已经成功上线
[root@node1 keepalived]# systemctl start keepalived
[root@node1 keepalived]# ifconfig
[root@node1 keepalived]# systemctl status keepalived
#使用client访问服务正常
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 1</h1>
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 2</h1>

四、高可用nginx配置示例

高可用nginx架构.png

@以上图的架构配置实验网络环境；node1和node2双网卡，dip桥接，私网地址VMent1，rs1和rs2为后端主机，配置两个访问index.html为RealServer 1和RealServer 2；

  • 在node1节点上安装配置keepalived服务

    [root@node1 ~]# yum -y install keepalived
[root@node1 ~]# vim /etc/keepalived/keepalived.conf
#按如下内容编辑配置文件
      ! Configuration File for keepalived

      global_defs {
             notification_email {
              root@localhost
         }                    
             notification_email_from keepalived@localhost
             smtp_server 127.0.0.1
             smtp_connect_timeout 30
             router_id node1
              vrrp_mcast_group4 224.1.105.33
          }
      
      vrrp_script chk_down {
              script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
              weight -10
              interval 1
              fall 1
              rise 1
          }   
      
      
      vrrp_instance VI_1 {
              state MASTER
              interface ens33
              virtual_router_id 33
              priority 100
              advert_int 1                    
              authentication {
                  auth_type PASS
                  auth_pass XXXX1111
              }
              virtual_ipaddress {
                  192.168.1.99/24 dev ens33 label ens33:0
              }
              track_script {
                  chk_down
              }
           notify_master "/etc/keepalived/notify.sh master"
           notify_backup "/etc/keepalived/notify.sh backup"
           notify_fault "/etc/keepalived/notify.sh fault"
      }

  • 安装配置nginx服务

      [root@node1 ~]# yum -y install nginx
  [root@node1 ~]# vim /etc/nginx/nginx.conf
      #在配置文件中http{...}段中添加如下内容
      upstream websrvs {
              server 192.168.10.111:80;
              server 192.168.10.122:80;
      }
      #在location段中配置反向代理读物
      location / {
              proxy_pass http://websrvs;
      }

  • 修改通知脚本

      [root@node1 ~]# vim /etc/keepalived/notify.sh 
      #!/bin/bash
      #keepalived 邮件通知脚本
      #2018-8-26
      contact = 'root@localhost'
      notify () {
          local mailsubject="$(hostname) to be $1 vip floating"
          local mailbody="$(data + '%F %T'): vrrp transition, $(hostname) changed to be $1"
          echo "$mailbody" | mail -s "$mailsubject" $contact
      }
      case $1 in
      master)
          systemctl start nginx
          notify master
          ;;
      backup)
          systemctl start nginx
          notify backup
          ;;
      fault)
          systemctl start nginx
          notify fault
          ;;
      *)
          echo "Usage: $(basename $0) {master|backup|fault}"
          exit 1
          ;;
      esac

  • 在keepalived配置中定义nginx脚本并调用

  • 配置双主模式的keepalived服务，在keepalived.conf配置如下内容

      vrrp_instance VI_2 {
              state BACKUP
              interface ens33
              virtual_router_id 43
              priority 96
              advert_int 1
           authentication {
                  auth_type PASS
                  auth_pass XXXX1111
              }   
              virtual_ipaddress {
                  192.168.1.98/24 dev ens33 label ens33:0
              }   
                  track_script {
                          chk_down
                          chk_ngx
                  }       
                   notify_master "/etc/keepalived/notify.sh master"
                   notify_backup "/etc/keepalived/notify.sh backup"
                   notify_fault "/etc/keepalived/notify.sh fault"
          }

  • 配置检测网卡状态，在keepalived.conf配置文件最后添加如下内容

      track_interface {
      ens33
      ens37
  }

  • 至此node1节点配置完成，下面配置node2节点

      [root@node1 ~]# scp /etc/keepalived/keepalived.conf /etc/nginx/nginx.conf root@192.168.1.109:      #复制keepalived和nginx服务的配置文件到node2
  [root@node2 ~]# mv nginx.conf /etc/nginx/     
  [root@node2 ~]# nginx -t
  [root@node2 ~]# systemctl start nginx
  [root@node2 ~]# mv keepalived.conf /etc/keepalived/ 
  [root@node2 ~]# vim /etc/keepalived/keepalived.conf
      #修改如下内容
      router_id node2
      state BACKUP
      priority 96
      vrrp_instance VI_2 {
          state MASTER
          virtual_router_id 43
          priority 100

  • 至此配置完成，启动keepalived服务后可以使用client访问测试

      [root@localhost ~]# curl http://www.ilinux.io
  <h1>RealServer 1</h1>
  [root@localhost ~]# curl http://www.ilinux.io
  <h1>RealServer 2</h1>
  [root@localhost ~]# curl http://www.ilinux.io
  <h1>RealServer 1</h1>
最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 212,029评论 6赞 492
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 90,395评论 3赞 385
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 157,570评论 0赞 348
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 56,535评论 1赞 284
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 65,650评论 6赞 386
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 49,850评论 1赞 290
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 39,006评论 3赞 408
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 37,747评论 0赞 268
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 44,207评论 1赞 303
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 36,536评论 2赞 327
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 38,683评论 1赞 341
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 34,342评论 4赞 330
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 39,964评论 3赞 315
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 30,772评论 0赞 21
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 32,004评论 1赞 266
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 46,401评论 2赞 360
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 43,566评论 2赞 349

推荐阅读更多精彩内容