- 配置hosts文件
[root@mysql132 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.132 mysql132 mysql132.com
192.168.40.133 mysql133 mysql133.com
192.168.40.134 mysql134 mysql134.com
192.168.40.139 mysql139 mysql139.com
- 在所有MySQL节点的/etc/my.cnf中增加参数（要重启）
[root@mysql132 ~]# vim /etc/my.cnf
---------------------------------------------------------------
[mysqld]
read_only=1
#read_only是因为MMM对数据需严格的读写控制
#此参数不影响replication;root用户依然可写。
---------------------------------------------------------------
- 安装依赖软件
#在所有机器上执行下面的安装脚本
[root@db-master1 ~]# cd /usr/local/src
[root@db-master1 ~]# mkdir -p /usr/local/src/mysql-mmm/
[root@db-master1 ~]# cd mysql-mmm
[root@db-master1 ~]# wget http://mysql-mmm.org/_media/:mmm2:mysql-mmm-2.2.1.tar.gz
[root@db-master1 ~]# mv :mmm2:mysql-mmm-2.2.1.tar.gz mysql-mmm-2.2.1.tar.gz
[root@db-master1 ~]# yum install -y perl perl-CPAN *YAML* perl-CPAN* mysql-client mysql-devel wget
[root@db-master1 ~]# touch /usr/local/src/mysql-mmm/install.sh
[root@db-master1 ~]# chmod 755 /usr/local/src/mysql-mmm/install.sh
[root@db-master1 ~]# vim /usr/local/src/mysql-mmm/install.sh
[root@db-master1 ~]# vim /usr/local/src/mysql-mmm/install.sh
---------------------------------------------------------------
#!/bin/bash
wget http://xrl.us/cpanm --no-check-certificate
mv cpanm /usr/bin
chmod 755 /usr/bin/cpanm
cat > /root/list << EOF
install Test::More
install Algorithm::Diff
install Class::Singleton
install DBI
install DBD::mysql
install File::Basename
install File::stat
install File::Temp
install Log::Dispatch
install Log::Log4perl
install Mail::Send
install Net::ARP
install Net::Ping
install Proc::Daemon
install Thread::Queue
install Time::HiRes
EOF
for package in `cat /root/list`
do
cpanm $package
done
--------------------------------------------------------------------
[root@db-master1 ~]# /usr/local/src/mysql-mmm/install.sh
可以再执行一次/usr/local/src/mysql-mmm/install.sh，确认perl的依赖软件全都安装成功：
[root@db-master1 ~]# /usr/local/src/mysql-mmm/install.sh
[root@db-master1 ~]# tar -zvxf mysql-mmm-2.2.1.tar.gz
[root@db-master1 ~]# mv mysql-mmm-2.2.1 mysql-mmm
[root@db-master1 ~]# mv mysql-mmm /usr/local/
安装MySQL-MMM：
[root@db-master1 ~]# cd /usr/local/mysql-mmm
[root@db-master1 ~]# make install
-----------------------------------------------------------------------------------------------------------
mkdir -p /usr/share/perl5/vendor_perl/MMM /usr/lib/mysql-mmm /usr/sbin /var/log/mysql-mmm /etc /etc/mysql-mmm /etc/init.d/
cp -r lib/Common/ /usr/share/perl5/vendor_perl/MMM
[ -f /etc/mysql-mmm/mmm_common.conf ] || cp etc/mysql-mmm/mmm_common.conf /etc/mysql-mmm/
mkdir -p /usr/lib/mysql-mmm/agent/
cp -r lib/Agent/ /usr/share/perl5/vendor_perl/MMM
cp -r bin/agent/* /usr/lib/mysql-mmm/agent/
cp -r etc/init.d/mysql-mmm-agent /etc/init.d/
cp sbin/mmm_agentd /usr/sbin
[ -f /etc/mysql-mmm/mmm_agent.conf ] || cp etc/mysql-mmm/mmm_agent.conf /etc/mysql-mmm/
mkdir -p /usr/lib/mysql-mmm/monitor/
cp -r lib/Monitor/ /usr/share/perl5/vendor_perl/MMM
cp -r bin/monitor/* /usr/lib/mysql-mmm/monitor/
cp -r etc/init.d/mysql-mmm-monitor /etc/init.d/
cp sbin/mmm_control sbin/mmm_mond /usr/sbin
[ -f /etc/mysql-mmm/mmm_mon.conf ] || cp etc/mysql-mmm/mmm_mon.conf /etc/mysql-mmm/
mkdir -p /usr/lib/mysql-mmm/tools/
cp -r lib/Tools/ /usr/share/perl5/vendor_perl/MMM
cp -r bin/tools/* /usr/lib/mysql-mmm/tools/
cp sbin/mmm_backup sbin/mmm_clone sbin/mmm_restore /usr/sbin
[ -f /etc/mysql-mmm/mmm_tools.conf ] || cp etc/mysql-mmm/mmm_tools.conf /etc/mysql-mmm/
----------------------------------------------------------------------------------------------------
- 配置MySQL-MMM相关目录
shell> mkdir /usr/local/mysql-mmm/conf
shell> mv /etc/mysql-mmm/* /usr/local/mysql-mmm/conf
shell> rm -rf /etc/mysql-mmm
shell> ln -s /usr/local/mysql-mmm/conf /etc/mysql-mmm
shell> mkdir /usr/local/mysql-mmm/log
shell> mkdir /usr/local/mysql-mmm/status
shell> mkdir /usr/local/mysql-mmm/run
- 在MySQL中配置MMM用户(如非全库同步则每个库都要做)
当建立好MySQL集群后，在其中一台主服务器中登入MySQL并执行以下指令建立并授权监控用户：
mysql> GRANT SUPER, REPLICATION CLIENT, PROCESS ON *.* TO 'mmm_agent'@'%' IDENTIFIED BY 'mmm_agent_123456';
mysql> GRANT REPLICATION CLIENT ON *.* TO 'mmm_monitor'@'%' IDENTIFIED BY 'mmm_monitor_123456';
mysql> flush privileges;
然后在db-master2和db-slave两台机器上查看，发现上面在db-master1机器上授权的账号已经同步过来了！
- 配置具体的配置文件，其中mmm_common.conf，mmm_agent.conf为agent端的配置文件，mmm_mon.conf为monitor端的配置文件
#先在db-master1主机上配置agent的mmm_common.conf文件（这个在所有机器上都要配置,包括monitor机器）
MySQL-MMM的mmm_common.conf配置文件在Agent端及Monitor端都存在mmm_common.conf配置文件且配置内容相同，在mysql-monitor、mysql-db1、mysql-db2及mysql-db3设定mmm_common.conf配置文件如下：
在mysql-monitor主机上进行mmm_common.conf文件配置
[root@mmm-monitor ~]# vim /usr/local/mysql-mmm/conf/mmm_common.conf
-------------------------------------------------------------------------------------------------------------------
active_master_role writer
<host default>
cluster_interface eth0
agent_port 9989
mysql_port 3306
pid_path /usr/local/mysql-mmm/run/mmm_agentd.pid
bin_path /usr/lib/mysql-mmm
replication_user repl
replication_password rep123
agent_user mmm_agent
agent_password mmm_agent_123456
</host>
<host mysql132>
ip 192.168.40.132
mode master
peer mysql133
</host>
<host mysql133>
ip 192.168.40.133
mode master
peer mysql132
</host>
<host mysql134>
ip 192.168.40.134
mode slave
</host>
<role writer>
hosts mysql132,mysql133
ips 192.168.40.136
mode exclusive
</role>
<role reader>
hosts mysql133,mysql134
ips 192.168.40.137,192.168.40.138
mode balanced
</role>
-------------------------------------------------------------------------------------------------------------------
配置解释，其中：
replication_user 用于检查复制的用户
agent_user为agent的用户
mode标明是否为主或者备选主，或者从库。
mode exclusive主为独占模式，同一时刻只能有一个主
<role write>中hosts表示目前的主库和备选主的真实主机ip或者主机名，ips为对外提供的虚拟机ip地址
<role readr>中hosts代表从库真实的ip和主机名，ips代表从库的虚拟ip地址。
在mysql139 主机配置mmm_common.conf文件后把其复制到mysql132、mysql133、mysql134中：
shell> scp -r /usr/local/mysql-mmm/conf/mmm_common.conf \
mysql132:/usr/local/mysql-mmm/conf/
shell> scp -r /usr/local/mysql-mmm/conf/mmm_common.conf \
mysql133:/usr/local/mysql-mmm/conf/
shell> scp -r /usr/local/mysql-mmm/conf/mmm_common.conf \
mysql134:/usr/local/mysql-mmm/conf/
分别在mysql132，mysql133，mysql134三台主机的/etc/mysql-mmm配置mmm_agent.conf文件，分别用不同的字符标识。注意这个文件的this db1这行内容要修改
为各自的主机名。比如本环境中，db-master1要配置this db-master1，db-master2要配置为this db-master2，而db-slave要配置为this db-slave。
在mysql132（1492.168.40.132）上：
[root@db-master1 ~]# vim /usr/local/mysql-mmm/conf/mmm_agent.conf
include mmm_common.conf
this mysql132
在mysql133（1492.168.40.133）上：：
[root@db-master2 ~]# vim /usr/local/mysql-mmm/conf/mmm_agent.conf
include mmm_common.conf
this mysql133
在mysql134（1492.168.40.134）上：：
[root@db-slave ~]# vim /usr/local/mysql-mmm/conf/mmm_agent.conf
include mmm_common.conf
this mysql134
接着在mmm-monit（192.168.40.139）配置monitor的配置文件：
[root@mmm-monit ~]# vim /usr/local/mysql-mmm/conf/mmm_mon.conf
--------------------------------------------------------------------------------------------------------
include mmm_common.conf
<monitor>
ip 127.0.0.1
pid_path /usr/local/mysql-mmm/run/mmm_mond.pid
bin_path /usr/lib/mysql-mmm
status_path /usr/local/mysql-mmm/status/mmm_mond.status
ping_ips 192.168.40.132,192.168.40.133,192.168.40.134
auto_set_online 10
# The kill_host_bin does not exist by default, though the monitor will
# throw a warning about it missing. See the section 5.10 "Kill Host
# Functionality" in the PDF documentation.
#
# kill_host_bin /usr/lib/mysql-mmm/monitor/kill_host
#
</monitor>
<host default>
monitor_user mmm_monitor
monitor_password mmm_monitor_123456
</host>
debug 0
-----------------------------------------------------------------------------------------------------------
7.启动agent和monitor服务
修改/etc/init.d/中mmm的启动服务脚本：
mysql132、mysql133、mysql134服务器：
shell> sed -i 's/var\/run\/mmm_agentd/usr\/local\/mysql-mmm\/run\/mmm_agentd/g' /etc/init.d/mysql-mmm-agent
mysql139服务器：
shell> sed -i 's/var\/run\/mmm_mond/usr\/local\/mysql-mmm\/run\/mmm_mond/g' /etc/init.d/mysql-mmm-monitor
在MySQL数据库的服务器 (mysql132、mysql133、mysql134) 中启动MMM的agent服务，并设置开机启动MMM Agent服务：
shell> chkconfig mysql-mmm-agent on
shell> service mysql-mmm-agent start
在Mysql-Monitor服务器中启动MMM的monitor服务，并设置开机启动MMM Monitor服务：
shell> chkconfig mysql-mmm-monitor on
shell> service mysql-mmm-monitor start
在mysql-monitor服务器中执行以下指令查看各个MySQL的读写及VIP状态：
shell> mmm_control show
查看mysql-monitor服务器中以monitor连接agent的端口：
shell> netstat -utlnap | grep 9989
monitor进程启动后，如下查看，发现进程并没有起来！
[root@mmm-monit ~]# /etc/init.d/mysql-mmm-monitor status or service mysql-mmm-monitor status
在monitor主机上检查集群主机的状态
[root@mmm-monit ~]# mmm_control checks all
- MMM高可用测试
[root@mmm-monit ~]# mmm_control show
----------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/ONLINE. Roles: reader(192.168.40.137)
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.138)
------------------------------------------------------------------------------
#模拟db-master2（133）宕机，手动停止mysql服务
[root@MYSQL ~]# service mysqld stop
在mmm-monit机器上观察monitor日志
[root@mmm-monit ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
.........
2017/06/01 21:28:17 FATAL State of host 'db-master2' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
重新查看mmm集群的最新状态：
[root@mmm-monit ~]# mmm_control show
---------------------------------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/HARD_OFFLINE. Roles:
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.137), reader(192.168.40.138)
-----------------------------------------------------------------------------------------------------
发现之前添加到db-master2对外提供读服务器的虚拟ip，即137已经漂移到134机器上了.
[root@db-slave ~]# ip addr
测试mysql数据同步：
虽然db-master2机器的mysql服务关闭，但是由于它的vip漂移到db-slave机器上了，
所以此时db-master1和db-slave这个时候是主从复制关系。
在db-master1数据库里更新数据，会自动更新到db-slave数据库里。
------------------
接着重启db-master2的mysql服务，可以看到db-master2由HARD_OFFLINE转到AWAITING_RECOVERY。
这时候db-master2再次接管读请求。
[root@db-master2 ~]# service mysqld start
Starting MySQL.. SUCCESS!
在mmm-monit机器上观察monitor日志
[root@mmm-monit ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
.........
2017/06/01 21:36:00 FATAL State of host 'db-master2' changed from HARD_OFFLINE to AWAITING_RECOVERY
2017/06/01 21:36:12 FATAL State of host 'db-master2' changed from AWAITING_RECOVERY to ONLINE because of auto_set_online(10 seconds). It was in state AWAITING_RECOVERY for 12 seconds
[root@mmm-monit ~]# mmm_control show
--------------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/ONLINE. Roles: reader(192.168.40.137)
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.138)
---------------------------------------------------------------------------------
[root@db-master2 mysql-mmm]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:1b:6e:53 brd ff:ff:ff:ff:ff:ff
inet 182.48.115.237/27 brd 182.48.115.255 scope global eth0
inet 182.48.115.235/32 scope global eth0
inet6 fe80::5054:ff:fe1b:6e53/64 scope link
valid_lft forever preferred_lft forever
发现之前的vip资源又回到了db-master2机器上，db-master2重新接管了服务。
并且db-master2恢复后，在故障期间更新的数据也会自动和其它两台机器同步！
---------------------------------------------------------------------------------------------------
2）模拟db-master1主库宕机,手动关闭mysql服务
[root@db-master1 ~]# service mysqld stop
Shutting down MySQL.... SUCCESS!
在mmm-monit机器上观察monitor日志
[root@mmm-monit ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
.........
2017/06/01 21:43:36 FATAL State of host 'db-master1' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
查看mmm集群状态：
[root@mmm-monit ~]# mmm_control show
--------------------------------------------------------------------------------------------------------------
mysql132(192.168.40.132) master/HARD_OFFLINE. Roles:
mysql133(192.168.40.133) master/ONLINE. Roles: reader(192.168.40.137), writer(192.168.40.136)
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.138)
-----------------------------------------------------------------------------------------------------------------
从上面可以发现，db-master1由以前的ONLINE转化为HARD_OFFLINE，移除了写角色，
因为db-master2是备选主，所以接管了写角色，db-slave指向新的主库db-master2，
应该说db-slave实际上找到了db-master2的sql现在的位置，
即db-master2的show master返回的值，然后直接在db-slave上change master to到db-master2。
db-master2机器上可以发现，db-master1对外提供写服务的vip漂移过来了
[root@db-master2 mysql-mmm]# ip addr
-----------------------------------------------------------------------------------------------------------------
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:0b:c1:c7 brd ff:ff:ff:ff:ff:ff
inet 192.168.40.133/24 brd 192.168.40.255 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.40.137/32 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.40.136/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe0b:c1c7/64 scope link
valid_lft forever preferred_lft forever
-------------------------------------------------------------------------------------------------------------------
这个时候，在db-master2数据库里更新数据，db-slave数据库会自动同步过去。
------------------------
接着重启db-master1的mysql
[root@db-master1 ~]# service mysqld start
Starting MySQL.. SUCCESS!
在mmm-monit机器上观察monitor日志
[root@mmm-monit ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
.........
2017/06/01 21:52:14 FATAL State of host 'db-master1' changed from HARD_OFFLINE to AWAITING_RECOVERY
再次查看mmm集群状态（发现写服务的vip转移到db-master2上了）：
[root@mmm-monit ~]# mmm_control show
db-master1(182.48.115.236) master/ONLINE. Roles:
db-master2(182.48.115.237) master/ONLINE. Roles: reader(182.48.115.235), writer(182.48.115.234)
db-slave(182.48.115.238) slave/ONLINE. Roles: reader(182.48.115.239)
发现db-master1虽然恢复了，并已经上线在集群中，但是其之前绑定的写服务的vip并没有从db-master2上转移回来，
即db-master1恢复后没有重新接管服务。只有等到db-master2发生故障时，才会把136的写服务的vip转移到
db-master1上，同时把137的读服务的vip转移到db-slave机器上
（然后db-master2恢复后，就会把转移到db-slave上的137的读服务的vip再次转移回来）。
133 上 mysql 关闭后
-----------------------------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/HARD_OFFLINE. Roles:
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.137), reader(192.168.40.138)
133 上 mysql 启动后
-----------------------------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/ONLINE. Roles: reader(192.168.40.137)
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.138)
---------------------------------------------------------------------------------------------------
再接着模拟db-slave从库宕机，手动关闭mysql服务
[root@db-slave ~]# service mysqld stop
Shutting down MySQL.. [确定]
在mmm-monit机器上观察monitor日志
[root@mmm-monit ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
.........
2017/06/01 22:42:24 FATAL State of host 'db-slave' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
查看mmm集群的最新状态：
[root@mmm-monit ~]# mmm_control show
------------------------------------------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/ONLINE. Roles: reader(192.168.40.137), reader(192.168.40.138)
mysql134(192.168.40.134) slave/HARD_OFFLINE. Roles:
-------------------------------------------------------------------------------------------------------------
发现db-slave发生故障后，其读服务的182.48.115.239的vip转移到db-master2上了。
[root@db-master2 mysql-mmm]# db-slave从库 启动后
----------------------------------------------------------------------------------------------------------------
mysql132(192.168.40.132) master/ONLINE. Roles: writer(192.168.40.136)
mysql133(192.168.40.133) master/ONLINE. Roles: reader(192.168.40.138)
mysql134(192.168.40.134) slave/ONLINE. Roles: reader(192.168.40.137)
----------------------------------------------------------------------------------------------------------------
当db-slave恢复后，读服务的vip还是再次转移回来，即重新接管服务。并且故障期间更新的数据会自动同步回来。
需要注意：
db-master1，db-master2，db-slave之间为一主两从的复制关系，
一旦发生db-master2，db-slave延时于db-master1时，这个时刻db-master1 mysql宕机，
db-slave将会等待数据追上db-master1后，再重新指向新的主db-master2，
进行change master to db-master2操作，在db-master1宕机的过程中，一旦db-master2落后于db-master1，
这时发生切换，db-master2变成了可写状态，数据的一致性将会无法保证。
9.其他
[root@mysql-vip ~]# service mysql-mmm-monitor status
Daemon bin: '/usr/sbin/mmm_mond'
Daemon pid: '/usr/local/mysql-mmm/run/mmm_mond.pid'
Checking MMM Monitor process: running.
[root@mysql-vip ~]# mmm_control set_offline mysql-db2
OK: State of 'mysql-db2' changed to ADMIN_OFFLINE. Now you can wait some time and check all roles!
[root@mysql-vip ~]# mmm_control show
mysql-db1(10.0.101.205) master/ONLINE. Roles: writer(10.0.101.230)
mysql-db2(10.0.101.206) master/ADMIN_OFFLINE. Roles:
mysql-db3(10.0.101.216) slave/ONLINE. Roles: reader(10.0.101.231), reader(10.0.101.232)
[root@mysql-vip ~]# mmm_control set_offline mysql-db3
OK: State of 'mysql-db3' changed to ADMIN_OFFLINE. Now you can wait some time and check all roles!
[root@mysql-vip ~]# mmm_control show
mysql-db1(10.0.101.205) master/ONLINE. Roles: writer(10.0.101.230)
mysql-db2(10.0.101.206) master/ADMIN_OFFLINE. Roles:
mysql-db3(10.0.101.216) slave/ADMIN_OFFLINE. Roles:
[root@mysql-vip ~]# mmm_control set_online mysql-db3
OK: State of 'mysql-db3' changed to ONLINE. Now you can wait some time and check its new roles!
[root@mysql-vip ~]# mmm_control show
mysql-db1(10.0.101.205) master/ONLINE. Roles: writer(10.0.101.230)
mysql-db2(10.0.101.206) master/ADMIN_OFFLINE. Roles:
mysql-db3(10.0.101.216) slave/REPLICATION_DELAY. Roles:
[root@mysql-vip ~]# mmm_control show
mysql-db1(10.0.101.205) master/ONLINE. Roles: writer(10.0.101.230)
mysql-db2(10.0.101.206) master/ADMIN_OFFLINE. Roles:
mysql-db3(10.0.101.216) slave/ONLINE. Roles: reader(10.0.101.231), reader(10.0.101.232)
[root@mysql-vip ~]# mmm_control set_online mysql-db2
OK: State of 'mysql-db2' changed to ONLINE. Now you can wait some time and check its new roles!
[root@mysql-vip ~]# mmm_control show
mysql-db1(10.0.101.205) master/ONLINE. Roles: writer(10.0.101.230)
mysql-db2(10.0.101.206) master/ONLINE. Roles: reader(10.0.101.232)
mysql-db3(10.0.101.216) slave/ONLINE. Roles: reader(10.0.101.231)
