一、前言
尝试通过ansible 来在centos 7系统环境下部署实现nginx+keepalived+LAMP环境,以加深对ansible的理解。部署拓扑如下:
两台nginx作为web代理服务器,配置实现keepalived做主备;后端两个apache服务器,一个部署apache+php,另一个部署apache+mysql。通过ansible管理配置以上服务器,配置完成后,能通过VIP访问到后端主机主页。
二、环境准备
因为控制终端的程序包什么的都可以通过ansible主机来管理安装配置,因此这里的我们只需要在ansible主机上安装ansible服务,并将相应的远程主机添加管理即可。
1、安装ansible服务
在ansible主机上安装ansible服务:
[root@ansible ~]# yum install -y ansible
2、添加管理远程主机
编辑/etc/ansible/hosts:
[nginx]
192.168.0.87
192.168.0.89
[apache]
192.168.0.83
192.168.0.84
[php]
192.168.0.83
[mysql]
192.168.0.84
编辑/etc/hosts,添加相应的主机名解析:
[root@ansible ~]# vim /etc/hosts
192.168.0.83 ap1.ilinux.io ap1
192.168.0.84 ap2.ilinux.io ap2
192.168.0.87 nginx1.ilinux.io nginx1
192.168.0.89 nginx2.ilinux.io nginx2
3、配置使用ssh免密钥认证管理远程主机
首先在ansible主机上创建相应的密钥文件:
[root@ansible ~]# ssh-keygen -t rsa -N ''
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
fc:af:d8:8f:41:dd:d2:00:2a:d2:07:e7:68:b1:c7:d3 root@ansible
The key's randomart image is:
+--[ RSA 2048]----+
| o . . |
| . O o . |
| . * B E . |
| o = . . + |
| S . o o |
| o . |
| o |
| o + |
| . +oo |
+-----------------+
然后将生成的公钥文件送往远程主机:
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.0.83
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.0.84
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.0.87
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.0.89
测试认证访问:
[root@ansible ~]# ansible all -m ping
192.168.0.83 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.0.89 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.0.87 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.0.84 | SUCCESS => {
"changed": false,
"ping": "pong"
}
4、配置时间同步
在部署前确保每个服务器的时间及时区是同步的:
[root@ansible ~]# ansible all -m shell -a 'echo "TZ='Asia/Shanghai'; export TZ" > /etc/profile '
配置定期同步时间:
[root@ansible ~]# ansible all -m cron -a "minute=*/3 job='/usr/sbin/ntpdate ntp1.aliyun.com &> /dev/null' name=dateupdate"
192.168.0.84 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
192.168.0.89 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
192.168.0.83 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
192.168.0.87 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
5、关闭firewalld和selinux
因此firewalld和selinux很容易对演示结果造成影响,因此这里我们统一把相应的服务器的firewalld和selinux关闭。生产环境中请按照实际需求进行定制。
[root@ansible ~]# ansible all -m shell -a 'systemctl stop firewalld; systemctl disable firewalld; setenforce 0'
192.168.0.87 | SUCCESS | rc=0 >>
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
192.168.0.89 | SUCCESS | rc=0 >>
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
192.168.0.84 | SUCCESS | rc=0 >>
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
192.168.0.83 | SUCCESS | rc=0 >>
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
三、配置roles
接着我们就可以在ansible主机上配置需要下发到各远程主机上的playbook了,这里我以roles角色定义各服务器上需要配置的服务,最后用playbook调用相应的roles进行下发配置。
1、 配置apache服务role
首先在/etc/ansible/roles目录下创建相关的目录:
[root@ansible ~]# mkdir -pv /etc/ansible/roles/apache/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/apache"
mkdir: 已创建目录 "/etc/ansible/roles/apache/files"
mkdir: 已创建目录 "/etc/ansible/roles/apache/templates"
mkdir: 已创建目录 "/etc/ansible/roles/apache/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/apache/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/apache/vars"
mkdir: 已创建目录 "/etc/ansible/roles/apache/meta"
mkdir: 已创建目录 "/etc/ansible/roles/apache/default"
[root@ansible ~]# ll /etc/ansible/roles/apache/
总用量 0
接着配置apache的配置模板:
[root@ansible ~]# vim /etc/ansible/roles/apache/templates/vhost1.conf.j2
<virtualhost *:80>
servername www.ilinux.io
DirectoryIndex index.html index.php
Documentroot /var/www/html
ProxyRequest off
ProxyPassMatch ^/(.*\.php)$ fcgi://192.168.0.83:9000/var/www/html/$1
ProxyPassMatch ^/(ping|status)$ fcgi://192.168.0.83:9000/$1
<Directory / >
options FollowSymlinks
Allowoverride none
Require all granted
</Directory>
</virtualhost>
配置apache的主页文件模板:
[root@ansible ~]# vim /etc/ansible/roles/apache/templates/
index.html
<h1>This is {{ ansible_hostname }}</h1>
[root@ansible ~]# vim /etc/ansible/roles/apache/templates/
index.php
<?php
phpinfo();
?>
配置apache的task任务:
[root@ansible ~]# vim /etc/ansible/roles/apache/tasks/main.yml
- name: install apache
yum: name=apache state=latest
- name: install vhost file
template: src=/etc/ansible/roles/apache/templates/vhost1.conf.j2 dest=/etc/httpd/conf.d/vhost.conf
- name: install index.html
template: src=/etc/ansible/roles/apache/templates/index.html dest=/var/www/html/index.html
- name: install index.php
template: src=/etc/ansible/roles/apache/templates/index.php dest=/var/www/html/index.php
- name: start httpd
service: name=httpd state=started
2、配置php-fpm服务的role
首先创建对应的roles角色目录:
[root@ansible ~]# mkdir -pv /etc/ansible/roles/php-fpm/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/files"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/templates"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/vars"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/meta"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/default"
接着将事先准备好的php-fpm配置模板文件,复制到指定的模板目录下并进行编辑:
[root@ansible ~]# cp /etc/php-fpm.d/www.conf /etc/ansible/roles/php-fpm/templates/www.conf
[root@ansible ~]# vim /etc/ansible/roles/php-fpm/templates/www.conf
#修改以下配置
listen = 0.0.0.0:9000
;listen.allowed_clients = 127.0.0.1
pm.status_path = /status
ping.path = /ping
ping.response = pong
然后配置相应的task任务文件:
[root@ansible ~]# vim /etc/ansible/roles/php-fpm/tasks/main.yml
- name: install epel repo
yum: name=epel-release state=latest
- name: install php package
yum: name={{ item }} state=latest
with_items:
- php-fpm
- php-mysql
- php-mbstring
- php-mcrypt
- name: install config file
template: src=/etc/ansible/roles/php-fpm/templates/www.conf dest=/etc/php-fpm.d/www.conf
- name: install session directory
file: path=/var/lib/php/session group=apache owner=apache state=directory
- name: start php-fpm
service: name=php-fpm state=started
3、配置mysql服务role
首先创建对应的mysql服务的roles目录:
[root@ansible ~]# mkdir -pv /etc/ansible/roles/mysql/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/mysql"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/files"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/templates"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/vars"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/meta"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/default"
将事先准备的mysql配置文件复制到指定的模板目录下并编辑:
[root@ansible ~]# cp /etc/my.cnf /etc/ansible/roles/mysql/templates/
[root@ansible ~]# vim /etc/ansible/roles/mysql/templates/my.cnf
#添加下面两行配置
skip-name-resolve=ON
innodb-file-per-table=ON
接着配置mysql服务的task任务:
[root@ansible ~]# vim /etc/ansible/roles/mysql/tasks/main.yml
- name: install mysql
yum: name=mariadb-server state=latest
- name: install config file
template: src=/etc/ansible/roles/mysql/templates/my.cnf dest=/etc/my.cnf
- name: start mysql
service: name=mariadb-server state=started
4、配置nginx服务的role
首先创建对应的ngixn服务的目录:
[root@ansible ~]# mkdir -pv /etc/ansible/roles/nginx/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/nginx"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/files"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/templates"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/vars"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/meta"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/default"
复制nginx的配置文件到指定的模板目录下并修改其内容:
[root@ansible ~]# cp /etc/nginx/nginx.conf /etc/ansible/roles/nginx/templates/
[root@ansible ~]# vim /etc/ansible/roles/nginx/templates/
http {
......
upstream apservers {
server 192.168.0.83:80;
server 192.168.0.84:80;
}
......
server {
......
location / {
proxy_pass http://apservers;
proxy_set_header host $http_host;
proxy_set_header X-Forward-For $remote_addr;
}
......
}
最后配置nignx服务role的task任务:
[root@ansible ~]# vim /etc/ansible/roles/nginx/tasks/main.yml
- name: install epel
yum: name=epel-release state=latest
- name: install nginx
yum: name=nginx state=latest
- name: install config file
template: src=/etc/ansible/roles/nginx/templates/nginx.conf dest=/etc/nginx/nginx.conf
- name: start nginx
service: name=nginx state=started
5、配置keepalived服务role
首先创建keepalived的role目录:
[root@ansible ~]# mkdir -pv /etc/ansible/roles/keepalived/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/keepalived"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/files"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/templates"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/vars"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/meta"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/default"
接着复制对应的配置文件到指定的模板目录下,并编辑:
[root@ansible ~]# cp /etc/keepalived/keepalived.conf /etc/ansible/roles/keepalived/templates/
[root@ansible ~]# vim /etc/ansible/roles/keepalived/templates/keepalived.conf
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id {{ ansible_nodename }}
vrrp_mcast_group4 224.1.101.33
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state {{ keepalived_role }}
interface ens33
virtual_router_id 51
priority {{ keepalived_pri }}
advert_int 1
authentication {
auth_type PASS
auth_pass IKHN^2(1
}
virtual_ipaddress {
192.168.0.99/24 dev ens33 label ens33:0
}
}
编辑/etc/ansible/hosts文件,给nginx主机添加指定的对应变量:
[root@ansible ~]# vim /etc/ansible/hosts
[nginx]
192.168.0.87 keepalived_role=MASTER keepalived_pri=100
192.168.0.89 keepalived_role=BACKUP keepalived_pri=98
接着配置nginx服务的task服务:
[root@ansible ~]# vim /etc/ansible/roles/keepalived/tasks/main.yml
- name: install keepalived
yum: name=keepalived state=latest
- name: install config file
template: src=/etc/ansible/roles/keepalived/templates/keepalived.conf dest=/etc/keepalived/keepalived.conf
- name: start keepalived
service: name=keepalived state=started
至此所有相关的服务的playbook roles已经定义完毕。
四、配置playbook下发配置
接着我们就来定义相应的playbook调用roles下发配置。
在/etc/ansible目录下创建目录playbooks用于存放playbook文件:
[root@ansible ~]# mkdir /etc/ansible/playbooks
1、定义ap1的playbook并下发
在/etc/ansible/playbook目录下创建ap1.yaml文件:
[root@ansible ~]# vim /etc/ansible/playbooks/ap1.yaml
#因为ap1又是apache服务器,也php-fpm服务器,所以调用apache和php-fpm两个role
- hosts: php
remote_user: root
roles:
- apache
- php-fpm
检查配置语法是否正确:
[root@ansible ~]# ansible-playbook --syntax-check /etc/ansible/playbooks/ap1.yaml
playbook: /etc/ansible/playbooks/ap1.yaml
下发安装ap1.yaml:
[root@ansible ~]# ansible-playbook /etc/ansible/playbooks/ap1.yaml
PLAY [php] ***************************************************************************************************************************************************
TASK [Gathering Facts] ***************************************************************************************************************************************
ok: [192.168.0.83]
TASK [apache : install apache] *******************************************************************************************************************************
changed: [192.168.0.83]
TASK [apache : install vhost file] ***************************************************************************************************************************
ok: [192.168.0.83]
TASK [apache : install index.html] ***************************************************************************************************************************
ok: [192.168.0.83]
TASK [apache : install index.php] ****************************************************************************************************************************
ok: [192.168.0.83]
TASK [apache : start httpd] **********************************************************************************************************************************
changed: [192.168.0.83]
TASK [php-fpm : install epel repo] ***************************************************************************************************************************
ok: [192.168.0.83]
TASK [php-fpm : install php package] *************************************************************************************************************************
changed: [192.168.0.83] => (item=[u'php-fpm', u'php-mysql', u'php-mbstring', u'php-mcrypt'])
TASK [php-fpm : install config file] *************************************************************************************************************************
changed: [192.168.0.83]
TASK [php-fpm : install session directory] *******************************************************************************************************************
changed: [192.168.0.83]
TASK [php-fpm : start php-fpm] *******************************************************************************************************************************
changed: [192.168.0.83]
PLAY RECAP ***************************************************************************************************************************************************
192.168.0.83 : ok=11 changed=6 unreachable=0 failed=0
此时在ap1上的apache服务和php-fpm服务已经已经正常启动并监控了。
2、 定义ap2的playbook并下发
编辑配置ap2.yaml:
[root@ansible ~]# vim /etc/ansible/playbooks/ap2.yaml
- hosts: mysql
remote_user: root
roles:
- apache
- mysql
检查是否有语法错误:
[root@ansible ~]# ansible-playbook --syntax-check /etc/ansible/playbooks/ap2.yaml
playbook: /etc/ansible/playbooks/ap2.yaml
下发安装ap2.yaml:
[root@ansible ~]# ansible-playbook /etc/ansible/playbooks/ap2.yaml
PLAY [mysql] *************************************************************************************************************************************************
TASK [Gathering Facts] ***************************************************************************************************************************************
ok: [192.168.0.84]
TASK [apache : install apache] *******************************************************************************************************************************
changed: [192.168.0.84]
TASK [apache : install vhost file] ***************************************************************************************************************************
ok: [192.168.0.84]
TASK [apache : install index.html] ***************************************************************************************************************************
ok: [192.168.0.84]
TASK [apache : install index.php] ****************************************************************************************************************************
ok: [192.168.0.84]
TASK [apache : start httpd] **********************************************************************************************************************************
changed: [192.168.0.84]
TASK [mysql : install mysql] *********************************************************************************************************************************
changed: [192.168.0.84]
TASK [mysql : install config file] ***************************************************************************************************************************
changed: [192.168.0.84]
TASK [mysql : start mysql] ***********************************************************************************************************************************
changed: [192.168.0.84]
PLAY RECAP ***************************************************************************************************************************************************
192.168.0.84 : ok=9 changed=5 unreachable=0 failed=0
此时ap2上的apache服务和mysql服务应该已经监听启用。
3、定义两台nginx服务器的playbook并下发
编辑创建loadbalance.yaml:
[root@ansible ~]# vim /etc/ansible/playbooks/loadbalance.yaml
- hosts: nginx
remote_user: root
roles:
- nginx
- keepalived
检查是否有语法错误:
[root@ansible ~]# ansible-playbook --syntax-check /etc/ansible/playbooks/loadbalance.yaml
playbook: /etc/ansible/playbooks/ap2.yaml
下发安装loadbalance.yaml:
[root@ansible ~]# ansible-playbook /etc/ansible/playbooks/loadbalance.yaml
PLAY [nginx] *************************************************************************************************************************************************
TASK [Gathering Facts] ***************************************************************************************************************************************
ok: [192.168.0.87]
ok: [192.168.0.89]
TASK [nginx : install epel] **********************************************************************************************************************************
changed: [192.168.0.87]
changed: [192.168.0.89]
TASK [nginx : install nginx] *********************************************************************************************************************************
changed: [192.168.0.87]
changed: [192.168.0.89]
TASK [nginx : install config file] ***************************************************************************************************************************
changed: [192.168.0.89]
changed: [192.168.0.87]
TASK [nginx : start nginx] ***********************************************************************************************************************************
changed: [192.168.0.89]
changed: [192.168.0.87]
TASK [keepalived : install keepalived] ***********************************************************************************************************************
changed: [192.168.0.89]
changed: [192.168.0.87]
TASK [keepalived : install config file] **********************************************************************************************************************
changed: [192.168.0.89]
changed: [192.168.0.87]
TASK [keepalived : start keepalived] *************************************************************************************************************************
changed: [192.168.0.89]
changed: [192.168.0.87]
PLAY RECAP ***************************************************************************************************************************************************
192.168.0.87 : ok=8 changed=7 unreachable=0 failed=0
192.168.0.89 : ok=8 changed=7 unreachable=0 failed=0
此时在两台nginx服务器上的keepalived和nginx应该都正常启动了。
4、测试访问
此时尝试通过vip访问后端ap1和ap2的主页:
[root@ansible ~]# for i in {1..10} ; do curl http://192.168.0.99/ ; done
<h1>This is ap2</h1>
<h1>This is ap1</h1>
<h1>This is ap2</h1>
<h1>This is ap1</h1>
<h1>This is ap2</h1>
<h1>This is ap1</h1>
<h1>This is ap2</h1>
<h1>This is ap1</h1>
<h1>This is ap2</h1>
<h1>This is ap1</h1>
#访问成功
把其中一个nginx主机的keepalived服务停用,观察主备是否切换:
[root@nginx2 ~]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2018-06-07 15:26:47 CST; 8min ago
Main PID: 10857 (keepalived)
CGroup: /system.slice/keepalived.service
├─10857 /usr/sbin/keepalived -D
├─10858 /usr/sbin/keepalived -D
└─10859 /usr/sbin/keepalived -D
Jun 07 15:26:47 nginx2 Keepalived_healthcheckers[10858]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 07 15:34:50 nginx2 Keepalived_vrrp[10859]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: Sending gratuitous ARP on ens33 for 192.168.0.99
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.0.99
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: Sending gratuitous ARP on ens33 for 192.168.0.99
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: Sending gratuitous ARP on ens33 for 192.168.0.99
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: Sending gratuitous ARP on ens33 for 192.168.0.99
Jun 07 15:34:51 nginx2 Keepalived_vrrp[10859]: Sending gratuitous ARP on ens33 for 192.168.0.99
keepalived主备切换成功,此时服务依旧能正常访问。
至此使用ansible部署管理高可用的LAMP环境就已经完成了,后续可在此环境上搭建相应的wordpress或其他的应用服务。(不得不说,ansible真方便。。。)