使用ansible 来在centos 7系统环境下部署实现nginx+keepalived+LAMP环境
一、环境准备
1、安装ansible服务
[root@ansible-70 ~]# yum install epel-release -y #安装epel源
[root@ansible-70 ~]# yum -y install ansible
2、添加管理远程主机和本地解析
编辑/etc/ansible/hosts:
[root@ansible-70 ~]# vim /etc/ansible/hosts
....
[nginx] #定义nginx主机组
192.168.1.71
192.168.1.72
[apache] #定义Apache主机组
192.168.1.73
192.168.1.74
[php] #定义php主机组
192.168.1.73
[mysql] #定义mysql主机组
192.168.1.74
编辑/etc/hosts,添加相应的主机名解析
[root@ansible-70 ~]# vim /etc/hosts
......
192.168.1.71 keepalive1-71
192.168.1.72 keepalive2-72
192.168.1.73 httpd1-73
192.168.1.74 httpd2-74
3、配置使用ssh免密钥认证管理远程主机
在ansible主机上使用ssh生成秘钥:
[root@ansible-70 ~]# ssh-keygen -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:cGdyniKMaOzYLxH9qetFr6LADwBCtxL/NmRtvFPWVgk root@ansible-70
The key's randomart image is:
+---[RSA 2048]----+
| o . E... |
|. + . o . .. |
|o. + o.+oo+o |
|o.o.=o.o+*.. |
|. +..=++S o |
|o=. ..+o.. |
|ooo. .. . |
| .+.o. . |
| .=+o. |
+----[SHA256]-----+
向目标主机传递密钥,完成认证管理远程主机
[root@ansible-70 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.1.71
[root@ansible-70 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.1.72
[root@ansible-70 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.1.73
[root@ansible-70 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.1.74
列出所有目标主机并ping测试
[root@ansible-70 ~]# ansible all --list-hosts
hosts (4):
192.168.1.71
192.168.1.72
192.168.1.74
192.168.1.73
[root@ansible-70 ~]# ansible all -m ping #对所有目标主机测试
192.168.1.73 | SUCCESS => { #成功
"changed": false,
"ping": "pong"
}
192.168.1.74 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.1.71 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.1.72 | SUCCESS => {
"changed": false,
"ping": "pong"
}
4、配置时间同步
在部署前确保每个服务器的时间及时区必须是同步的:
[root@ansible-70 ~]# ansible all -m shell -a 'echo "TZ='Asia/Shanghai'; export TZ" > /etc/profile '
192.168.1.74 | SUCCESS | rc=0 >>
192.168.1.72 | SUCCESS | rc=0 >>
192.168.1.71 | SUCCESS | rc=0 >>
192.168.1.73 | SUCCESS | rc=0 >
指定周期时间同步计划:
[root@ansible-70 ~]# ansible all -m cron -a "minute=*/3 job='/usr/sbin/ntpdate ntp1.aliyun.com &> /dev/null' name=dateupdate"
192.168.1.72 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
192.168.1.71 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
192.168.1.73 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
192.168.1.74 | SUCCESS => {
"changed": true,
"envs": [],
"jobs": [
"dateupdate"
]
}
5、关闭firewalld和selinux
为了便于演示需要关闭firewalld和selinux
[root@ansible-70 ~]# ansible all -m shell -a 'systemctl stop firewalld; systemctl disable firewalld; setenforce 0'
二、配置角色(roles)
1、 配置后端apache服务role
在/etc/ansible/roles目录下创建相关的目录:
[root@ansible-70 ~]# mkdir -pv /etc/ansible/roles/apache/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/apache"
mkdir: 已创建目录 "/etc/ansible/roles/apache/files"
mkdir: 已创建目录 "/etc/ansible/roles/apache/templates"
mkdir: 已创建目录 "/etc/ansible/roles/apache/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/apache/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/apache/vars"
mkdir: 已创建目录 "/etc/ansible/roles/apache/meta"
mkdir: 已创建目录 "/etc/ansible/roles/apache/default"
接着配置apache的配置模板:
[root@ansible-70 ~]# vim /etc/ansible/roles/apache/templates/vhost1.conf.j2
<virtualhost *:80>
servername www.hehe.io
DirectoryIndex index.html index.php
Documentroot /var/www/html
ProxyRequests off
ProxyPassMatch ^/(.*\.php)$ fcgi://192.168.1.73:9000/var/www/html/$1 #匹配动态页面
ProxyPassMatch ^/(ping|status)$ fcgi://192.168.0.73:9000/$1
<Directory / >
options FollowSymlinks
Allowoverride none
Require all granted
</Directory>
</virtualhost>
[root@ansible-70 ~]# httpd -t #语法检查
编辑apache测试主页模板:
[root@ansible-70 ~]# vim /etc/ansible/roles/apache/templates/index.html#静态请求页面
<h1>This is {{ ansible_hostname }}</h1>
[root@ansible-70 ~]# vim /etc/ansible/roles/apache/templates/index.php#动态请求页面
<?php
phpinfo();
?>
配置apache的task标签任务:
[root@ansible-70 ~]# vim /etc/ansible/roles/apache/tasks/main.yml
- name: install apache #安装httpd
yum: name=httpd state=latest
- name: install vhost file #传递httpd虚拟机配置文件
template: src=/etc/ansible/roles/apache/templates/vhost1.conf.j2 dest=/etc/httpd/conf.d/vhost.conf
- name: install index.html #传递httpd测试页
template: src=/etc/ansible/roles/apache/templates/index.html dest=/var/www/html/index.html
- name: install index.php #传递php测试页
template: src=/etc/ansible/roles/apache/templates/index.php dest=/var/www/html/index.php
- name: start httpd #启动httpd
service: name=httpd state=started
2、配置php-fpm服务的role
先创建对应的roles角色目录:
[root@ansible-70 ~]# mkdir -pv /etc/ansible/roles/php-fpm/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/files"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/templates"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/vars"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/meta"
mkdir: 已创建目录 "/etc/ansible/roles/php-fpm/default"
安装php-fpm用于得到配置文件,复制到指定的模板目录下并进行编辑
[root@ansible-70 ~]# yum install php-fpm -y #安装php-fpm用于得到配置文件
[root@ansible-70 ~]# cp /etc/php-fpm.d/www.conf /etc/ansible/roles/php-fpm/templates/www.conf #拷贝配置文件到角色目录下
[root@ansible-70 ~]# vim /etc/ansible/roles/php-fpm/templates/www.conf
#修改这些配置
listen = 0.0.0.0:9000
;listen.allowed_clients = 127.0.0.1
pm.status_path = /status
ping.path = /ping
ping.response = pong
配置相应的task标签任务文件:
[root@ansible-70 ~]# vim /etc/ansible/roles/php-fpm/tasks/main.yml
- name: install epel repo #安装epel仓库
yum: name=epel-release state=latest
- name: install php package #安装php包
yum: name={{ item }} state=latest #依次执行安装with_item中的程序
with_items:
- php-fpm
- php-mysql
- php-mbstring
- php-mcrypt
- name: install config file #传递php-fpm配置文件
template: src=/etc/ansible/roles/php-fpm/templates/www.conf dest=/etc/php-fpm.d/www.conf
- name: install session directory #创建php会话保存目录
file: path=/var/lib/php/session group=apache owner=apache state=directory
- name: start php-fpm #启动php-fpm
service: name=php-fpm state=started
3、配置mysql服务role
先创建对应的mysql服务的roles目录:
[root@ansible-70 ~]# mkdir -pv /etc/ansible/roles/mysql/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/mysql"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/files"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/templates"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/vars"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/meta"
mkdir: 已创建目录 "/etc/ansible/roles/mysql/default"
本机安装mysql或到其他主机找一个mysql的my.cnf配置文件,拷贝到角色指定目录下进行编辑
[root@ansible-70 ~]# cp /etc/my.cnf /etc/ansible/roles/mysql/templates/#拷贝配置文件
[root@ansible-70 ~]# vim /etc/ansible/roles/mysql/templates/my.cnf
skip-name-resolve=ON #添加此句
innodb-file-per-table=ON #添加此句
配置mysql服务的task标签任务:
[root@ansible-70 ~]# vim /etc/ansible/roles/mysql/tasks/main.yml
- name: install mysql #安装mariadb服务
yum: name=mariadb-server state=latest
- name: install config file #传递mariadb配置文件
template: src=/etc/ansible/roles/mysql/templates/my.cnf dest=/etc/my.cnf
- name: start mysql #启动mariadb服务
service: name=mariadb state=started
4、配置nginx服务的role
先创建对应的ngixn服务的目录:
[root@ansible-70 ~]# mkdir -pv /etc/ansible/roles/nginx/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/nginx"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/files"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/templates"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/vars"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/meta"
mkdir: 已创建目录 "/etc/ansible/roles/nginx/default"
本机安装或到其他主机找一个复制nginx的配置文件到指定的模板目录下并修改其内容:
[root@ansible-70 ~]# cp /etc/nginx/nginx.conf /etc/ansible/roles/nginx/templates/
[root@ansible-70 ~]# vim /etc/ansible/roles/nginx/templates/nginx.conf
http {
......
upstream apservers {
server 192.168.0.83:80;
server 192.168.0.84:80;
}
......
server {
......
location / {
proxy_pass http://apservers; #反向代理到apservers组
proxy_set_header host $http_host;
proxy_set_header X-Forward-For $remote_addr;
}
......
}
[root@ansible-70 ~]# nginx -t #语法检查
配置nignx服务role的task任务:
[root@ansible-70 ~]# vim /etc/ansible/roles/nginx/tasks/main.yml
- name: install epel #安装epel仓库
yum: name=epel-release state=latest
- name: install nginx #安装nginx
yum: name=nginx state=latest
- name: install config file #传递nginx配置文件
template: src=/etc/ansible/roles/nginx/templates/nginx.conf dest=/etc/nginx/nginx.conf
- name: start nginx #启动nginx服务
service: name=nginx state=started
5、配置keepalived服务role
先创建keepalived的role目录:
[root@ansible-70 ~]# mkdir -pv /etc/ansible/roles/keepalived/{files,templates,tasks,handlers,vars,meta,default}
mkdir: 已创建目录 "/etc/ansible/roles/keepalived"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/files"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/templates"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/tasks"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/handlers"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/vars"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/meta"
mkdir: 已创建目录 "/etc/ansible/roles/keepalived/default"
本机安装或到其他主机找一个keepalive的配置文件,复制文件到指定的模板目录下,并编辑:
[root@ansible-70 ~]# cp /etc/keepalived/keepalived.conf /etc/ansible/roles/keepalived/templates/
[root@ansible-70 ~]# vim /etc/ansible/roles/keepalived/templates/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id {{ ansible_nodename }}
vrrp_mcast_group4 224.1.101.33
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state {{ keepalived_role }}
interface ens33
virtual_router_id 51
priority {{ keepalived_pri }}
advert_int 1
authentication {
auth_type PASS
auth_pass IKHN^2(1
}
virtual_ipaddress {
192.168.1.99/24 dev ens33 label ens33:0
}
}
编辑/etc/ansible/hosts文件,给nginx主机添加指定的对应变量:
[root@ansible-70 ~]# vim /etc/ansible/hosts
[nginx]
192.168.1.71 keepalived_role=MASTER keepalived_pri=100 #传递参数设置为主机,优先级100
192.168.1.72 keepalived_role=BACKUP keepalived_pri=98 #传递参数设置为从机,优先级98
配置nginx服务的task标签服务:
[root@ansible-70 ~]# vim /etc/ansible/roles/keepalived/tasks/main.yml
- name: install keepalived #安装keepalived服务
yum: name=keepalived state=latest
- name: install config file #传递配置文件
template: src=/etc/ansible/roles/keepalived/templates/keepalived.conf dest=/etc/keepalived/keepalived.conf
- name: start keepalived #启动keepalived服务
service: name=keepalived state=started
所有相关的服务的playbook roles已经设置完成。
四、配置playbook下发配置
定义相应的playbook调用roles,对目标主机下发配置。
在/etc/ansible目录下创建目录playbooks用于存放playbook文件:
[root@ansible-70 ~]# mkdir /etc/ansible/playbooks
1、定义httpd1的playbook并下发
在/etc/ansible/playbook目录下创建httpd1.yaml文件:
[root@ansible-70 ~]# vim /etc/ansible/playbooks/httpd1.yaml
- hosts: php #面对PHP主机组
remote_user: root
roles:
- apache #执行apache模板
- php-fpm #执行PHP-fpm模板
[root@ansible-70 ~]# ansible-playbook --syntax-check /etc/ansible/playbooks/httpd1.yaml #语法测试
playbook: /etc/ansible/playbooks/httpd1.yaml
下发安装httpd1.yaml:
[root@ansible-70 ~]# ansible-playbook /etc/ansible/playbooks/httpd1.yaml
PLAY [php] *********************************************************************************************
TASK [Gathering Facts] *********************************************************************************
ok: [192.168.1.73]
TASK [apache : install apache] *************************************************************************
ok: [192.168.1.73]
TASK [apache : install vhost file] *********************************************************************
ok: [192.168.1.73]
TASK [apache : install index.html] *********************************************************************
ok: [192.168.1.73]
TASK [apache : install index.php] **********************************************************************
ok: [192.168.1.73]
TASK [apache : start httpd] ****************************************************************************
changed: [192.168.1.73]
TASK [php-fpm : install epel repo] *********************************************************************
changed: [192.168.1.73]
TASK [php-fpm : install php package] *******************************************************************
changed: [192.168.1.73] => (item=[u'php-fpm', u'php-mysql', u'php-mbstring', u'php-mcrypt'])
TASK [php-fpm : install config file] *******************************************************************
changed: [192.168.1.73]
TASK [php-fpm : install session directory] *************************************************************
changed: [192.168.1.73]
TASK [php-fpm : start php-fpm] *************************************************************************
changed: [192.168.1.73]
PLAY RECAP *********************************************************************************************
192.168.1.73 : ok=11 changed=6 unreachable=0 failed=0
目标主机httpd1此时的apache和php安装配置完毕,并且已经启动
2、 定义httpd2的playbook并下发
编辑配置httpd2.yaml:
[root@ansible-70 ~]# vim /etc/ansible/playbooks/httpd2.yaml
- hosts: mysql #面向mysql主机组
remote_user: root
roles:
- apache #执行apache模板
- mysql #执行mysql模板
下发安装httpd2.yaml:
[root@ansible-70 ~]# ansible-playbook /etc/ansible/playbooks/httpd2.yaml
PLAY [mysql] *******************************************************************************************
TASK [Gathering Facts] *********************************************************************************
ok: [192.168.1.74]
TASK [apache : install apache] *************************************************************************
ok: [192.168.1.74]
TASK [apache : install vhost file] *********************************************************************
ok: [192.168.1.74]
TASK [apache : install index.html] *********************************************************************
ok: [192.168.1.74]
TASK [apache : install index.php] **********************************************************************
ok: [192.168.1.74]
TASK [apache : start httpd] ****************************************************************************
ok: [192.168.1.74]
TASK [mysql : install mysql] ***************************************************************************
ok: [192.168.1.74]
TASK [mysql : install config file] *********************************************************************
ok: [192.168.1.74]
TASK [mysql : start mysql] *****************************************************************************
changed: [192.168.1.74]
PLAY RECAP *********************************************************************************************
192.168.1.74 : ok=9 changed=1 unreachable=0 failed=0
目标主机httpd2此时的apache和mysql安装配置完毕,并且已经启动
3、定义两台nginx服务器的playbook并下发
编辑创建HAnginx.yaml
[root@ansible-70 ~]# vim /etc/ansible/playbooks/HAnginx.yaml
- hosts: nginx #面向nginx主机组
remote_user: root
roles:
- nginx #执行nginx模块
- keepalived #执行keepalived模块
语法检查:
[root@ansible-70 ~]# ansible-playbook --syntax-check /etc/ansible/playbooks/HAnginx.yaml
playbook: /etc/ansible/playbooks/HAnginx.yaml
下发HAnginx.yaml
[root@ansible-70 ~]# ansible-playbook /etc/ansible/playbooks/HAnginx.yaml
PLAY [nginx] *******************************************************************************************
TASK [Gathering Facts] *********************************************************************************
ok: [192.168.1.71]
ok: [192.168.1.72]
TASK [nginx : install epel] ****************************************************************************
changed: [192.168.1.71]
changed: [192.168.1.72]
TASK [nginx : install nginx] ***************************************************************************
ok: [192.168.1.71]
ok: [192.168.1.72]
TASK [nginx : install config file] *********************************************************************
changed: [192.168.1.72]
changed: [192.168.1.71]
TASK [nginx : start nginx] *****************************************************************************
changed: [192.168.1.72]
changed: [192.168.1.71]
TASK [keepalived : install keepalived] *****************************************************************
changed: [192.168.1.72]
changed: [192.168.1.71]
TASK [keepalived : install config file] ****************************************************************
changed: [192.168.1.71]
changed: [192.168.1.72]
TASK [keepalived : start keepalived] *******************************************************************
changed: [192.168.1.71]
changed: [192.168.1.72]
PLAY RECAP *********************************************************************************************
192.168.1.71 : ok=8 changed=6 unreachable=0 failed=0
192.168.1.72 : ok=8 changed=6 unreachable=0 failed=0
目标主机keepalive1和keepalived2此时的keepalived和nginx安装配置完毕,并且已经启动
4、客户机测试:
[root@clien-69 ~]# for i in {1..10} ; do curl http://192.168.1.99/ ; done #负载均衡测试
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
把其中一个nginx主机的keepalived服务停用,观察主备是否切换:
当keepalived1停止服务时候,keepalived2从机变成服务主机
客户机测试:
[root@clien-69 ~]# for i in {1..10} ; do curl http://192.168.1.99/ ; done #负载均衡测试
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
<h1>This is httpd1-73</h1>
<h1>This is httpd2-74</h1>
keepalived主备切换成功,服务高可用,页面访问正常。