CDH5.7.0安装实践
规划:
ip1 master
ip2 slave
ip3 slave
ip4 slave (add node && yum repo)--ip4作为后备的添加结点和yum源
一.基础配置(每台机器都要操作)
1.更改hostname
vi /etc/sysconfig/network 然后service network restart使之生效
2.更改ip与hostname的对应关系
vi /etc/hosts 使用uname -a查看是否生效
3.关闭防火墙
service iptables stop 关闭防火墙
service iptables status 查看防火墙是否正在运行
chkconfig iptables off 关闭防火墙的开机自动运行
4.关闭SELinux
修改/etc/selinux/config和/etc/sysconfig/selinux的SELINUX=disabled
机器重启后生效
通过sestatus验证
5.ntp(用于时间同步服务)
5.1安装配置ntp服务
5.2crontab脚本用于时间同步
01 * * * * /bin/bash /home/adminscripts/ntpd_sync.sh
二.创建本地yum源
注:由于CDH5.7.0使用的是https协议,所以要配置https可访问的httpd
A.准备yum源文件
1.mkdir -p /var/www/html/cm5/redhat/6/x86_64
下载 http://archive.cloudera.com/cm5/repo-as-tarball/5.7.0/cm5.7.0-centos6.tar.gz
解压后 mv cm /var/www/html/cm5/redhat/6/x86_64/
2.mkdir -p /var/www/html/cdh5/parcels/5.7.0/
下载 http://archive.cloudera.com/cdh5/parcels/5.7.0/CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel
下载 http://archive.cloudera.com/cdh5/parcels/5.7.0/CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel.sha1并改名为CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel.sha
mv CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel* /var/www/html/cdh5/parcels/5.7.0/
3.mkdir -p /var/www/html/cdh5/redhat/6/x86_64/
下载 http://archive.cloudera.com/cdh5/repo-as-tarball/5.7.0/cdh5.7.0-centos6.tar.gz
解压后 mv cdh /var/www/html/cdh5/redhat/6/x86_64/
4.mkdir -p /var/www/html/redhat/cdh/
下载 http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera
然后mv RPM-GPG-KEY-cloudera /var/www/html/redhat/cdh/
B.ip4配置https访问的yum源
1.安装ssl
2.编译安装httpd
2.1编译安装httpd所需要的依赖(参考http://xtony.blog.51cto.com/3964396/836508)
2.2下载httpd-2.4.20.tar.gz并解压(参考http://cwind.blog.51cto.com/62582/1061168)
编译安装: cd httpd-2.4.20
./configure --prefix=/usr/local/apache2 --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util/ --with-pcre=/usr/local/pcre --enable-ssl --enable-so
make && make install
2.3生成证书
mkdir -p /usr/local/apache2/conf/ssl.key
cd /usr/local/apache2/conf/ssl.key
//生成服务器私钥：
openssl genrsa -des3 -out server.key 1024
//生成服务器证书请求，并按要求填些相关证书信息：
openssl req -new -key server.key -out server.csr
//签证:
openssl x509 -req -days 700 -in server.csr -signkey server.key -out server.crt
2.4修改httpd.conf文件
①打开LoadModule ssl_module modules/mod_ssl.so和LoadModule rewrite_module modules/mod_rewrite.so
②加入代码:
Listen 443
NameVirtualHost *:443
NameVirtualHost *:80
ServerName ip4
DocumentRoot /var/www/html
ErrorLog /usr/local/apache2/logs/error.log
CustomLog /usr/local/apache2/logs/access.log combined
ServerName ip4
DocumentRoot /var/www/html
ErrorLog /usr/local/apache2/logs/error.log
CustomLog /usr/local/apache2/logs/access.log combined
SSLEngine on
SSLCertificateFile /usr/local/apache2/conf/ssl.key/server.crt
SSLCertificateKeyFile /usr/local/apache2/conf/ssl.key/server.key
SSLEngine off
③修改为
Options Indexes FollowSymLinks
AllowOverride all
Require all granted
RewriteEngine on
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*) https://%{SERVER_NAME} [L,R]
2.5启动
/usr/local/apache2/bin/apachectl start
2.6访问:https://ip4 (访问http也会自动跳转到https)
2.7去掉校验
在/etc/yum.conf中添加sslverify=false
三.安装Cloudera Manager
1.cm安装使用的http服务，所以安装节点需要是可以访问的http协议yum源
2.下载
下载 http://archive-primary.cloudera.com/cm5/installer/5.3.0/cloudera-manager-installer.bin
chmod +x cloudera-manager-installer.bin
./cloudera-manager-installer.bin
3.界面一直next 和 yes操作
四.安装CDH parcel
1.打开http://ip1:7180的操作页面进行安装
2.注意如果出现:未在已配置的存储库中找到任何 parcel
在"更多选项里"将"远程parcel存储库url"更改为"https://ip4/cdh5/parcels/5.7.0/"即可找到parcel
3.修改swappiness分区
cat /proc/sys/vm/swappiness
sudo sysctl vm.swappiness=0
五.添加节点
CM管理页面操作
六.添加服务
cluster1——添加服务
