安装版本:5.16.1
服务器列表:
编号 | 服务器名称 | 内网IP | 内存 | 规划 |
---|---|---|---|---|
1 | cdhserver1 | 192.168.26.21 | 16G | cloudera-scm-server、cloudera-scm-agent |
2 | cdhserver2 | 192.168.26.22 | 10G | cloudera-scm-agent |
3 | cdhserver3 | 192.168.26.23 | 10G | cloudera-scm-agent |
1.1 所需安装包下载并上传到所有节点
安装所有依赖:
yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb mod_ssl unzip
所有节点:
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz
主节点:
http://archive.cloudera.com/cdh5/parcels/5.16.1/manifest.json
http://archive.cloudera.com/cdh5/parcels/5.16.1/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.16.1/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1
1.2 在所有节点创建CM目录
mkdir -p /opt/cloudera-manager
1.3 解压安装文件
tar -zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager/
1.4 在所有节点创建cloudera-scm用户
Cloudera管理器服务器和托管服务被配置为在默认情况下使用用户帐户Cloudera-scm,创建具有这个名称的用户是最简单的方法。创建用户,在安装完成后自动使用。
useradd --system --home=/opt/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
在所有节点为/opt/cloudera-manager目录赋权
chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager
1.5 配置cloudera-scm-agent
vi /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini
修改server_host为主节点的主机名或IP
# Hostname of the CM server.
server_host=cdhserver1
将修改好的文件同步分发到所有集群节点。
1.6 初始化数据库
1.6.1 添加MySQL驱动
http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.43/mysql-connector-java-5.1.43.jar
将mysql-connector-Java.jar复制到/usr/share/java(无则创建)目录下(所有节点)
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.43/mysql-connector-java-5.1.43.jar mkdir /usr/share/java
cp mysql-connector-java-5.1.43.jar /usr/share/java/mysql-connector-java.jar
将mysql-connector-java.jar复制到/var/lib/ambari-server/resources目录下(主节点)
cp mysql-connector-java-5.1.43.jar /opt/cloudera-manager/cm-5.16.1/share/cmf/lib/
service mysqld restart
1.6.2 在主节点初始化数据库
cd /opt/cloudera-manager/cm-5.16.1/share/cmf/schema/
./scm_prepare_database.sh mysql -hcdhserver1 --port 6033 -uroot -pRoot@2019 --scm-host % scm scm scm#CDH5
参数说明:
参数 | 说明 |
---|---|
postgresq|mysql|oracle | 必选,数据库类型 |
-h | 数据库主机ip或hostname,默认是"localhost" |
-u | 数据库账号,需要具备增删改查的权限,默认是"root" |
-p | 账号密码,默认无密码 |
--port | 数据库端口号,默认3306 |
--scm-host | SCM server主机名,默认是"localhost"。填写主机名是发现报如下异常:Access denied for user 'scm'@'cdhserver1' (using password: YES) ,使用 % 创建成功。 |
最后三个参数 | 分别为:数据库名(必选),SCM库账号(必选),SCM库密码(可选, 不指定会提示输入) |
成功信息:
Cloudera服务需要的相关database:
服务名称 | 自定义数据库 | 自定义数据库用户名 |
---|---|---|
Cloudera Manager Server | scm | scm |
Activity Monitor | amon | amon |
Reports Manager | rman | rman |
Hue | hue | hue |
Hive Metastore Server | hive | hive |
Sentry Server | sentry | sentry |
Cloudera Navigator Audit Server | nav | nav |
Cloudera Navigator Metadata Server | navms | navms |
Oozie | oozie | oozie |
./scm_prepare_database.sh mysql -hcdhserver1 --port 6033 -uroot -pRoot@2019 --scm-host % hue hue hue#CDH5
./scm_prepare_database.sh mysql -hcdhserver1 --port 6033 -uroot -pRoot@2019 --scm-host % hive hive hive#CDH5
./scm_prepare_database.sh mysql -hcdhserver1 --port 6033 -uroot -pRoot@2019 --scm-host % oozie oozie oozie#CDH5
......
1.7 设置开机启动系统服务
1.7.1 设置server服务自启动(主节点)
cp /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-server /etc/rc.d/init.d/
chown cloudera-scm:cloudera-scm /etc/rc.d/init.d/cloudera-scm-server
修改"CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}"的路径
vi /etc/rc.d/init.d/cloudera-scm-server
CMF_DEFAULTS=/opt/cloudera-manager/cm-5.16.1/etc/default
添加系统启动服务
chkconfig --add cloudera-scm-server
chkconfig --level 35 cloudera-scm-server on
chkconfig --list
1.7.2 设置agent服务自启动(所有节点)
cp /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent /etc/rc.d/init.d/
chown cloudera-scm:cloudera-scm /etc/rc.d/init.d/cloudera-scm-agent
修改"CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}"的路径"-/etc/default"
vi /etc/rc.d/init.d/cloudera-scm-agent
CMF_DEFAULTS=/opt/cloudera-manager/cm-5.16.1/etc/default
chkconfig --add cloudera-scm-agent
chkconfig --level 35 cloudera-scm-agent on
chkconfig --list
1.8 创建本地parcel源(主节点)
创建本地parcel源目录,并将下载的三个parcel文件放到该目录下
mkdir -p /opt/cloudera/parcel-repo
将"CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1"重命名为"CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha",否则会重新下载"CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel"安装包
mv /opt/cloudera/parcel-repo/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha
所有节点创建软件安装目录并授权
mkdir -p /opt/cloudera/parcels
chown -R cloudera-scm:cloudera-scm /opt/cloudera/
1.9 启动CM服务
1.9.1 在主节点启动server服务
service cloudera-scm-server start
service cloudera-scm-server status -l
1.9.2 所有节点启动agent服务
service cloudera-scm-agent start
service cloudera-scm-agent status -l
http://cdhserver1:7180 默认账号/密码:admin/admin