hadoop2.7.7 +hbase2.0.6 + apache-phoenix-5.0.0-HBase-2.0 安装使用
1、安装环境准备
1.1）、准备机器说明
| 主机名(机器) | ip | 说明 |
|---|---|---|
| master | 192.168.2.10 | 主节点（安装hadoop、hbase、phoenix、zookeeper） |
| node1 | 192.168.2.20 | 节点1（安装hadoop节点） |
| node2 | 192.168.2.30 | 节点2（安装hadoop节点） |
三台机器均关闭防火墙并关闭ipv6
1.2）、三台机器均匀配置host
192.168.2.10 master
192.168.2.20 node1
192.168.2.30 node2
1.3）、master机器设置免登录node1、node2（在master上执行以下命令）
ssh-keygen -t rsa (一路默认回车即可)
ssh-copy-id hadoop-master
ssh-copy-id hadoop-node1
ssh-copy-id hadoop-node2
1.4）、master机器增加以下环境变量配置/etc/profile（第二步开始逐一安装）
export JAVA_HOME=/software/jdk1.8.0_151
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_HOME=/software/hadoop-2.7.7
export PATH=${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH
export HBASE_HOME=/software/hbase-2.0.6
export PATH=$HBASE_HOME/bin:$PATH
export PHOENIX_HOME=/software/apache-phoenix-5.0.0-HBase-2.0-bin
export PHOENIX_CLASSPATH=$PHOENIX_HOME
export PATH=$PATH:$PHOENIX_HOME/bin
在master机器上安装jdk、hadoop、hbase、phoenix，安装包的解压目录均为
/software
1.5）、node1、node2机器增加以下环境变量配置/etc/profile
export JAVA_HOME=/software/jdk1.8.0_151
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_HOME=/software/hadoop-2.7.7
export PATH=${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH
在两个node节点上安装jdk、hadoop，安装包的解压目录均为
/software
2、安装hadoop2.7.7
解压安装包
hadoop-2.7.7.tar.gz到目录/software，与高版本的不同，此版本需修改/software/hadoop-2.7.7/etc/hadoop目录下的配置文件core-site.xml、hdfs-site.xml、hadoop-env.sh、slaves、yarn-env.sh共五个文件并同步到node1、node2一，两个节点
首先将安装包同步到节点上并解压
scp jdk-8u151-linux-x64.tar.gz hadoop-2.7.7.tar.gz root@node1:/software
scp jdk-8u151-linux-x64.tar.gz hadoop-2.7.7.tar.gz root@node2:/software
2.1）、core-site.xml配置文件新增以下配置
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:8020</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/software/hadoop-2.7.7/tmp</value>
</property>
</configuration>
2.2）、hadoop-env.sh配置文件新增jdk路径配置
export JAVA_HOME=/software/jdk1.8.0_151
2.3）、hdfs-site.xml配置文件新增以下配置
<configuration>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>node1:50090</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/var/data/hadoop/hdfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/var/data/hadoop/hdfs/data</value>
</property>
<property>
<name>dfs.namenode.http-address</name>
<value>master:8084</value>
</property>
</configuration>
2.4）、slaves配置文件新增节点配置
master
node1
node2
2.5）、yarn-env.sh配置文件新增jdk路径配置
export JAVA_HOME=/software/jdk1.8.0_151
2.6）、同步配置文件到node1、node2
scp -r core-site.xml hadoop-env.sh hdfs-site.xml slaves yarn-env.sh root@node1:/software/hadoop-2.7.7/etc/hadoop
scp -r core-site.xml hadoop-env.sh hdfs-site.xml slaves yarn-env.sh root@node2:/software/hadoop-2.7.7/etc/hadoop
2.7）、在master上执行格式化命令完成格式化
hdfs namenode -format
2.8）、启动hadoop集群
启动
start-all.sh
jps可查看相关进程情况
停止
stop-all.sh
重启nodemanager
yarn --daemon start nodemanager
重启datanode
hdfs --daemon start datanode
master上查看节点状态
yarn node -list
master查看hdfs状态
hdfs dfsadmin -report
查看hdfs占用空间
hadoop fs -du -h /
启动完成后，可访问地址
http://192.168.2.10:8084/dfshealth.html#tab-overview查看详情
3、master安装zookeeper
zookeeper简单安装单机版的作为演示，解压安装包apache-zookeeper-3.6.1-bin.tar.gz到目录/software，拷贝配置文件启动即可
[root@master conf]# cd /software/apache-zookeeper-3.6.1-bin/conf
[root@master conf]# cp zoo_sample.cfg zoo.cfg
[root@master conf]# cd ../bin/
[root@master bin]# ./zkServer.sh start
4、master安装hbase2.0.6
下载安装包
hbase-2.0.6-bin.tar.gz解压到目录/software，并配置/software/hbase-2.0.6/conf中的配置文件hbase-env.sh、hbase-site.xml
4.1）、hbase-env.sh配置jdk环境变量并配置zk不使用内部zk（使用外部zookeeper）
export JAVA_HOME=/software/jdk1.8.0_151
export HBASE_MANAGES_ZK=false
4.2）、hbase-site.xml配置文件新增以下配置
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>./tmp</value>
</property>
<property>
<name>hbase.unsafe.stream.capability.enforce</name>
<value>false</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://master:8020/hbase</value>
</property>
<property>
<name>hbase.master.maxclockskew</name>
<value>150000</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/data/zookeeper</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master:2181</value>
</property>
</configuration>
4.3）、启动hbase
启动
start-hbase.sh
jps可查看hbase相关进程
[root@master bin]# jps
3600 HMaster
2258 QuorumPeerMain
3724 HRegionServer
关闭hbase
stop-hbase.sh
若stop-hbase.sh关闭不了直接使用以下两句
hbase-daemon.sh stop master
hbase-daemon.sh stop regionserver
5、master安装phoenix5.0.0
解压安装包
apache-phoenix-5.0.0-HBase-2.0-bin.tar.gz到目录/software，环境变量第一步已配置，将两个jar包拷贝到hbase的lib目录中，命令如下
cd /software/apache-phoenix-5.0.0-HBase-2.0-bin
cp phoenix-5.0.0-HBase-2.0-server.jar phoenix-core-5.0.0-HBase-2.0.jar /software/hbase-2.0.6/lib/
拷贝
jar包到hbase后需重启hbase
5.1）、修改phoenix执行脚本权限
cd /software/apache-phoenix-5.0.0-HBase-2.0-bin/bin
chmod 777 psql.py sqlline.py
5.2）、master上使用phoenix操作hbase
[root@master ~]# sqlline.py master:2181
Setting property: [incremental, false]
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:master:2181 none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:master:2181
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/software/apache-phoenix-5.0.0-HBase-2.0-bin/phoenix-5.0.0-HBase-2.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/software/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
20/10/18 22:18:54 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connected to: Phoenix (version 5.0)
Driver: PhoenixEmbeddedDriver (version 5.0)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
Building list of tables and columns for tab-completion (set fastconnect to true to skip)...
133/133 (100%) Done
Done
sqlline version 1.2.0
0: jdbc:phoenix:master:2181>
0: jdbc:phoenix:master:2181> !tables
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+-+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | TABLE_TYPE | REMARKS | TYPE_NAME | SELF_REFERENCING_COL_NAME | REF_GENERATION | |
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+-+
| | SYSTEM | CATALOG | SYSTEM TABLE | | | | | |
| | SYSTEM | FUNCTION | SYSTEM TABLE | | | | | |
| | SYSTEM | LOG | SYSTEM TABLE | | | | | |
| | SYSTEM | SEQUENCE | SYSTEM TABLE | | | | | |
| | SYSTEM | STATS | SYSTEM TABLE | | | | | |
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+-+
0: jdbc:phoenix:master:2181> create table test (mykey integer not null primary key, mycolumn varchar);
No rows affected (0.871 seconds)
0: jdbc:phoenix:master:2181> select * from test;
+--------+-----------+
| MYKEY | MYCOLUMN |
+--------+-----------+
+--------+-----------+
No rows selected (0.081 seconds)
0: jdbc:phoenix:master:2181> upsert into test values (1,'Hello');
1 row affected (0.045 seconds)
0: jdbc:phoenix:master:2181> upsert into test values (2,'World!');
1 row affected (0.009 seconds)
0: jdbc:phoenix:master:2181> select * from test;
+--------+-----------+
| MYKEY | MYCOLUMN |
+--------+-----------+
| 1 | Hello |
| 2 | World! |
+--------+-----------+
2 rows selected (0.025 seconds)
0: jdbc:phoenix:master:2181> !exit
Closing: org.apache.phoenix.jdbc.PhoenixConnection
[root@master ~]#
