Hive环境搭建

Hive的安装需要提前安装Hadoop和MySQL，Hadoop的安装请参考Hadoop环境搭建，下面是MySQL的安装。

MySQL安装

1. MySQL安装请点击

2. 启动MySQL
   service mysqld restart

3. 创建hive用户
   create user 'hive' identified by 'hive';

4. 创建hive数据库
   create database hive;

5. 将hive数据库权限设置给hive用户
   grant all privileges on . to hive@'localhost' identified by 'hive';

6. 刷新
   flush privileges;

Hive的安装

软件安装

1. 软件下载官网地址

2. 解压安装
   tar -zxvf apache-hive-2.1.1-bin.tar.gz ../software

环境配置

• 新建hive-site.xml(hive的conf的目录下)
  cp hive-default.xml.template hive-site.xml

• 修改hive-site.xml
  ```
  在Habase的目录下新建temp目录
  替换hive-site.xml中的${system:java.io.tmpdir}为/usr/local/bigdata/software/hive-2.1.1/temp

    <property>
            <name>javax.jdo.option.ConnectionURL</name>
            <value>jdbc:mysql://master:3306/hive?characterEncoding=UTF-8</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionDriverName</name>
            <value>com.mysql.jdbc.Driver</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionUserName</name>
            <value>hive</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionPassword</name>
            <value>hive</value>
        </property>
        <property>
            <name>datanucleus.autoCreateSchema</name>
            <value>true</value>
        </property>
        <property>
            <name>datanucleus.autoCreateTables</name>
            <value>true</value>
        </property>
        <property>
            <name>datanucleus.autoCreateColumns</name>
            <value>true</value>
        </property>
    ```
  

• 下载MySQL连接jar
  下载mysql-connector-java-5.1.41-bin.jar到hive-2.1.1/lib下

• 配置环境变量 /etc/profile

启动Hive

1. 初始化数据库

schematool -initSchema -dbType mysql

2. 启动hiveserver2

[root@hadoop1 bin]# hiveserver2

Hive整合Hadoop

1. 创建本地hive_test文件（字段使用tab分割）

   zhangshan 20
   lisi 21
   wangwu 19

2. hdfs创建hive_test文件夹

   hdfs dfs -mkdir /hive_test

3. 将本地文件导入到hdfs中

   hdfs dfs -copyFromLocal hive_test /hive_test

4. 创建Hive表

   create table hive_test (
   name string,
   age int
   )row format delimited fields terminated by '\t' stored as textfile;

5. hive关联Hadoop

   load data inpath /hive_test into table hive_test;

6. 查询测试

   zhangshan 20
   lisi 21
   wangwu 19

Hive和HBase互通

配置

修改hive-site.xml

   <property>
      <name>hbase.zookeeper.quorum</name>
      <value>master</value>
      <description>zookeeper</description>
    </property>

    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
        <description>port</description>
    </property>

数据交互

1. HBase新建表

create 'student','baseinfo','otherinfo'

2. 插入数据

put 'student','1','baseinfo:name','zhangsan'
put 'student','1','baseinfo:age','21'
put 'student','1','otherinfo:hobby','sleep'
put 'student','2','baseinfo:name','lisi'

3. 查看数据

scan 'student'

4. Hive中新建基于HBase的表

CREATE EXTERNAL TABLE student (
rowkey string,
name string,
base map<STRING,STRING>,
other map<STRING,STRING>
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,baseinfo:name,baseinfo:,otherinfo:")
TBLPROPERTIES ("hbase.table.name" = "student");

5. 查询HBase表

select * from student;

配置Hive高可用

基于zookeeper

配置hive-site.xml，新增如下配置

<property>
    <name>hive.server2.support.dynamic.service.discovery</name>
    <value>true</value>
</property>
<property>
    <name>hive.server2.zookeeper.namespace</name>
    <value>hiveserver2_zk</value>
</property>
<property>
    <name>hive.zookeeper.quorum</name>
    <value>master,slave01,slave02</value>
</property>

基于HAProxy

大家请参考链接基于HAProxy配置Hive HA

验证配置

• 在ZooKeeper下查看是否注册成功
  ls /hiveserver2_zk

• 使用Hive JDBC来测试
  启动master和slave01的hiveserver2

  执行beeline命令
  beeline

  连接数据库
  !connect jdbc:hive2://master:2181,slave01:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2_zk

  //说明连接到master
  [main]: INFO jdbc.HiveConnection: Connected to master:10000
  Connected to: Apache Hive (version 2.1.1)

  kill掉master的 hiveserver2我们继续连接数据库，到时候会连接到slave01