hive 版本:version 2.1.1
conf/hive-site.xml
配置
hadoop
etc/hadoop/hdfs-site.xml 添加
<!-- web访问hdfs -->
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
etc/hadoop/core-site.xml 添加 此处注意这里是你的用户名
<!-- 允许代理用户 -->
<property>
<name>hadoop.proxyuser.root.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.root.groups</name>
<value>*</value>
</property>
配置hive
hive.server2.enable.doAs设置成false, 不然后面会连不上。
conf/hive-site.xml 修改
<!-- python连接hive -->
<property>
<name>hive.server2.authentication</name>
<value>NOSASL</value>
</property>
<!-- hiveserver2 默认端口 -->
<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
</property>
<property>
<name>hive.server2.enable.doAs</name>
<value>false</value>
</property>
测试
重启[root@host hadoop/sbin]# cd hadoop/sbin && ./stop-all.sh ./start-all.sh
启动[root@host local]# hiveserver2
新开连接窗口(hiveserver2的启动窗口不要关闭) 新的窗口 > jps
查看端口是否起来。
[root@host local]# ss -ntlp|grep 1000
LISTEN 0 50 *:10000 *:* users:(("java",pid=21551,fd=461))
LISTEN 0 50 *:10002 *:* users:(("java",pid=21551,fd=462))
本地连接到hiveserver2查看是否可以连接
[root@host local]# beeline
此处指定了sasl所以要加上auth这个参数否则连接并发限制; user pass,可以随便写
beeline> !connect jdbc:hive2://127.0.0.1:10000/default;auth=noSasl user pass
Connecting to jdbc:hive2://127.0.0.1:10000/default;auth=noSasl
Connected to: Apache Hive (version 2.1.1)
Driver: Hive JDBC (version 2.1.1)
21/03/05 10:17:34 [main]: WARN jdbc.HiveConnection: Request to set autoCommit to false; Hive does not support autoCommit=false.
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://127.0.0.1:10000/default> show databases;
+----------------+--+
| database_name |
+----------------+--+
| default |
+----------------+--+
1 row selected (0.203 seconds)
python连接hive
安装依赖:
yum install cyrus-sasl-devel
pip2 install pyhive[hive]
连接
# -*- coding: utf-8 -*-
from pyhive import hive
conn = hive.Connection(host='172.17.1.151', port=10000, username='hive_test', database='default',auth='NOSASL')
cursor = conn.cursor()
def get_data():
query_sql = "select * from test"
cursor.execute(query_sql)
for result in cursor.fetchall():
print(result)
def main():
get_data()
if __name__ == "__main__":
main()
参考链接:https://blog.csdn.net/weixin_42662249/article/details/104819034