如何启动HDFS、Hive

  1. 下载Hadoop & 解压
  2. 添加环境变量HADOOP_HOME
  3. 安装JDK,配置JAVA_HOME环境变量
  4. 如果是单机环境需要修改 core-site.xml
  5. 配置 name node、 secondary name node、data node、node manager 、yarn node manager、 yarn resource manager的运行账户
    hadoop-env.sh
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export HDFS_RESOURCEMANAGER_USER=root
export HDFS_NODEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root
export YARN_RESOURCEMANAGER_USER=root

启动HIVE
配置hive-site.xml

<configuration>
    <property>
            <name>javax.jdo.option.ConnectionURL</name>
            <value>jdbc:mysql://7.225.95.61:3306/hivedb?createDatabaseIfNotExist=true</value>
            <description>
                JDBC connect string for a JDBC metastore.
                To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
                For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
            </description>
    </property>
    <property>
            <name>javax.jdo.option.ConnectionDriverName</name>
            <value>com.mysql.cj.jdbc.Driver</value>
            <description>Driver class name for a JDBC metastore</description>
    </property>
 
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>zwshao1</value>
    <description>Username to use against metastore database</description>
  </property>
 
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>zwshao</value>
    <description>password to use against metastore database</description>
  </property>
 
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>
  <property>
    <name>hive.metastore.urls</name>
    <value>thrift://0.0.0.0:9083</value>
</property>
<property>
    <name>hive.metastore.client.socket.timeout</name>
    <value>3600</value>
</property>
</configuration>

copy驱动到 $HIVE_HOME/lib目录下

初始化hive meta

schematool -initSchema -dbType mysql

启动hive metastore

nohup hive --service metastore 2>&1 >> /var/log.log &

启动hive server2

nohup hive --service hiveserver2 2>&1 >> /var/log.log &

启动spark thrift server

sbin/start-thriftserver.sh --master yarn
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容