在上文中我们得知,Hive的入口脚本都在bin/etc目录下,启动hive客户端的过程就是通过这些脚本启动的。细心一点的话,我们在hive脚本中看到了服务列表(SERVICE_LIST),这些脚本正是Hive Server清单。
Hive 服务清单(Hive Server List)
选项 | 名称 | 描述 |
---|---|---|
cli | 命令行界面 | hive 的命令行交互界面 |
beeline | beeline连接 | beeline连接服务 |
hiveserve | Hive Serve | 监听来自其他进程的thrift连接的守护进程 |
hiveserve2 | Hive Serve 2 | 另一个版本的Hive Server |
hwi | Hive Web Interface | Hive的Web界面服务 |
jar | hadoop jar命令的包装,提供Hive环境下的hadoop jar支持 | |
lineage | 血缘关系 | 提供hive血缘关系分析 |
metastore | 元数据存储 | 元数据存储服务 |
metatool | 元数据工具 | 提供元数据操作的一些服务 |
orcfiledump | - | - |
rcfilecat | - | - |
schemaTool | - | - |
version | - | - |
beeline.sh
我们可以看出,脚本里主要定义了beeline函数。
- beeline主类。
org.apache.hive.beeline.Beeline
- beeline启动jar。
hive-beeline-*.jar
THISSERVICE=beeline
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
beeline () {
CLASS=org.apache.hive.beeline.BeeLine;
# include only the beeline client jar and its dependencies
beelineJarPath=`ls ${HIVE_LIB}/hive-beeline-*.jar`
superCsvJarPath=`ls ${HIVE_LIB}/super-csv-*.jar`
jlineJarPath=`ls ${HIVE_LIB}/jline-*.jar`
jdbcStandaloneJarPath=`ls ${HIVE_LIB}/../jdbc/hive-jdbc-*-standalone.jar`
hadoopClasspath=""
if [[ -n "${HADOOP_CLASSPATH}" ]]
then
hadoopClasspath="${HADOOP_CLASSPATH}:"
fi
export HADOOP_CLASSPATH="${hadoopClasspath}${HIVE_CONF_DIR}:${beelineJarPath}:${superCsvJarPath}:${jlineJarPath}:${jdbcStandaloneJarPath}"
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dlog4j.configuration=beeline-log4j.properties "
exec $HADOOP jar ${beelineJarPath} $CLASS $HIVE_OPTS "$@"
}
beeline_help () {
beeline "--help"
}
cli.sh
Command line interface。命令行界面。cli.sh脚本主要定义了cli函数。
- cli主类。
org.apache.hadoop.hive.cli.CliDriver
THISSERVICE=cli
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
cli () {
CLASS=org.apache.hadoop.hive.cli.CliDriver
execHiveCmd $CLASS "$@"
}
cli_help () {
CLASS=org.apache.hadoop.hive.cli.CliDriver
execHiveCmd $CLASS "--help"
}
hiveburninclient.sh
hiveburninclient.sh定义了hiveburninclient函数。
- hiveburninclient主类。
org.apache.hive.testutils.jdbc.HiveBurnInClient
THISSERVICE=hiveburninclient
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
hiveburninclient() {
echo "Starting hiveburninclient"
CLASS=org.apache.hive.testutils.jdbc.HiveBurnInClient
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
JAR=${HIVE_LIB}/hive-service-*.jar
exec $HADOOP jar $JAR $CLASS $HIVE_OPTS "$@"
}
hiveburninclient_help() {
hiveburninclient -H
}
hiveserver.sh
Hive Server。hiveserver.sh定义了hiveserver函数。
- hiveserver主类。
org.apache.hadoop.hive.service.HiveServer
THISSERVICE=hiveserver
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
hiveserver() {
echo "Starting Hive Thrift Server"
CLASS=org.apache.hadoop.hive.service.HiveServer
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
JAR=${HIVE_LIB}/hive-service-*.jar
# hadoop 20 or newer - skip the aux_jars option and hiveconf
exec $HADOOP jar $JAR $CLASS $HIVE_OPTS "$@"
}
hiveserver_help() {
hiveserver -h
}
hiveserver2.sh
Hive Server2。hiveserver2.sh中定义了hiveserver函数。
- hiveserver2主类。
org.apache.hive.service.server.HiveServer2
THISSERVICE=hiveserver2
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
hiveserver2() {
CLASS=org.apache.hive.service.server.HiveServer2
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
JAR=${HIVE_LIB}/hive-service-*.jar
exec $HADOOP jar $JAR $CLASS $HIVE_OPTS "$@"
}
hiveserver2_help() {
hiveserver2 -H
}
hwi.sh
Hive Web界面。hwi.sh定义了hwi函数。
- hwi主类。
org.apache.hadoop.hive.hwi.HWIServer
THISSERVICE=hwi
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
hwi() {
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
CLASS=org.apache.hadoop.hive.hwi.HWIServer
# The ls hack forces the * to be expanded which is required because
# System.getenv doesn't do globbing
export HWI_JAR_FILE=$(ls ${HIVE_LIB}/hive-hwi-*.jar)
export HWI_WAR_FILE=$(ls ${HIVE_LIB}/hive-hwi-*.war)
#hwi requires ant jars
if [ "$ANT_LIB" = "" ] ; then
ANT_LIB=/opt/ant/lib
fi
for f in ${ANT_LIB}/*.jar; do
if [[ ! -f $f ]]; then
continue;
fi
HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:$f
done
export HADOOP_CLASSPATH
# hadoop 20 or newer - skip the aux_jars option and hiveconf
exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $HIVE_OPTS "$@"
}
hwi_help(){
echo "Usage ANT_LIB=XXXX hive --service hwi"
}
jar.sh
这是一个hadoop jar命令的包装,用于支持Hive环境的hadoop jar命令。
THISSERVICE=jar
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
jar () {
RUNJAR=$1
shift
RUNCLASS=$1
shift
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
if [ -z "$RUNJAR" ] ; then
echo "RUNJAR not specified"
exit 3
fi
if [ -z "$RUNCLASS" ] ; then
echo "RUNCLASS not specified"
exit 3
fi
# hadoop 20 or newer - skip the aux_jars option and hiveconf
exec $HADOOP jar $RUNJAR $RUNCLASS $HIVE_OPTS "$@"
}
jar_help () {
echo "Used for applications that require Hadoop and Hive classpath and environment."
echo "./hive --service jar <yourjar> <yourclass> HIVE_OPTS <your_args>"
}
lineage.sh
lineage.sh脚本是用来做hive表的血缘分析。在脚本里定义了lineage函数。
- lineage主类。
org.apache.hadoop.hive.ql.tools.LineageInfo
THISSERVICE=lineage
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
lineage () {
CLASS=org.apache.hadoop.hive.ql.tools.LineageInfo
# cli specific code
if [ ! -f ${HIVE_LIB}/hive-exec-*.jar ]; then
echo "Missing Hive exec Jar"
exit 3;
fi
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
exec $HADOOP jar ${HIVE_LIB}/hive-exec-*.jar $CLASS "$@"
}
lineage_help () {
echo "usage ./hive 'hql' "
}
metastore.sh
元数据存储服务。metastore.sh中定义了metastore函数。
- metastore主类。
org.apache.hadoop.hive.metastore.HiveMetaStore
THISSERVICE=metastore
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
metastore() {
echo "Starting Hive Metastore Server"
CLASS=org.apache.hadoop.hive.metastore.HiveMetaStore
if $cygwin; then
HIVE_LIB=`cygpath -w "$HIVE_LIB"`
fi
JAR=${HIVE_LIB}/hive-service-*.jar
# hadoop 20 or newer - skip the aux_jars option and hiveconf
export HADOOP_OPTS="$HIVE_METASTORE_HADOOP_OPTS $HADOOP_OPTS"
exec $HADOOP jar $JAR $CLASS "$@"
}
metastore_help() {
metastore -h
}
metatool.sh
元数据工具。
主类:org.apache.hadoop.hive.metastore.tools.HiveMetaTool
THISSERVICE=metatool
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
metatool () {
HIVE_OPTS=''
CLASS=org.apache.hadoop.hive.metastore.tools.HiveMetaTool
execHiveCmd $CLASS "$@"
}
metatool_help () {
HIVE_OPTS=''
CLASS=org.apache.hadoop.hive.metastore.tools.HiveMetaTool
execHiveCmd $CLASS "--help"
}
orcfiledump.sh
后续再补充。
rcfilecat.sh
后续再补充。
schemaTool.sh
后续再补充。
version.sh
版本信息。
主类:org.apache.hadoop.hive.common.util.HiveVersionInfo
THISSERVICE=version
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
version () {
JAR=$1
if [ -z "$JAR" ] ; then
JAR=${HIVE_LIB}/hive-exec-*.jar
else
JAR=${HIVE_LIB}/$1
fi
# hadoop 20 or newer - skip the aux_jars option and hiveconf
CLASS=org.apache.hive.common.util.HiveVersionInfo
exec $HADOOP jar $JAR $CLASS
}
version_help () {
echo "Show Version information of hive jars"
echo "./hive --version [hiveJar]"
}
至此,我们大概对hive的服务有了一个初步的概念。