spark on yarn的一次爬坑

1. 问题如何出现

将代码打成jar包上传到远程主机，将jar包提交到yarn上

运行yarn-submit命令

export HADOOP_CONF_DIR=/home/hadoop/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop
$SPARK_HOME/bin/spark-submit \
--master yarn \
--class com.henry.com.SparkCore.LogServerApp \
--name LogServerApp \
/home/hadoop/test/g5SparkLearning-1.0-SNAPSHOT.jar \
hdfs://hadoop:9000/logs/input/data.txt hdfs://hadoop:9000/logs/output15

发生如下错误

org.apache.spark.SparkException: Application application_number1 failed 2 times due to AM Container for appattempt_number exited with  exitCode: 1
For more detailed output, check application tracking page:http://hadoop:8088/proxy/application_number/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_number
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
    at org.apache.hadoop.util.Shell.run(Shell.java:478)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

2. 如何检测到问题原因

首先进入到web UI界面

spark-on-yarn错误.png

点击application_1550206647497_0001

spark-on-yarn_1.png

点击logs

spark-on-yarn错误_2.png

点击stderr : Total file length is 3196 bytes.

此时发现了真正问题的原因

spark-on-yarn错误_3.png

从搜索引擎查询java.lang.UnsupportedClassVersionError原因
发现是JDK不匹配
Hadoop是2.6.0-cdh5.7.0版本使用的是jdk7
而spark使用jdk8

3. 解决方法

在hadoop的hadoop-env.sh中指定jdk