1、配置集群
(1)配置yarn-env.sh
获取jdk 的安装路径
[victor@node1 hadoop-2.8.2]$ echo $JAVA_HOME
/opt/module/jdk1.8.0_144
修改JAVA_HOME 路径
[victor@node1 hadoop-2.8.2]$ vim etc/hadoop/yarn-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_144
(2)配置yarn-site.xml
<!-- reducer获取数据的方式 -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<!-- 指定YARN的ResourceManager的地址 -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>node1</value>
</property>
<property>
<name>yarn.resourcemanager.nodes.include-path</name>
<value>/opt/module/hadoop-2.8.2/etc/hadoop/dnrm</value>
</property>
(3)配置dnrm
[victor@node1 hadoop]$ vim dnrm
node1
node2
node3
(4)配置mapred-env.sh
修改JAVA_HOME 路径
[victor@node1 hadoop-2.8.2]$ vim etc/hadoop/mapred-env.sh
export JAVA_HOME=/opt/module/jdk1.8.0_144
(5)配置mapred-site.xml
对mapred-site.xml.template重命名为mapred-site.xml
重命名mapred-site.xml文件
[victor@node1 hadoop-2.8.2]$ cd etc/hadoop/
[victor@node1 hadoop-2.8.2]$ pwd
/opt/module/hadoop-2.8.2/etc/hadoop
[victor@node1 hadoop]$ mv mapred-site.xml.template mapred-site.xml
添加配置
<!-- 指定mr运行在yarn上 -->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
2、启动集群
(1)启动resourcemanager
[victor@node1 hadoop-2.8.2]$ sbin/yarn-daemon.sh start resourcemanager
(2)启动nodemanager
[victor@node1 hadoop-2.8.2]$ sbin/yarn-daemon.sh start nodemanager
(3)集群操作
(1)yarn的浏览器页面查看
$ http://node1:8088/cluster
image.png
(2)删除文件系统上的output文件
[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs -rm -R /user/victor/wordcount/output
(3)执行mapreduce程序
[victor@node1 hadoop-2.8.2]$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.2.jar wordcount /user/victor/wordcount/input /user/victor/wordcount/output
(4)查看运行结果
[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs -cat /user/victor/wordcount/output/*
