hadoop win10环境搭建之踩坑

搭建方式可参考：
https://www.jianshu.com/p/faf038923093

参上以上简书内容搭建即可，但是存在如下四个问题需要补充：

1. 需要bin目录替换，至 https://github.com/steveloughran/winutils下载对于版本的bin，并添加到hadoop中即可。
2. nodemanger无法启动
3. 启动nodemanager存在路径无法打开的问题
4. wordcount无法运行

解决方案：
针对2,3,4三个问题，首先yarn-site.xml的内容需要修改为

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<configuration>

    <!-- Site specific YARN configuration properties -->

    <property>      

        <name>yarn.nodemanager.aux-services</name>      

        <value>mapreduce_shuffle</value>    

    </property>    

    <property>      

        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>              
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>    

    </property>

    <property>

        <!-- NodeManager总的可用虚拟CPU个数 -->

        <name>yarn.nodemanager.resource.cpu-vcores</name>

        <value>1</value>

    </property>

    <property>

        <!-- 每个节点可用的最大内存 -->

        <name>yarn.nodemanager.resource.memory-mb</name>

        <value>4096</value>

    </property>

    <property>

        <name>yarn.nodemanager.log-dirs</name>

        <value>/C:/hadoop-3.2.2/logs/yarn</value>

    </property>

</configuration>

2. hadoop启动时候需要使用管理员方式打开cmd启动start-all
3. 每个节点的可用最大内存需要注意分配的大一些，此处分配为4096，之前分配为2048，若分配的小了，会存在map 0 reduce 0 情况