本地环境配置
1、解压缩hadoop.*.tar.gz
2、解压缩hadoop.dll、winutils.exe等安装文件,并拷贝到hadopp解压缩文件夹的bin目录下
3、配置环境变量HADOOP_HOME,并将%HADOOP_HOME%\bin 添加到path中
4、重启IDE(eclipse)
问题汇总
winutils不存在
output文件已存在
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/d:/output already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1758)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at com.xuyang.mapreduce.WordCountDriver.main(WordCountDriver.java:37)