安装Linux CentOS系统
链接：https://pan.baidu.com/s/1eFdbQI1XFJaFBTvtIADkvg 提取码：dcia
工具：
1.WMware
2.Xshell
3.WinScp
4.虚拟机CentOS 3 台
前置知识
vi编辑
i 写入
esc 退出编辑
shift+; 唤出命令操作
wq 保存编辑
设置系统主机名
vi /etc/sysconfig/network
节点1
NETWORKING=yes
HOSTNAME=node1
节点2
NETWORKING=yes
HOSTNAME=node2
节点3
NETWORKING=yes
HOSTNAME=node3
修改网络配置
vi /etc/sysconfig/network-scripts/ifcfg-eth0
node1
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.0.110
NATMASK=255.255.255.0
GATEWAY=192.168.0.255
node2
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.0.111
NATMASK=255.255.255.0
GATEWAY=192.168.0.255
node3
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.0.112
NATMASK=255.255.255.0
GATEWAY=192.168.0.255
重启网络服务
service network restart
验证结果
ifconfig
配置映射
vi /etc/hosts
192.168.0.110 node1
192.168.0.111 node2
192.168.0.112 node3
生成钥匙
ssh-keygen
enerating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
12:94:b1:53:0b:ff:3d:35:00:2c:15:8a:30:3c:b2:09 root@node1
The key's randomart image is:
+--[ RSA 2048]----+
| .o +o.o+o. |
|E . o+.*.o. . |
| . + .= +. o |
| o o . . . . |
| . S . o |
| . . |
| |
| |
| |
+-----------------+
由上可知生成钥匙在/root/.ssh/目录下，其中id_rsa为私匙，id_rsa.pub为公匙
复制公匙
ssh-copy-id -i ./id_rsa.pub root@node1
ssh-copy-id -i ./id_rsa.pub root@node2
ssh-copy-id -i ./id_rsa.pub root@node3
验证是否实现免密登陆
[root@node3 .ssh]# ssh node1
Last login: Thu Jul 11 07:17:26 2019 from node3
[root@node1 ~]# ssh node2
Last login: Thu Jul 11 07:17:32 2019 from node1
[root@node2 ~]# ssh node3
Last login: Thu Jul 11 07:17:16 2019 from node2
安装JDK
安装命令
rpm -ivh jdk-7u79-linux-x64.rpm
配置环境变量
vi .bash_profile
export JAVA_HOME=/usr/java/jdk1.7.0_79
export PATH=$PATH:$JAVA_HOME/bin
source .bash_profile
配置hadoop完全分布式
解压hadoop
tar -zxvf hadoop-2.7.0.tar.gz
配置环境变量
export JAVA_HOME=/usr/java/jdk1.7.0_79
export HADOOP_HOME=/root/hadoop-2.5.1
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin
配置文件
配置hadoop-env.sh
vi ./hadoop-2.7.0/etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr/java/jdk1.7.0_79
export HDFS_NAMENODE=root
export HDFS_DATANODE=root
export HDFS_SECONDARYNAMENODE_USER=root
配置yarn-env.sh
vi ./hadoop-2.7.0/etc/hadoop/yarn-env.sh
if [ "$JAVA_HOME" != "" ]; then
#echo "run java in $JAVA_HOME"
JAVA_HOME=/usr/java/jdk1.7.0_79
配置core-site.xml
vi ./hadoop-2.7.0/etc/hadoop/core-site.xml
<configuration>
<property>
<name>dfs.defaultFS</name>
<value>hdfs://node1:9820</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoopdata</value>
</property>
</configuration>
配置hdfs-site.xml
vi ./hadoop-2.7.0/etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>node2:9000</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>node2:9001</value>
</property>
</configuration>
1.dfs.replication属性代表有几台副本节点
配置节点
cd ./hadoop-2.7.0/etc/hadoop/
touch maters
touch slaves
格式化HDFS
hdfs namenode -format
start-dfs.sh
注意事项
1.虚拟机允许外界访问必须关闭防火墙。
2.hadoop2.x通过端口号50070访问，hadoop3.x通过9870访问
