ssh-copy-id slave1
There appears to be a gap in the edit log. We expected txid 6865378, but got txid 6878917.
将activenamenode 的/app/hadoop/hdfs/namenode/current 的edits 拷贝到各个journal node
The data volumes are defined by param "dfs.datanode.data.dir" and in this case is set to use data volumes data1, data2, and data3.
<property>
<name>dfs.datanode.failed.volumes.tolerated</name>
<value>0</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/data1/dfs,/data2/dfs,/data3/dfs</value>
</property>
dfs.datanode.du.reserved
fs.defaultFS core-site.xml namenode
hadoop-daemon.sh start namenode
hadoop-daemon.sh start datanode
不推荐namenode和datanode 在一个节点上
1.修改dfs.datanode.max.transfer.threads = 4096 (如果运行hbase的话建议为16384)，指定用于在DataNode间传输block数据的最大线程数，老版本的对应参数为dfs.datanode.max.xcievers
2.修改dfs.datanode.balance.bandwidthPerSec =
52428800，指定
DataNode用于balancer的带宽为50MB，这个示情况而定，如果交换机性能好点的，完全可以 设定为100MB，单位是Byte，如果机器的网卡和交换机的带宽有限，可以适当降低该速度，默认是1048576(1MB)
3.修改dfs.datanode.balance.max.concurrent.moves = 50，指定DataNode上同时用于balance待移动block的最大线程个数，这个值默认是5
如果配置没生效或者不合理的话，Balancer会有如下警告信息：
16/05/17 11:54:59 WARN balancer.Dispatcher: Failed to move blk_1075360746_1920035 with size=134217728 from 192.168.28.48:50010:DISK to 192.168.28.37:50010:DISK through 192.168.28.40:50010: Got error, status message opReplaceBlock BP-647596829-192.168.28.30-1448614319339:blk_1075360746_1920035 received exception Java.io.IOException: Got error, status message Not able to copy block 1075360746 to /192.168.28.37:39630 because threads quota is exceeded., copy block BP-647596829-192.168.28.30-1448614319339:blk_1075360746_1920035 from /192.168.28.40:50010, block move is failed
Hadoop Balancer的步骤：
1、从namenode获取datanode磁盘的使用情况2、计算需要把哪些数据移动到哪些节点3、分别移动，完成后删除旧的block信息4、循环执行，直到达到平衡标准
