hdfs多副本删除策略

代码:

chooseExcessReplicates

// split nodes into two sets
// moreThanOne contains nodes on rack with more than one replica
// exactlyOne contains the remaining nodes
splitNodesWithRack(candidates, rackMap, moreThanOne, exactlyOne);

// pick one node to delete that favors the delete hint
// otherwise pick one with least space from priSet if it is not empty
// otherwise one node with least space from remains
  • 1.删除节点副本多余一个的
  • 2.删除心跳最久的
  • 3.删除空间最小的
//oldestHeartbeat
if(lastHeartbeat < oldestHeartbeat) {
  oldestHeartbeat = lastHeartbeat;
  oldestHeartbeatStorage = storage;
}
//HDFS可用空间
if (minSpace > free) {
  minSpace = free;
  minSpaceStorage = storage;
}

oldestHeartbeat

long oldestHeartbeat =
  now() - heartbeatInterval * tolerateHeartbeatMultiplier;
  
this.tolerateHeartbeatMultiplier = conf.getInt(
DFSConfigKeys.DFS_NAMENODE_TOLERATE_HEARTBEAT_MULTIPLIER_KEY,4);
this.heartbeatInterval = conf.getLong(
    DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY,3) * 1000;
public static long now() {
  return System.currentTimeMillis();
}
©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成,浏览时请结合常识与多方信息审慎甄别。
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

友情链接更多精彩内容