Hadoop 2.7 一些bug和性能改善
Hadoop 2.8
Support async call retry and failover which can be used in async DFS implementation with retry effort.
DFS也支持async操作
Hadoop2.9 版本新特性较多:
Common:
Aliyun OSS Support.
HADOOP Resource Estimator。本特性比较有用,用历史数据来预估资源及任务执行时间。
HDFS:
HDFS Router based federation. 数据分zone访问
YARN:
YARN Timeline Service v.2,这块比较有用。相比老版本timeline service,可以换job flow/yarn applications/attempts收集metrics,区分yarn和应用 metrics。metrics这块变化比较块,原来jobhistory收集,缺点:和任务调度在一块,容易搞死任务调度,后面单独一个服务timeline service,缺点是只能拿到attempts日志,映射不到JOB,现在好了,可以对应整个JOB。
Opportunistic Containers. NM附带Queue能力,提高workloads
后面就俩Capacity Scheduler新特性:
Changing queue configuration via API
Update Resources and Execution Type of an allocated/running container