目前发现该异常为task被kill掉的时候会报,当sparkstreaming任务开启推测执行时,如果有task启动推测执行成功,那么该task对应的原来的task将会被kill掉,此时原来的task异常终止,会报上述错误......
java.io.IOException:
Failed on local exception: java.nio.channels.ClosedByInterruptException;
Host Details : local host is: "hd-119.ld-hadoop.com/172.16.54.129"; destination host is: "hd-101.ld-hadoop.com":8020;
值得注意的是,该报错下方,应该会出现另一行提示代码:
scheduler.TaskSetManager: Task 1.0 in stage 2.0 (TID 4) failed,
but another instance of the task has already succeeded, so not re-queuing the task to be re-executed.
