case:
ERROR TransportChannelHandler: Connection to /172.31.47.51:29662 has been quiet for 500000 ms while there are outstanding requests. Assuming connection is dead; please adjust spark.network.timeout if this is wrong.

图片.png

官网对spark.network.timeout的解释， drive端在500000ms(spark.network.timeout 设置的是500) 时间内没有收到excutor端的响应， 断开网络连接， 然后后面用到这个excutor的temp数据时再报File does not exist。

解决方案:
参考 - http://blog.csdn.net/wesleyflagon/article/details/52839165

https://stackoverflow.com/questions/37260230/spark-cluster-full-of-heartbeat-timeouts-executors-exiting-on-their-own

一个方案就是增加spark.network.timeout的大小
另一个方案是在数据处理的时候repartition减少task的数量