递归深度报错
最近在写一些爬虫,按照URL获取html解析,成功后递归:
例:请求def get_detail(mid, page)方法,成功后return get_detail(mid, page+1),失败则更换代理重新return get_detail(mid, page)
然后长时间运行后出现 RuntimeError: maximum recursion depth exceeded,
之后去stackoverflow查发现是python自身的递归深度是999,也就是最多支持return get_detail 999次,否则就会报错。
可以使用如下解决
import sys
sys.setrecursionlimit(limit) #limit为自定义数目,表示深度
官方文档
Set the maximum depth of the Python interpreter stack tolimit. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python.
The highest possible limit is platform-dependent. A user may need to set the limit higher when she has a program that requires deep recursion and a platform that supports a higher limit. This should be done with care, because a too-high limit can lead to a crash.
官方api链接:https://docs.python.org/2/library/sys.html#sys.setrecursionlimit