- 第一次尝试
pip install pyspider - i https://pypi.douban.com/simple
- 报错
Command "python setup.py egg_info" failed with error code 10 in C:\Users\86134\AppData\Local\Temp\pip-install-et5e98b6\pycurl\
- 解决
在网站 (https://www.lfd.uci.edu/~gohlke/pythonlibs/) 下载对应的python版本
pip install 安装路径
重新执行一下
pip install pyspider
运行 pyspider all
Traceback (most recent call last):
File "C:\Users\86134\AppData\Local\Programs\Python\Python37\Scripts\pyspider-script.py", line 11, in <module>
load_entry_point('pyspider==0.3.10', 'console_scripts', 'pyspider')()
File "c:\users\86134\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 489, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "c:\users\86134\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 2793, in load_entry_point
return ep.load()
File "c:\users\86134\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 2411, in load
return self.resolve()
File "c:\users\86134\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 2417, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "c:\users\86134\appdata\local\programs\python\python37\lib\site-packages\pyspider\run.py", line 231
async=True, get_object=False, no_input=False):
^
SyntaxError: invalid syntax
什么玩意儿？语法错误？百度！
async在3.7中是关键字不能作为参数了
两种解决方案
1、替换关键字
2、降低python版本
- 选择第一种
async 分别在
１ ...site-packages/pyspider/run.py
２ .../site-packages/pyspider/fetcher/tornado_fetcher.py
将async替换成_async
继续运行 pyspider all
ValueError: Invalid configuration: - Deprecated option 'domaincontroller': use 'http_authenticator
在安装包中找到pyspider的资源包，然后找到webui文件里面的webdav.py文件打开，修改第209行即可。
将'domaincontroller': NeedAuthController(app),
改为
'http_authenticator':{
'HTTPAuthenticator':NeedAuthController(app),
}
在安装包中找到pyspider的资源包，然后找到webui文件里面的app.py文件打开，修改第95行即可。
'fetch': lambda x: tornado_fetcher.Fetcher(None, None, async=False).fetch(x),
改为
'fetch': lambda x: tornado_fetcher.Fetcher(None, None, _async=False).fetch(x),
作者：坤哥爱工作
链接：https://www.jianshu.com/p/b8abb93ca5e2
来源：简书
简书著作权归作者所有，任何形式的转载都请联系作者获得授权并注明出处。
