尝试在用Scrapy爬虫框架
做异步爬虫的过程中,有pipelines模块
涉及将爬取的数据存储到MySQL数据库,这样就涉及到MySQL-python
这个库
关于在windows环境下安装python的第三方库MySQL-python
库碰到的报错:
解决方法:
- 下载Microsoft Visual C++ Compiler for Python 2.7 Download
- 安装组件
解决方法
for 64-bit windows
- install using wheel
pip install wheel
- download from http://www.lfd.uci.edu/~gohlke/pythonlibs/#mysql-python
For python 3.x:
pip install mysqlclient‑1.3.10‑cp36‑cp36m‑win_amd64.whl
For python 2.x:
# version 1.2.5
pip install MySQL_python‑1.2.5‑cp27‑none‑win_amd64.whl
#latest version
pip install mysqlclient‑1.3.10‑cp27‑cp27m‑win_amd64.whl
引用 Stack Overflow链接
http://stackoverflow.com/questions/26866147/mysql-python-install-fatal-error
验证结果