Python APP Scraping

ConnectionError, openssl

If you encounter this issue:

ConnectionError: ('Connection aborted.', error(54, 'Connection reset by peer'))

it is possible that you are using python2, old python version that does not support openssl sni. 

To fix this problem, check out website

This process wasn't totally straightforward so I thought I'd post a new answer to make it easy to follow for others.

Following this thread, I needed to install these libraries get SNI to work with Python 2:

pyOpenSSL

ndg-httpsclient

pyasn1

However, pyOpenSSL may cause problems when installed withpip install pyOpenSSL. I actually had to remove my existing openssl, since pyOpenSSL version 0.14 didn't seem to work:

pip uninstall pyOpenSSL

The following command installed all necessary dependencies:

pip install pyOpenSSL==0.13 ndg-httpsclient pyasn1

This should get requests to now work with SNI on python 2.

Keep reading for the issues with pyOpenSSL ver. 0.14...

When installing ver. 0.14 I get the following error:

Command /usr/local/opt/python/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/04/3f_y5fw166v03k7b51j1tsl80000gn/T/pip_build_alex/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/04/3f_y5fw166v03k7b51j1tsl80000gn/T/pip-7QR71B-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/var/folders/04/3f_y5fw166v03k7b51j1tsl80000gn/T/pip_build_alex/cryptography

Storing debug log for failure in /Users/alex/.pip/pip.log

and pyOpenSSL installs as ver. 0.14 incompletely:

$ pip show pyOpenSSL

---

Name: pyOpenSSL

Version: 0.14

Location: /usr/local/lib/python2.7/site-packages

Requires: cryptography, six

as can be seen from therequests.get()attempt:

import requests

response = requests.get("http://datamuster.marketdatasuite.com")

(...lots of errors...)

raise ConnectionError(err, request=request)

requests.exceptions.ConnectionError: ('Connection aborted.', error(54, 'Connection reset by peer'))

The following commands revert to pyOpenSSL ver. 0.13 and correct the issue

pip uninstall pyOpenSSL

pip install pyOpenSSL==0.13

and then in python:

import requests

requests.get("http://datamuster.marketdatasuite.com")

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容