爬虫练习中遇到中文乱码的问题(记录)

代码如下

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res.text)

打开test2.txt,发现中文乱码

中文乱码.png

我们在res后面加一行代码看下返回内容的编码方式

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
print(res.encoding)
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res.text)

输出:

ISO-8859-1

第一种解决方法:

encode('iso-8859-1') 是将gbk编码编码成unicode编码
decode(‘gbk’) 是从unicode编码解码成gbk字符串

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
res = res.text.encode('iso-8859-1').decode('gbk')
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res)

第二种解决方法

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
res.encoding = 'gbk'
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res.text)

感谢https://blog.csdn.net/xfyangle/article/details/60969522

©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容