第五讲作业:爬取京东华为V20评论

代码

import requests
import json
import pymysql
import time

conn = pymysql.connect(host='localhost', user='root', passwd='123456', db='pythondb', port=3306, charset='utf8')
cursor = conn.cursor()

headers = {
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36',
    'referer': 'https://item.jd.com/100002881193.html'
}

urls = ['https://sclub.jd.com/comment/productPageComments.action?productId=100002881193&score=0&sortType=5&page={}&pageSize=10'.format(str(i)) for i in range(0, 32)]

for url in urls:
    res = requests.get(url, headers=headers)
    # print(res.text)
    json_data = json.loads(res.text)
    comments = json_data['comments']
    for comment in comments:
        content = comment['content']
        score = comment['score']
        creationTime = comment['creationTime']
        nickname = comment['nickname']
        productColor = comment['productColor']
        productSize = comment['productSize']
        userClientShow = comment['userClientShow']
        userLevelName = comment['userLevelName']
        cursor.execute("insert into t_comment_new (content, score, creationTime, nickname, productColor, productSize, userClientShow, userLevelName) values(%s, %s, %s, %s, %s, %s, %s, %s)",
                       (content, score, creationTime, nickname, productColor, productSize, userClientShow, userLevelName))
        conn.commit()
    time.sleep(1)
conn.close()


结果

结果

分析

各个版本购买数量
设备来源分布
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容