1.requests.get()请求
用法:
response= requests.get(url,headers=headers)
2.requests.post()请求
哪些地方会用到post请求?
登录注册(POST 比 GET更安全)
需要传出大文本内容的时候(POST 请求对数据长度没有要求)
用法:
response= requests.get(url,data,headers= headers)
3.使用request是提供的session类来请求登录之后的网站思路:
1)实例化session
2)使用Session发送请求,登录对应的网站,把cookie保存在session中
3)再次使用Session请求登录之后才能访问的网站,session能够自动的携带登录成功时保存在其中的cookie进行请求。
4.response 的 常用方法:
response.text 返回的是经过某种解码的string类型
response.content 返回的是没有经过任何解码的字节类型
response.status_code:查看状态码
200 说明访问的一个url成功,并 不代表我们要访问的url 成功,因为url 重定向
response.request.headers :查看请求头
response.headers :查看响应头
response.request.url :查看请求url
response.url :查看响应url
(response.request.url 和 response.url 可能会不一样,因为发送的url被重定向)
5.寻找登录的POST地址
1.在form 表单中寻找action对应的url地址
post 的 数据是imput 标签中name的值作为键,真正的用户名密码作为值的字典,post url 的地址就是action对应的url地址
2。并不是所有的表单都有后action对应的url地址,如果没有,抓包
6.requests模拟登录的三种方式
"""
方案一
"""
session = requests.session()
post_url ="http://www.renren.com/PLogin.do"
post_data = {"email":" ", "password":" "}
headers = {
"User-Agent":"Mozilla/5.0 (Linux; Android 5.1.1; Nexus 6 Build/LYZ28E) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Mobile Safari/537.36"}
# 使用session发送post请求,cooki保存其中
session.post(post_url, data=post_data, headers=headers)
# 在使用session进行请求登录之后才能访问的地址
response = session.get("http://www.renren.com/962358740/profile")
# 保存页面信息
with open("renren1.html", "w", encoding="utf-8")as f:
f.write(response.content.decode())
"""
方案二:
不发送cookie,使用cookie 后去登录后的页面
1.cookie过期时间很长的网站
2.在cookies 过期之前能够拿到所有的数据,比较麻烦
3.配合其他程序配合使用,其他程序获取cookie, 当前程序专门请求页面
"""
headers = {
"User-Agent":"Mozilla/5.0 (Linux; Android 5.1.1; Nexus 6 Build/LYZ28E) AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/63.0.3239.84 Mobile Safari/537.36",
"Cookie": "anonymid=jz6lo81xd5ugen; depovince=BJ; _r01_=1; JSESSIONID=abcANEf7iiTCe5BUtA-Xw; " \
"ick_login=9224a681-9e68-4620-9f0c-e86b1462a009; loginfrom=null; " \
"jebe_key=473e7eae-93ca-478d-8099- 07c05a7f317e%7C1dd0f5a21a75e7082c4cea72f175cf2b%7C1565505888579%7C1%7C1565505889504; " \
"jebe_key=473e7eae-93ca-478d-8099- 07c05a7f317e%7C1dd0f5a21a75e7082c4cea72f175cf2b%7C1565505888579%7C1%7C1565505889506; " \
"ick=d9088bc5-3482-4ca8-a0b2-bf1b94860a99; __utma=151146938.735476001.1565506549.1565506549.1565506549.1;" \
" __utmc=151146938; __utmz=151146938.1565506549.1.1.utmcsr=renren.com|utmccn= (referral)|utmcmd=referral|utmcct=/; " \
"__utmb=151146938.3.10.1565506549; wp_fold=0; t=d5a919c45ed7cd2ebac8ef6c3ba7fbbf0; " \
"societyguester=d5a919c45ed7cd2ebac8ef6c3ba7fbbf0; id=962358740; xnsid=807881c2; " \
"jebecookies=be77bdf0-fce7-49e7-80f7-f71119bd07fb|||||"(根据自己获取的session替换)
}
response = requests.post(url="http://www.renren.com/962358740",headers=headers)
print(response.content.decode())
# with open("renren2.html", "w", encoding="utf-8") as f:
# f.write(response.content.decode())
"""
方案三:
不发送cookie,使用cookie 后去登录后的页面
1.cookie过期时间很长的网站
2.在cookies 过期之前能够拿到所有的数据,比较麻烦
3.配合其他程序配合使用,其他程序获取cookie, 当前程序专门请求页面
"""
headers = {
"User-Agent":"Mozilla/5.0 (Linux; Android 5.1.1; Nexus 6 Build/LYZ28E) AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/63.0.3239.84 Mobile Safari/537.36"
}
cookies ="anonymid=jz6lo81xd5ugen; depovince=BJ; _r01_=1; JSESSIONID=abcANEf7iiTCe5BUtA-Xw; " \
"ick_login=9224a681-9e68-4620-9f0c-e86b1462a009; loginfrom=null; " \
"jebe_key=473e7eae-93ca-478d-8099-07c05a7f317e%7C1dd0f5a21a75e7082c4cea72f175cf2b%7C1565505888579%7C1%7C1565505889504; " \
"jebe_key=473e7eae-93ca-478d-8099-07c05a7f317e%7C1dd0f5a21a75e7082c4cea72f175cf2b%7C1565505888579%7C1%7C1565505889506; " \
"ick=d9088bc5-3482-4ca8-a0b2-bf1b94860a99; __utma=151146938.735476001.1565506549.1565506549.1565506549.1;" \
" __utmc=151146938; __utmz=151146938.1565506549.1.1.utmcsr=renren.com|utmccn=(referral)|utmcmd=referral|utmcct=/; " \
"__utmb=151146938.3.10.1565506549; wp_fold=0; t=d5a919c45ed7cd2ebac8ef6c3ba7fbbf0; " \
"societyguester=d5a919c45ed7cd2ebac8ef6c3ba7fbbf0; id=962358740; xnsid=807881c2; " \
"jebecookies=be77bdf0-fce7-49e7-80f7-f71119bd07fb|||||"
# 将cookies 转化成词典行驶
cookies= {i.split("=")[0]:i.split("=")[1]for iin cookies.split(";")}
response = requests.post(url="http://www.renren.com/962358740",headers=headers,cookies=cookies)
print(response.content.decode())
# with open("renren3.html", "w", encoding="utf-8") as f:
# f.write(response.content.decode())