最近想要在某网站上获取无损音乐资源，并通过自己的虾米音乐歌单来搜索自己喜欢听的歌曲名单
登录前准备
如下图所示，首先找到登录的POST请求（2处，1处为扫码登录），3处即为POST请求的URL，4处为参数设置，5处为返回值
image.png
接下来看看Params，看看POST请求需要提供哪些参数
image.png
1处和2处即为虾米的账号和密码，3处为一个随机参数，
image.png
根据1处提供的URL地址，获取相应cookies中的_xiamitoken参数
登录前代码
根据上面的分析，在登录前需要获取_xiamitoken参数，改代码如下：
def login_pre():
print('获取_xiamitoken的值，为登录必备参数')
url = 'https://login.xiami.com/member/qrcodelogin'
headers = {
'Host':'login.xiami.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Accept':'application/json, text/javascript, */*; q=0.01',
'Accept-Language':'zh-CN,en-US;q=0.7,en;q=0.3',
'Accept-Encoding':'gzip, deflate, br',
'Referer':'https://login.xiami.com/member/login',
'X-Requested-With':'XMLHttpRequest',
'Connection':'keep-alive'
}
payload = {
'lgToken':'7aac8d8dee47f354776a27d7af7cdeb2',
'defaulturl':'https%3A%2F%2Fwww.xiami.com%2F',
't':str(int(time()*1000)) # 时间随机数
}
req = session.get(url=url,params=payload)
# print(req)
print(req.text)
cookies = req.cookies
_xiamitoken = cookies['_xiamitoken']
# print(_xiamitoken)
return _xiamitoken
登录代码
获取参数_xiamitoken后即可进行登录相关操作，代码如下：
def login(_xiamitoken):
print('登录虾米')
url = 'https://login.xiami.com/passport/login'
headers = {
'Host':'login.xiami.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Accept':'application/json, text/javascript, */*; q=0.01',
'Accept-Language':'zh-CN,en-US;q=0.7,en;q=0.3',
'Accept-Encoding':'gzip, deflate, br',
'Referer':'https://login.xiami.com/member/login',
'Content-Type':'application/x-www-form-urlencoded; charset=UTF-8',
'X-Requested-With':'XMLHttpRequest',
'Content-Length':'164',
'Connection':'keep-alive'
}
account = input('虾米账号：')
pw = input('虾米密码：')
payload = {
'_xiamitoken':_xiamitoken,
'done':'https%3A%2F%2Fwww.xiami.com',
'verifycode':'',
'account':account,
'pw':pw,
'submit':'登+录'
}
req = session.post(url=url,data=payload,headers=headers)
# print(req)
print(req.text)
登陆后验证
为了验证是否登录成功，可以获取一下个人信息列表看看：
def login_after():
print('登录后个人信息')
url = 'https://www.xiami.com/index/home'
headers = {
'Host':'www.xiami.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Accept':'application/json, text/javascript, */*; q=0.01',
'Accept-Language':'zh-CN,en-US;q=0.7,en;q=0.3',
'Accept-Encoding':'gzip, deflate, br',
'Referer':'https://www.xiami.com/',
'X-Requested-With':'XMLHttpRequest',
'Connection':'keep-alive'
}
req = session.get(url=url,headers=headers)
req = json.loads(req.text)
# print('用户信息：',req)
nick_name = req['data']['userInfo']['nick_name']
print('nick_name:',nick_name)
如果此处能够正确打印个人用户名，说明已经登录成功，接下来可以查看“我的音乐”列表了
歌单下载
点击“我的音乐”获取个人音乐列表。
步骤如下：
- 首先，随机选取某页（1处）音乐的内容，并获取最大列表页数（2处）
- 其次，查看音乐列表的URL（3处），并搜索过滤出来（4处至6处），
image.png
由上图可见我们需要一个循环变量（pg_num）来表示页数。
然后我们需要获取列表中的歌名（可以顺带获取相应的URL）和演唱者
方法一，通过HTML标签获取
image.png
通过观察可知，这些歌单列表在<tbody>-><tr>-><td class="song_name">下
image.png
方法二，通过正则表达式获取
image.png
由上图可知，歌名和演唱者具有相同的结构，我们可以通过正则表达式获取红框中的内容
pattern = re.compile('</span><a title=".*?".*?<a class="artist_name".*?</a>.*?</td>',re.S)
items = re.findall(pattern,text)
对象items即为本列所有歌单列表，再通过for循环即可逐步提取歌名和演唱者：
for item in items:
yield{
'name':re.match('(.*)<a title="(.*?)"(.*?)',item).group(2),
'href':re.match('(.*)href="(.*?)"(.*?)',item).group(2),
'artist_name':re.search('(.*?)<a class="artist_name"(.*?)">(.*?)</a>.*>',item, re.S).group(3)
}
用yield即可在主函数中把该返回值当中iterable进行读取，完整代码如下：
def lib_song():
print('用户歌单')
headers = {
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0'
}
for pg_num in range(1,8):
url = 'https://www.xiami.com/space/lib-song/u/13963315/page/{page_num}'.format(page_num=pg_num)
req = session.get(url=url,headers=headers)
print(req)
text = req.text
pattern = re.compile('</span><a title=".*?".*?<a class="artist_name".*?</a>.*?</td>',re.S)
items = re.findall(pattern,text)
# print(items)
for item in items:
yield{
'name':re.match('(.*)<a title="(.*?)"(.*?)',item).group(2),
'href':re.match('(.*)href="(.*?)"(.*?)',item).group(2),
'artist_name':re.search('(.*?)<a class="artist_name"(.*?)">(.*?)</a>.*>',item, re.S).group(3)
}
主函数代码
if __name__ == '__main__':
session = requests.session()
_xiamitoken = login_pre()
login(_xiamitoken)
login_after()
items = lib_song()
# for item in items:
# print(item)
df = pd.DataFrame(items)
df.to_csv('XiamiMusic.csv',encoding='utf-8_sig') # 通过encoding解决保存中文乱码问题
lib_song的返回值是iterable对象，可用for循环逐个获取
完整代码
#!/usr/bin/env python3.6
# -*- coding: utf-8 -*-
'''
@author: Haffner2010
@contact: myprojtest@163.com
@Software: Pycharm + Python3.6
@OS:Windows 7 64 bit
@Site:https://www.jianshu.com/u/e031670b216b
@file: XiamiMusic.py
@time: 2018/6/5 20:14
@desc:
'''
import requests
import re
import json
import pandas as pd
from time import time
def login_pre():
print('获取_xiamitoken的值，为登录必备参数')
url = 'https://login.xiami.com/member/qrcodelogin'
headers = {
'Host':'login.xiami.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Accept':'application/json, text/javascript, */*; q=0.01',
'Accept-Language':'zh-CN,en-US;q=0.7,en;q=0.3',
'Accept-Encoding':'gzip, deflate, br',
'Referer':'https://login.xiami.com/member/login',
'X-Requested-With':'XMLHttpRequest',
'Connection':'keep-alive'
}
payload = {
'lgToken':'7aac8d8dee47f354776a27d7af7cdeb2',
'defaulturl':'https%3A%2F%2Fwww.xiami.com%2F',
't':str(int(time()*1000)) # 时间随机数
}
req = session.get(url=url,params=payload)
# print(req)
print(req.text)
cookies = req.cookies
_xiamitoken = cookies['_xiamitoken']
# print(_xiamitoken)
return _xiamitoken
def login(_xiamitoken):
print('登录虾米')
url = 'https://login.xiami.com/passport/login'
headers = {
'Host':'login.xiami.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Accept':'application/json, text/javascript, */*; q=0.01',
'Accept-Language':'zh-CN,en-US;q=0.7,en;q=0.3',
'Accept-Encoding':'gzip, deflate, br',
'Referer':'https://login.xiami.com/member/login',
'Content-Type':'application/x-www-form-urlencoded; charset=UTF-8',
'X-Requested-With':'XMLHttpRequest',
'Content-Length':'164',
'Connection':'keep-alive'
}
account = input('虾米账号：')
pw = input('虾米密码：')
payload = {
'_xiamitoken':_xiamitoken,
'done':'https%3A%2F%2Fwww.xiami.com',
'verifycode':'',
'account':account,
'pw':pw,
'submit':'登+录'
}
req = session.post(url=url,data=payload,headers=headers)
# print(req)
print(req.text)
def login_after():
print('登录后个人信息')
url = 'https://www.xiami.com/index/home'
headers = {
'Host':'www.xiami.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Accept':'application/json, text/javascript, */*; q=0.01',
'Accept-Language':'zh-CN,en-US;q=0.7,en;q=0.3',
'Accept-Encoding':'gzip, deflate, br',
'Referer':'https://www.xiami.com/',
'X-Requested-With':'XMLHttpRequest',
'Connection':'keep-alive'
}
req = session.get(url=url,headers=headers)
req = json.loads(req.text)
# print('用户信息：',req)
nick_name = req['data']['userInfo']['nick_name']
print('nick_name:',nick_name)
def lib_song():
print('用户歌单')
headers = {
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0'
}
for pg_num in range(1,8):
url = 'https://www.xiami.com/space/lib-song/u/13963315/page/{page_num}'.format(page_num=pg_num)
req = session.get(url=url,headers=headers)
print(req)
text = req.text
pattern = re.compile('</span><a title=".*?".*?<a class="artist_name".*?</a>.*?</td>',re.S)
items = re.findall(pattern,text)
# print(items)
for item in items:
yield{
'name':re.match('(.*)<a title="(.*?)"(.*?)',item).group(2),
'href':re.match('(.*)href="(.*?)"(.*?)',item).group(2),
'artist_name':re.search('(.*?)<a class="artist_name"(.*?)">(.*?)</a>.*>',item, re.S).group(3)
}
if __name__ == '__main__':
session = requests.session()
_xiamitoken = login_pre()
login(_xiamitoken)
login_after()
items = lib_song()
# for item in items:
# print(item)
df = pd.DataFrame(items)
df.to_csv('XiamiMusic.csv',encoding='utf-8_sig') # 通过encoding解决保存中文乱码问题
获取的歌单保存在本地csv文件当中，下步将通过csv文件来搜索FLAC音乐资源
注意事项
文中的部分代码有些需要完整的headers，有些只需要设置User-Agent即可，有些完全不需要headers，可以通过不断尝试来判断。
To be continued
有问题再补充
