结论：想要通过积分落户的方式获得北京户口，就要加入一家互联网公司，然后努力工作，最有机会在35岁到50岁之间成功。
抓取
公示只有几天，幸亏我在接口关闭之前把数据抓了（应该是官方怕泄露个人信息，关的很早）。
# 抓取北京市2018年积分落户公示名单
# http://www.bjrbj.gov.cn/integralpublic/settlePerson
import csv
import json
import requests
fw = open('luohu.csv', 'w')
writer = csv.writer(fw)
writer.writerow(['id','name','birthday','company','score'])
def get_publicity(page_number):
url = 'http://www.bjrbj.gov.cn/integralpublic/settlePerson/settlePersonJson?sort=pxid&order=asc&limit=100&offset=0&name=&rows=100&page={}'.format(page_number*100)
header = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36'
}
response = requests.get(url,
headers=header,
timeout=5
)
result = json.loads(response.text)
for item in result['rows']:
id = item['pxid']
name = item['name']
birthday = item['csrq']
company = item['unit']
score = item['score']
print(id,name,birthday,company,score)
writer.writerow([id, name, birthday, company, score])
def main():
for i in range(0,61):
get_publicity(i)
if __name__ == '__main__':
main()
分析&作图
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
用describe()展示一些基本信息
df_luohu = pd.read_csv('luohu.csv')
print(df_luohu.describe())
id score
count 6019.000000 6019.000000
mean 3010.000000 95.654552
std 1737.679967 4.354445
min 1.000000 90.750000
25% 1505.500000 92.330000
50% 3010.000000 94.460000
75% 4514.500000 97.750000
max 6019.000000 122.590000
找到前20名落户人数最多的公司
看看自己的公司在不在榜上
df_luohu_grouped = df_luohu.groupby('company', as_index=False, sort=False).count()[['company','id']]
df_luohu_grouped_sorted = df_luohu_grouped.sort_values('id', ascending=False)
df_luohu_grouped_sorted.head(20)
落户积分分数分布
bins = [90,95,100,105,110,115,120,125]
bins = pd.cut(df_luohu['score'], bins)
def get_stats(group):
return {'count': group.count()}
grouped = df_luohu['score'].groupby(bins)
bin_counts = grouped.apply(get_stats).unstack()
bin_counts.index = ['90~95', '95~100', '100~105', '105~110', '110~115', '115~120', '120~125']
bin_counts.index.name = 'score'
bin_counts.plot(kind='bar', alpha=1, rot=0)
前面的describe信息里可以看到，落户积分的中位数是94.46分（最低90.75，最高122.59）。评分差不多呈几何级数下降。
年龄分布
df_luohu['age'] = ((pd.to_datetime('2018-11') - pd.to_datetime(df_luohu['birthday'])) / pd.Timedelta('365 days'))
bins = [25,30,35,40,45,50,55,60,65,70]
bins = pd.cut(df_luohu['age'], bins)
def get_stats(group):
return {'count': group.count()}
grouped = df_luohu['age'].groupby(bins)
bin_counts = grouped.apply(get_stats).unstack()
bin_counts.index = ['25~30', '30~35', '35~40', '40~45', '45~50', '50~55', '55~60', '60~65', '65~70',]
bin_counts.index.name = 'age'
bin_counts.plot(kind='bar', alpha=1, rot=0)
基本上是人到中年才拿到了北京户口。
