1，基础知识
- 利用pymongo库进行python与mongoDB数据库的连接
import pymongo
client = pymongo.MongoClient('localhost', 27017)
walden = client['walden'] #创建表格文件
sheet_lines = walden['sheet_tag'] #创建表格中的sheet
使用find()函数展示数据库中数据
$lt, $lte, $gt, $gte, $ne
分别对应
<, <=, >, >=, !=
l == less; g ==greater; e == equal; n == not
2，practice
爬取小猪租房中前三页的房源信息，并筛选出价格高于500RMB的房源
The Code：
import pymongo, requests, time
from bs4 import BeautifulSoup
client = pymongo.MongoClient('localhost', 27017)
walden = client['2_1homework']
sheet_lines = walden['2_1homework']
urls = ['http://bj.xiaozhu.com/search-duanzufang-p{}-0/'.format(i) for i in range(1, 4)]
def get_details(url, data = None):
wb_data = requests.get(url)
soup = BeautifulSoup(wb_data.text, 'lxml')
titles = soup.select('#page_list > ul > li > div.result_btm_con.lodgeunitname > div > a > span')
prices = soup.select('#page_list > ul > li > div > span.result_price > i')
#print(titles, prices)
for i in range(len(titles)):
index = i
title = titles[i].get_text()
price = prices[i].get_text()
data = {
'index' : index,
'title' : title,
'price' : float(price)
}
#print(index, title, price)
sheet_lines.insert_one(data)
def find_price(url, data = None):
for item in sheet_lines.find({'price': {'$gte' : 500}}):
print(item['title'])
for url_single in urls:
get_details(url_single)
find_price(url_single)
time.sleep(2)
3， 总结与反思
需要注意的几点：
- 如何将数据插入数据库
- 字典的创建
Practice makes perfect！
