Xc.Li
Apr.2016
Python部分
应用Python的BeautifulSoup库和RegEx获取图书馆网页上的座位信息。连续多天在相同时间获取数据，便于横向比较。
简单来说就是在每天7：00~23：00之间在每个小时的00，10，20，30，40，50分钟获取图书馆当时所有楼层的上座情况，然后保存到一个文件里。
获取数据的设计思路参考(这个是上学期编的程序，略有不同)：用python自动获取图书馆空座信息
获得的数据保存为csv格式，数据格式：
date.csv
'time', dec_time,一楼在用，一楼空闲，二楼在用，二楼空闲...六楼在用，六楼空闲
例如：
"D:\OneDrive\Python\data\lib\Apr13.csv"
'22:30:07', 22.5, 19, 17, 150, 172, 203, 258, 153, 219, 135, 214, 34, 22
Stata部分
csv文件导入到Stata：
可以生存一张描绘所有楼层各个时间的图像，生成dta文件，便于进一步处理
Apr13.png
change-format.do
clear
local date Apr13
local day Apr13_Wed3
infile str8 `day'time dec_time `day'l1a `day'l1b `day'l2a `day'l2b `day'l3a `day'l3b `day'l4a `day'l4b `day'l5a `day'l5b `day'l6a `day'l6b using D:\OneDrive\Python\data\lib\\`date'.csv
graph twoway line `day'l1a dec_time ||line `day'l2a dec_time || line `day'l3a dec_time || line `day'l4a dec_time || line `day'l5a dec_time || line `day'l6a dec_time, xlabel(7(1)23) ylabel(0(30)350)
save "D:\lib-data\\`date'.dta"
相关的python程序：
kol模块：自动切换Wifi网络保证在线
timer模块：每五分钟运行一次命令的实现
writocsv模块：见下文
主程序
# get library seats
# Designed By Xc.Li
import urllib
import BeautifulSoup
import re
import time
from os import system
import kol
import writetocsv
import timer
def find_number(keyword):
for tag in tags:
s_tag = str(tag)
flag = re.match(keyword, s_tag)
if flag is not None:
return s_tag[54:-18]
while True:
timer.func(10)
kol.func()
url = 'http://lib.ecust.edu.cn:8081/gateseat/lrp.aspx'
html = urllib.urlopen(url).read()
soup = BeautifulSoup.BeautifulSoup(html)
tags = soup('span')
output = list()
output.append(time.ctime()[11:-5])
# time in decimal
hour = int(time.ctime()[11:13])
minute = int(time.ctime()[14:16])
decimal_time = round(hour + minute/60.0, 2)
output.append(decimal_time)
for floor in [1, 2, 3, 4, 5, 6]:
# print 'Floor:', floor
keyword = '^<span id="Label' + str(floor) + 'f1' + '".*>'
# print 'Used:', find_number(keyword)
output.append(int(find_number(keyword)))
keyword = '^<span id="Label' + str(floor) + 'f2' + '".*>'
# print 'Left:', find_number(keyword)
output.append(int(find_number(keyword)))
print output
str_output = str(output)[1:-1]
writetocsv.write('lib', 'default', str_output)
time.sleep(60)
system('cls')
# auto-quit
if hour == 22 and minute > 29:
print "program ends!"
writetocsv.write('lib', 'default', "program end at:%s" % time.ctime())
time.sleep(5)
break
writetocsv模块
# Write to CSV
# Designed By Xc.Li @ Mar.2016
import time
import os
def write(dirname, filename, content):
if filename == "default":
filename = time.ctime()[4:7]+time.ctime()[8:10]
file_name = "D:\OneDrive\Python\\data\%s\%s.csv" % (dirname, filename)
dir_name = "D:\OneDrive\Python\\data\%s" % dirname
try:
tf = open(file_name, 'a')
except:
os.system("mkdir "+dir_name)
tf = open(file_name, 'a')
tf.write(content)
tf.write('\n')
tf.close()