摘要:python3.x 版本将urllib与urllib2合并为urllib,便于操作。简单介绍一下用过的urllib.request和urllib.parse
urllib is a package that collects several modules for working with URLs:
- urllib.request for opening and reading URLs
- urllib.error containing the exceptions raised by urllib.request
- urllib.parse for parsing URLs
- urllib.robotparser for parsing robots.txt files
urllib.request.Request()
Request是一个url请求抽象类,将一个url封装成为一个Request对象。
class urllib.request.Request(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None)
urllib.request.urlopen()
能够打开一个string类型或者Request对象的url链接。
urllib.request.urlopen(url, data=None, [timeout, ]*, cafile=None, capath=None, cadefault=False, context=None)
urllib.request.urlopen('http://www.python.org/')
#或者是打开一个封装好的Request对象
urllib.request.urlretrieve()
同样也是打开一个链接进行检索,并且,将返回的内容保存到文件中。
urllib.request.urlretrieve(url, filename=None, reporthook=None, data=None)
urllib.parse.urlencode() 、urllib.parse.urlquote() 和urllib.parse.urlunquote()
前两个是将一个utf-8类型的字符串url,解码成ascii格式的方法,其中
urllib.parse.urlencode()
只将连接中utf-8编码不在ascii表中的字符翻译成带百分号的ascii表示形式urllib.parse.quote()
将所有的字符,除09、aZ外,全部转换成百分号的ascii形式。urllib.parse.unquote()
是上述方法的逆操作,将带百分号的ascii编码恢复成utf-8的字符形式。
>>>params = {'query':'中文','submit':'search'}
>>>data = urllib.parse.urlencode(params)
>>>data
'query=%E4%B8%AD%E6%96%87&submit=search'
>>>url='http://wwww.baidu.com/search?q=中文'
>>>quotedurl = urllib.parse.quote(url)
'http%3A//wwww.baidu.com/search%3Fq%3D%E4%B8%AD%E6%96%87'
>>>unquotedurl = urllib.parse.unquote(quotedurl)
>>>unquotedurl
'http://wwww.baidu.com/search?q=中文'
串联一下
>>> import urllib.request
>>> import urllib.parse
>>> url = 'http://pythonprogramming.net/?'
>>> values={'submit':'search','query':'basic'}
>>> data = urllib.parase.urlencode(values)
>>> req = urllib.request.Request(url+data)
>>> resp = urllib.request.urlopen(req)
>>> content = resp.read()
>>> print(content[:100])