Volley已是老牌的网络请求库了(虽然也就3岁不到),虽然是Google官方的库,但是目前OkHttp才正是大行其道。这里之所以写文分析Volley,主要是因为很久以前看过它的源码,但是没有做记录,一想起竟然没什么印象。最近有时间再看看,记录下来。毕竟Google官方出品,必属精品。
官方文档:Transmitting Network Data Using Volley
官方设计图:
其实要说分析它的请求流程,这张设计图已经足够简洁明了了:
- 创建请求
- 添加到请求队列
- 检查本地是否有缓存
- 如有,读取缓存并解析,并将结果分发到主线程
- 如没有,由网络请求分发器分发网络请求
- 请求网络
- 解析网络响应
- 写入缓存
- 分发结果到主线程
但是它具体是怎么实现的,还得看源码。
看看一个最简单的String请求(没错这段测试代码也拷贝自官方文档示例):
private void testVolley() {
// Instantiate the RequestQueue.
RequestQueue queue = Volley.newRequestQueue(this);
String url = "https://www.baidu.com";
// Request a string response from the provided URL.
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
@Override
public void onResponse(String response) {
// Display the first 500 characters of the response string.
Log.e("Volley", response);
}
}, new Response.ErrorListener() {
@Override
public void onErrorResponse(VolleyError error) {
Log.e("Volley", error.getCause().toString());
}
});
// Add the request to the RequestQueue.
queue.add(stringRequest);
}
接下来,按照上图中的步骤,一步一步分析它的请求流程。
1. Request Added to queue in priority order
Volley.java
public static RequestQueue newRequestQueue(Context context) {
return newRequestQueue(context, (BaseHttpStack) null);
}
public static RequestQueue newRequestQueue(Context context, BaseHttpStack stack) {
BasicNetwork network;
if (stack == null) {
if (Build.VERSION.SDK_INT >= 9) {
// 2.3版本及以上,使用HttpURLConnection进行网络请求
network = new BasicNetwork(new HurlStack());
} else {
// 这里在2.3以下使用HttpClient进行网络请求忽略
} else {
network = new BasicNetwork(stack);
}
return newRequestQueue(context, network);
}
HurlStack是干嘛的?先进去看看大概:
/**
* An {@link HttpStack} based on {@link HttpURLConnection}.
*/
public class HurlStack extends BaseHttpStack {
public HurlStack() {
this(null);
}
public HurlStack(UrlRewriter urlRewriter) {
this(urlRewriter, null);
}
public HurlStack(UrlRewriter urlRewriter, SSLSocketFactory sslSocketFactory) {
mUrlRewriter = urlRewriter;
mSslSocketFactory = sslSocketFactory;
}
暂时它还没做些什么,类注释看是基于HttpURLConnection的一个请求。
最后构造器中传进来的两个参数都是null,那么看看BaseHttpStack:
/** An HTTP stack abstraction. */
@SuppressWarnings("deprecation") // for HttpStack
public abstract class BaseHttpStack implements HttpStack {
public abstract HttpResponse executeRequest(
Request<?> request, Map<String, String> additionalHeaders)
throws IOException, AuthFailureError;
/**
* @deprecated use {@link #executeRequest} instead to avoid a dependency on the deprecated
* Apache HTTP library. Nothing in Volley's own source calls this method. However, since
* {@link BasicNetwork#mHttpStack} is exposed to subclasses, we provide this implementation in
* case legacy client apps are dependent on that field. This method may be removed in a future
* release of Volley.
*/
@Deprecated
@Override
public final org.apache.http.HttpResponse performRequest(
Request<?> request, Map<String, String> additionalHeaders)
throws IOException, AuthFailureError {
// 这个方法只供HttpClient请求用,已经废弃!
}
}
HurlStack就是一个负责网络请求的类,我们现在几乎已经没有2.3以下的设备了,所以不用管BaseHttpStack了,到时候直接看HurlStack就行了。
接着继续看BasicNetwork:
/**
* A network performing Volley requests over an {@link HttpStack}.
*/
public class BasicNetwork implements Network {
private static final int SLOW_REQUEST_THRESHOLD_MS = 3000;
private static final int DEFAULT_POOL_SIZE = 4096;
private final BaseHttpStack mBaseHttpStack;
/**
* @param httpStack HTTP stack to be used
*/
public BasicNetwork(BaseHttpStack httpStack) {
// If a pool isn't passed in, then build a small default pool that will give us a lot of
// benefit and not use too much memory.
this(httpStack, new ByteArrayPool(DEFAULT_POOL_SIZE));
}
/**
* @param httpStack HTTP stack to be used
* @param pool a buffer pool that improves GC performance in copy operations
*/
public BasicNetwork(BaseHttpStack httpStack, ByteArrayPool pool) {
mBaseHttpStack = httpStack;
// Populate mHttpStack for backwards compatibility, since it is a protected field. However,
// we won't use it directly here, so clients which don't access it directly won't need to
// depend on Apache HTTP.
mHttpStack = httpStack;
mPool = pool;
}
}
这时候暂时也看不出它具体职责,看看它实现的Network接口:
/**
* An interface for performing requests.
*/
public interface Network {
/**
* Performs the specified request.
* @param request Request to process
* @return A {@link NetworkResponse} with data and caching metadata; will never be null
* @throws VolleyError on errors
*/
NetworkResponse performRequest(Request<?> request) throws VolleyError;
}
嗯,也是负责网络请求的,它跟HurlStack有什么区别等下再分析。
接着继续看Volley中的newRequestQueue(Context, Network)方法:
/** Default on-disk cache directory. */
private static final String DEFAULT_CACHE_DIR = "volley";
private static RequestQueue newRequestQueue(Context context, Network network) {
File cacheDir = new File(context.getCacheDir(), DEFAULT_CACHE_DIR);
RequestQueue queue = new RequestQueue(new DiskBasedCache(cacheDir), network);
queue.start();
return queue;
}
先获取我们的App的Cache目录下"volley"目录的一个File对象,然后创建DiskBasedCache和RequestQueue。DiskBasedCache就是Volley的缓存了。
看看RequestQueue.java
/**
* A request dispatch queue with a thread pool of dispatchers.
*
* Calling {@link #add(Request)} will enqueue the given Request for dispatch,
* resolving from either cache or network on a worker thread, and then delivering
* a parsed response on the main thread.
*/
public class RequestQueue {
/** Number of network request dispatcher threads to start. */
private static final int DEFAULT_NETWORK_THREAD_POOL_SIZE = 4;
/** Cache interface for retrieving and storing responses. */
private final Cache mCache;
/** Network interface for performing requests. */
private final Network mNetwork;
/** Response delivery mechanism. */
private final ResponseDelivery mDelivery;
/** The network dispatchers. */
private final NetworkDispatcher[] mDispatchers;
/** The cache dispatcher. */
private CacheDispatcher mCacheDispatcher;
/** The cache triage queue. */
private final PriorityBlockingQueue<Request<?>> mCacheQueue =
new PriorityBlockingQueue<>();
/** The queue of requests that are actually going out to the network. */
private final PriorityBlockingQueue<Request<?>> mNetworkQueue =
new PriorityBlockingQueue<>();
public RequestQueue(Cache cache, Network network) {
this(cache, network, DEFAULT_NETWORK_THREAD_POOL_SIZE);
}
public RequestQueue(Cache cache, Network network, int threadPoolSize) {
this(cache, network, threadPoolSize,
new ExecutorDelivery(new Handler(Looper.getMainLooper())));
}
public RequestQueue(Cache cache, Network network, int threadPoolSize,
ResponseDelivery delivery) {
mCache = cache;
mNetwork = network;
mDispatchers = new NetworkDispatcher[threadPoolSize];
mDelivery = delivery;
}
RequestQueue可以说是请求的管理者了,在整个请求的过层中,比较重要的几个角色都在这了:
- PriorityBlockingQueue<Request<?>>:请求队列,分为缓存请求队列和网络请求队列、
- CacheDispatcher:负责缓存请求的管理和检查、读取本地缓存、
- Cache:负责缓存的读写、
- NetworkDispatcher:负责网络请求的管理、
- Network:负责具体的网络请求、
- ResponseDelivery:请求结果的分发
它的start()方法:
/**
* Starts the dispatchers in this queue.
*/
public void start() {
stop(); // Make sure any currently running dispatchers are stopped.
// Create the cache dispatcher and start it.
mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery);
mCacheDispatcher.start();
// Create network dispatchers (and corresponding threads) up to the pool size.
for (int i = 0; i < mDispatchers.length; i++) {
NetworkDispatcher networkDispatcher = new NetworkDispatcher(mNetworkQueue, mNetwork,
mCache, mDelivery);
mDispatchers[i] = networkDispatcher;
networkDispatcher.start();
}
}
/**
* Stops the cache and network dispatchers.
*/
public void stop() {
if (mCacheDispatcher != null) {
mCacheDispatcher.quit();
}
for (final NetworkDispatcher mDispatcher : mDispatchers) {
if (mDispatcher != null) {
mDispatcher.quit();
}
}
}
这个start()方法,会先将缓存请求分发器和网络请求分发器都停止,然后再新建,并启动。
这里存在什么潜在的问题呢?
如果我一次性有大量的请求,如果多次调用RequestQueue的start()方法,明显,将会导致大量其他请求失败。
所以在Android Developer官网上,有一部分是将要将RequestQueue封装在一个单例里面,详见Setting Up a RequestQueue。
我们看看官方设计图总第一步的最后:往RequestQueue中添加请求:
public <T> Request<T> add(Request<T> request) {
// Tag the request as belonging to this queue and add it to the set of current requests.
request.setRequestQueue(this);
synchronized (mCurrentRequests) {
mCurrentRequests.add(request);
}
// Process requests in the order they are added.
request.setSequence(getSequenceNumber());
request.addMarker("add-to-queue");
// If the request is uncacheable, skip the cache queue and go straight to the network.
if (!request.shouldCache()) {
mNetworkQueue.add(request);
return request;
}
mCacheQueue.add(request);
return request;
}
它会同步地将新请求加入到当前请求的一个Set中。
每个请求都有一个加入的顺序数字,就是Request里的mSequence;
它会判断这个请求是否需要缓存,需要的话(默认就是),会讲该请求加入到缓存的请求队列中,否则,加入到网络请求队列中。
至此,官方解析图的第一步完成。
接下来,第二步:
2. Request dequeued by CacheDispatcher
之前在RequestQueue的start()方法中,我们就看到它它会先将CacheDispatcher和NetworkDispatcher都停止,然后再创建新的。那么我们就来看看这个CacheDispatcher:
public class CacheDispatcher extends Thread {
public CacheDispatcher(
BlockingQueue<Request<?>> cacheQueue, BlockingQueue<Request<?>> networkQueue,
Cache cache, ResponseDelivery delivery) {
mCacheQueue = cacheQueue;
mNetworkQueue = networkQueue;
mCache = cache;
mDelivery = delivery;
mWaitingRequestManager = new WaitingRequestManager(this);
}
public void quit() {
mQuit = true;
interrupt();
}
@Override
public void run() {
if (DEBUG) VolleyLog.v("start new dispatcher");
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
// Make a blocking call to initialize the cache.
mCache.initialize();
// 死循环
while (true) {
try {
processRequest();
} catch (InterruptedException e) {
// We may have been interrupted because it was time to quit.
if (mQuit) {
return;
}
}
}
}
}
它其实就是一个Thread,quit()也就是调用它的interrupt()方法。我们重点关注它的run()方法,可以看到,它先进行初始化,然后在死循环中不停地执行processRequest()。
mCache.initialize()方法就是初始化本地缓存文件的过程,暂且略过。直接看processRequest()方法:
private void processRequest() throws InterruptedException {
// Get a request from the cache triage queue, blocking until
// at least one is available.
final Request<?> request = mCacheQueue.take();
request.addMarker("cache-queue-take");
// If the request has been canceled, don't bother dispatching it.
if (request.isCanceled()) {
request.finish("cache-discard-canceled");
return;
}
// 尝试从缓存中取出请求数据
Cache.Entry entry = mCache.get(request.getCacheKey());
if (entry == null) {
request.addMarker("cache-miss");
// Cache miss; send off to the network dispatcher.
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
mNetworkQueue.put(request);
}
return;
}
// 如果取出来发现已经过时
if (entry.isExpired()) {
request.addMarker("cache-hit-expired");
request.setCacheEntry(entry);
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
mNetworkQueue.put(request);
}
return;
}
// 缓存命中!
request.addMarker("cache-hit");
Response<?> response = request.parseNetworkResponse(
new NetworkResponse(entry.data, entry.responseHeaders));
request.addMarker("cache-hit-parsed");
if (!entry.refreshNeeded()) {
// Completely unexpired cache hit. Just deliver the response.
mDelivery.postResponse(request, response);
} else {
// Soft-expired cache hit. We can deliver the cached response,
// but we need to also send the request to the network for
// refreshing.
request.addMarker("cache-hit-refresh-needed");
request.setCacheEntry(entry);
// Mark the response as intermediate.
response.intermediate = true;
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
// Post the intermediate response back to the user and have
// the delivery then forward the request along to the network.
mDelivery.postResponse(request, response, new Runnable() {
@Override
public void run() {
try {
mNetworkQueue.put(request);
} catch (InterruptedException e) {
// Restore the interrupted status
Thread.currentThread().interrupt();
}
}
});
} else {
// request has been added to list of waiting requests
// to receive the network response from the first request once it returns.
mDelivery.postResponse(request, response);
}
}
}
其中mCache.get(request.getCacheKey()),进去可以看到,Volley的缓存是以请求的url作为key来缓存的?那么,一个post请求是怎么缓存的呢?这里留个疑问。
至于那个mWaitingRequestManager.maybeAddToWaitingRequests(request),它的作用就相当于是个正在进行和等待的请求的“登记处”,防止重复的网络请求。如果没有请求过,那么就要加入到网络请求队列中。
如果缓存完全命中,那么就调用request的parseNetworkResponse()方法解析数据,并分发请求结果。然后后面还有个Soft-expired,这种情况,也算命中,可以分发此缓存结果,但是在分发之后,还需要请求一次网络,刷新缓存的数据。
好了,我们这里已经分析完CacheDispatcher的分发,它的分发逻辑总结起来就是:
- 它是一个线程,死循环地处理缓存请求队列中的请求
- 使用请求的url作为key,去缓存里读取缓存数据
- 如果命中,就解析缓存数据,并分发请求结果
- 没有命中,就将请求加入到网络请求队列中
那么接下来,我们看看缓存没有命中,请求被加入到网络请求队列中的情况。
3. Request dequeued by NetworkDispatcher
之前在RequestQueue的start()方法中,我们就看到它它会先将CacheDispatcher和NetworkDispatcher都停止,然后再创建新的。而且NetworkDispatcher它创建了4个。那么我们就来看看这个NetworkDispatcher:
public class NetworkDispatcher extends Thread {
public NetworkDispatcher(BlockingQueue<Request<?>> queue,
Network network, Cache cache, ResponseDelivery delivery) {
mQueue = queue;
mNetwork = network;
mCache = cache;
mDelivery = delivery;
}
public void quit() {
mQuit = true;
interrupt();
}
@Override
public void run() {
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
while (true) {
try {
processRequest();
} catch (InterruptedException e) {
// We may have been interrupted because it was time to quit.
if (mQuit) {
return;
}
}
}
}
}
同样地,它也是个Thread,而且它也是死循环,那么就是说,Volley中同时至少有5个线程在不停地运行着。
它也是循环调用processRequest()方法,看看:
private void processRequest() throws InterruptedException {
long startTimeMs = SystemClock.elapsedRealtime();
// Take a request from the queue.
Request<?> request = mQueue.take();
try {
request.addMarker("network-queue-take");
// If the request was cancelled already, do not perform the
// network request.
if (request.isCanceled()) {
request.finish("network-discard-cancelled");
request.notifyListenerResponseNotUsable();
return;
}
addTrafficStatsTag(request);
// 执行网络请求
NetworkResponse networkResponse = mNetwork.performRequest(request);
request.addMarker("network-http-complete");
// If the server returned 304 AND we delivered a response already,
// we're done -- don't deliver a second identical response.
if (networkResponse.notModified && request.hasHadResponseDelivered()) {
request.finish("not-modified");
request.notifyListenerResponseNotUsable();
return;
}
// 解析网络响应数据
Response<?> response = request.parseNetworkResponse(networkResponse);
request.addMarker("network-parse-complete");
// 将请求结果缓存起来
// TODO: Only update cache metadata instead of entire record for 304s.
if (request.shouldCache() && response.cacheEntry != null) {
mCache.put(request.getCacheKey(), response.cacheEntry);
request.addMarker("network-cache-written");
}
// Post the response back.
request.markDelivered();
// 分发请求结果!
mDelivery.postResponse(request, response);
request.notifyListenerResponseReceived(response);
} catch (VolleyError volleyError) {
volleyError.setNetworkTimeMs(SystemClock.elapsedRealtime() - startTimeMs);
parseAndDeliverNetworkError(request, volleyError);
request.notifyListenerResponseNotUsable();
} catch (Exception e) {
VolleyLog.e(e, "Unhandled exception %s", e.toString());
VolleyError volleyError = new VolleyError(e);
volleyError.setNetworkTimeMs(SystemClock.elapsedRealtime() - startTimeMs);
mDelivery.postError(request, volleyError);
request.notifyListenerResponseNotUsable();
}
}
代码有点长,跟着我的注释一个个看,那么我们就进入了下一步:
4. HTTP transaction
这里我们就分析这一行代码:
NetworkResponse networkResponse = mNetwork.performRequest(request);
我们之前的分析中,已经看过Network接口,它的实现是BasicNetwork,进来看看它的performRequest()方法,代码有点长啊,我精简了下:
BasicNetwork.java
@Override
public NetworkResponse performRequest(Request<?> request) throws VolleyError {
long requestStart = SystemClock.elapsedRealtime();
// 又是个死循环
while (true) {
HttpResponse httpResponse = null;
byte[] responseContents = null;
List<Header> responseHeaders = Collections.emptyList();
try {
// 收集请求的Header
Map<String, String> additionalRequestHeaders =
getCacheHeaders(request.getCacheEntry());
// 执行网络请求
httpResponse = mBaseHttpStack.executeRequest(request, additionalRequestHeaders);
int statusCode = httpResponse.getStatusCode();
responseHeaders = httpResponse.getHeaders();
// 处理cache的验证
if (statusCode == HttpURLConnection.HTTP_NOT_MODIFIED) {
Entry entry = request.getCacheEntry();
if (entry == null) {
return new NetworkResponse(HttpURLConnection.HTTP_NOT_MODIFIED, null, true,
SystemClock.elapsedRealtime() - requestStart, responseHeaders);
}
// Combine cached and response headers so the response will be complete.
List<Header> combinedHeaders = combineHeaders(responseHeaders, entry);
return new NetworkResponse(HttpURLConnection.HTTP_NOT_MODIFIED, entry.data,
true, SystemClock.elapsedRealtime() - requestStart, combinedHeaders);
}
// 这里处理响应的数据,对于没有响应内容的,直接给它设一个空的字节数组
InputStream inputStream = httpResponse.getContent();
if (inputStream != null) {
// !!!这里将InputStream转为字节数组了!
responseContents =
inputStreamToBytes(inputStream, httpResponse.getContentLength());
} else {
// Add 0 byte response as a way of honestly representing a
// no-content request.
responseContents = new byte[0];
}
if (statusCode < 200 || statusCode > 299) {
throw new IOException();
}
return new NetworkResponse(statusCode, responseContents, false,
SystemClock.elapsedRealtime() - requestStart, responseHeaders);
} catch (SocketTimeoutException e) {
// 处理各种异常
}
}
}
其中执行网络请求是在我们之前看到过的HurlStack的executeRequest()方法中:
@Override
public HttpResponse executeRequest(Request<?> request, Map<String, String> additionalHeaders)
throws IOException, AuthFailureError {
String url = request.getUrl();
HashMap<String, String> map = new HashMap<>();
map.putAll(request.getHeaders());
map.putAll(additionalHeaders);
if (mUrlRewriter != null) {
String rewritten = mUrlRewriter.rewriteUrl(url);
if (rewritten == null) {
throw new IOException("URL blocked by rewriter: " + url);
}
url = rewritten;
}
URL parsedUrl = new URL(url);
// 到这里,终于看到了HttpURLConnection的身影!
HttpURLConnection connection = openConnection(parsedUrl, request);
for (String headerName : map.keySet()) {
connection.addRequestProperty(headerName, map.get(headerName));
}
// 设置各种请求参数
setConnectionParametersForRequest(connection, request);
// Initialize HttpResponse with data from the HttpURLConnection.
int responseCode = connection.getResponseCode();
if (responseCode == -1) {
// -1 is returned by getResponseCode() if the response code could not be retrieved.
// Signal to the caller that something was wrong with the connection.
throw new IOException("Could not retrieve response code from HttpUrlConnection.");
}
// 下面的代码返回请求的结果
if (!hasResponseBody(request.getMethod(), responseCode)) {
return new HttpResponse(responseCode, convertHeaders(connection.getHeaderFields()));
}
return new HttpResponse(responseCode, convertHeaders(connection.getHeaderFields()),
connection.getContentLength(), inputStreamFromConnection(connection));
}
我们看到了HttpURLConnection请求网络,Android Developer官网上有关于HttpURLConnection的使用介绍。
private HttpURLConnection openConnection(URL url, Request<?> request) throws IOException {
HttpURLConnection connection = createConnection(url);
int timeoutMs = request.getTimeoutMs();
connection.setConnectTimeout(timeoutMs);
connection.setReadTimeout(timeoutMs);
connection.setUseCaches(false);
connection.setDoInput(true);
// use caller-provided custom SslSocketFactory, if any, for HTTPS
if ("https".equals(url.getProtocol()) && mSslSocketFactory != null) {
((HttpsURLConnection)connection).setSSLSocketFactory(mSslSocketFactory);
}
return connection;
}
protected HttpURLConnection createConnection(URL url) throws IOException {
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setInstanceFollowRedirects(HttpURLConnection.getFollowRedirects());
return connection;
}
}
其中设置各种请求参数:
@SuppressWarnings("deprecation")
/* package */ static void setConnectionParametersForRequest(HttpURLConnection connection,
Request<?> request) throws IOException, AuthFailureError {
switch (request.getMethod()) {
case Method.DEPRECATED_GET_OR_POST:
// This is the deprecated way that needs to be handled for backwards compatibility.
// If the request's post body is null, then the assumption is that the request is
// GET. Otherwise, it is assumed that the request is a POST.
byte[] postBody = request.getPostBody();
if (postBody != null) {
connection.setRequestMethod("POST");
addBody(connection, request, postBody);
}
break;
case Method.GET:
// Not necessary to set the request method because connection defaults to GET but
// being explicit here.
connection.setRequestMethod("GET");
break;
case Method.DELETE:
connection.setRequestMethod("DELETE");
break;
case Method.POST:
connection.setRequestMethod("POST");
addBodyIfExists(connection, request);
break;
case Method.PUT:
connection.setRequestMethod("PUT");
addBodyIfExists(connection, request);
break;
case Method.HEAD:
connection.setRequestMethod("HEAD");
break;
case Method.OPTIONS:
connection.setRequestMethod("OPTIONS");
break;
case Method.TRACE:
connection.setRequestMethod("TRACE");
break;
case Method.PATCH:
connection.setRequestMethod("PATCH");
addBodyIfExists(connection, request);
break;
default:
throw new IllegalStateException("Unknown method type.");
}
}
private static void addBodyIfExists(HttpURLConnection connection, Request<?> request)
throws IOException, AuthFailureError {
byte[] body = request.getBody();
if (body != null) {
addBody(connection, request, body);
}
}
private static void addBody(HttpURLConnection connection, Request<?> request, byte[] body)
throws IOException, AuthFailureError {
// Prepare output. There is no need to set Content-Length explicitly,
// since this is handled by HttpURLConnection using the size of the prepared
// output stream.
connection.setDoOutput(true);
connection.addRequestProperty(
HttpHeaderParser.HEADER_CONTENT_TYPE, request.getBodyContentType());
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.write(body);
out.close();
}
这里有两个地方需要注意:
- 在HurlStack#executeRequest()中,setConnectionParametersForRequest()会调用addBodyIfExists()方法,
private static void addBodyIfExists(HttpURLConnection connection, Request<?> request)
throws IOException, AuthFailureError {
byte[] body = request.getBody();
if (body != null) {
addBody(connection, request, body);
}
}
这里request.getBoty()会返回byte[],也就是说,如果同时来多个请求,请求body中带有大量的数据,那么内存就会吃紧。
- 在BasicNetwork中,执行完executeRequest之后,会得到HttpResponse;然后Volley会将HttpResponse中的InputStream转为byte[]!代码如下:
/** Reads the contents of an InputStream into a byte[]. */
private byte[] inputStreamToBytes(InputStream in, int contentLength)
throws IOException, ServerError {
PoolingByteArrayOutputStream bytes =
new PoolingByteArrayOutputStream(mPool, contentLength);
byte[] buffer = null;
try {
if (in == null) {
throw new ServerError();
}
buffer = mPool.getBuf(1024);
int count;
while ((count = in.read(buffer)) != -1) {
bytes.write(buffer, 0, count);
}
return bytes.toByteArray();
} finally {
try {
// Close the InputStream and release the resources by "consuming the content".
if (in != null) {
in.close();
}
} catch (IOException e) {
// This can happen if there was an exception above that left the stream in
// an invalid state.
VolleyLog.v("Error occurred when closing InputStream");
}
mPool.returnBuf(buffer);
bytes.close();
}
}
这里的mPool是一个ByteArrayPool类型的对象:
public class ByteArrayPool {
// 按照使用的先后顺序,保存字节数组
private final List<byte[]> mBuffersByLastUse = new LinkedList<byte[]>();
// 按照字节数组的大小,从小到大排列
private final List<byte[]> mBuffersBySize = new ArrayList<byte[]>(64);
// 当前的缓存池中拥有的总的字节数
private int mCurrentSize = 0;
// 当前缓存池中设定的字节上限。超过的话,会从mBuffersByLastUse中最久没使用的那个,也就是list中的第0个开始删除,同时也相应删除mBuffersBySize中的那个字节数组
private final int mSizeLimit;
// 这个方法从缓存池中获取一个不小于目标长度的字节数组,如果没有,就new出一个
public synchronized byte[] getBuf(int len) {
for (int i = 0; i < mBuffersBySize.size(); i++) {
byte[] buf = mBuffersBySize.get(i);
if (buf.length >= len) {
mCurrentSize -= buf.length;
mBuffersBySize.remove(i);
mBuffersByLastUse.remove(buf);
return buf;
}
}
return new byte[len];
}
// 这个方法会将getBuf获取到的字节数组“归还”给缓存池,并且如果归还之后发现缓存池的总字节数超过了上限,就会执行trim()方法,开始删除最久未使用的那些。
public synchronized void returnBuf(byte[] buf) {
if (buf == null || buf.length > mSizeLimit) {
return;
}
// 先将数组保存到mBuffersByLastUse.add,这是按照使用顺序排列的。
mBuffersByLastUse.add(buf);
// 归还时,会按照大小插入到mBuffersBySize中的合适位置,使用二分搜索算法
int pos = Collections.binarySearch(mBuffersBySize, buf, BUF_COMPARATOR);
if (pos < 0) {
pos = -pos - 1;
}
mBuffersBySize.add(pos, buf);
mCurrentSize += buf.length;
// 检查是否总大小超出限制,如超出就开始删除最久未使用的
trim();
}
上面代码的注释我已经写得很清楚了。
Volley会将网络数据从InputStream转换为byte[],在读取字节的过程中,需要一个byte[]的缓冲池。当请求的接口很多而且又很频繁的时候,如果每次都去创建一个缓冲池,将造成GC的频繁回收,导致内存抖动,出现性能问题。那么Volley是怎么处理的呢?
答案就是这个ByteArrayPool了。它保存了一个按照长度从小到大的byte[]的List,和一个按照添加顺序保存的byte[]的LinkedList。当网络请求后需要byte[]来读取字节流时,就从这里取一个出去,是真正的取出来(remove),然后用完了之后又归还回去(add)。这样就避免了频繁地分配内存。
既然Volley把请求结果都用byte[]数组保存到了内存中,那么想一下,如果同时请求几个大文件,那么内存。。。是不是就爆掉了?
所以,这就是Volley不适合数据量很大的网络请求的原因。
好,终于,网络请求完成了,返回了一个HttpResponse类型的结果,那么我们的第4步就分析完了,接下来:
5. response parse
这里我们就分析NetworkDispatcher的processRequest()方法中的这一行:
Response<?> response = request.parseNetworkResponse(networkResponse);
解析网络响应。
可以看到,它使用的是request的parseNetworkResponse()方法。不同的请求类型,自然解析数据的方式也会不一样。我们这里看看StringRequest,它继承自Request;
public class StringRequest extends Request<String> {
@Override
protected Response<String> parseNetworkResponse(NetworkResponse response) {
String parsed;
try {
parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
} catch (UnsupportedEncodingException e) {
parsed = new String(response.data);
}
return Response.success(parsed, HttpHeaderParser.parseCacheHeaders(response));
}
}
代码很简单,毕竟只是一个String类型的请求,就不多看了。接下来,看看Volley是怎么将请求缓存起来的。
6. cache write (if applicable)
这里我们分析的是NetworkDispatcher的processRequest()方法的
mCache.put(request.getCacheKey(), response.cacheEntry);
这一行。
我们之前提到过,Volley缓存的key其实就是url,现在我们一看究竟:
Request.java
public abstract class Request<T> implements Comparable<Request<T>> {
public Request(int method, String url, Response.ErrorListener listener) {
mMethod = method;
mUrl = url;
mErrorListener = listener;
setRetryPolicy(new DefaultRetryPolicy());
mDefaultTrafficStatsTag = findDefaultTrafficStatsTag(url);
}
public String getCacheKey() {
return getUrl();
}
public String getUrl() {
return mUrl;
}
}
终于,我们要来看看Cache接口了:
public interface Cache {
Entry get(String key);
void put(String key, Entry entry);
void initialize();
void invalidate(String key, boolean fullExpire);
void remove(String key);
void clear();
/**
* Data and metadata for an entry returned by the cache.
*/
class Entry {
/** The data returned from cache. */
public byte[] data;
/** ETag for cache coherency. */
public String etag;
/** Date of this response as reported by the server. */
public long serverDate;
public long lastModified;
public long ttl;
public long softTtl;
public Map<String, String> responseHeaders = Collections.emptyMap();
public List<Header> allResponseHeaders;
public boolean isExpired() {
return this.ttl < System.currentTimeMillis();
}
/** True if a refresh is needed from the original data source. */
public boolean refreshNeeded() {
return this.softTtl < System.currentTimeMillis();
}
}
}
提供的方法也不多,就是存取删除初始化之类的。它有一个内部类Entry,里面保存的是一个请求的数据。
它的实现者就是DiskBasedCache,看看它:
public class DiskBasedCache implements Cache {
/** Map of the Key, CacheHeader pairs */
private final Map<String, CacheHeader> mEntries =
new LinkedHashMap<String, CacheHeader>(16, .75f, true);
/** Total amount of space currently used by the cache in bytes. */
private long mTotalSize = 0;
/** The maximum size of the cache in bytes. */
private final int mMaxCacheSizeInBytes;
private static final int DEFAULT_DISK_USAGE_BYTES = 5 * 1024 * 1024;
/** High water mark percentage for the cache */
private static final float HYSTERESIS_FACTOR = 0.9f;
public DiskBasedCache(File rootDirectory) {
this(rootDirectory, DEFAULT_DISK_USAGE_BYTES);
}
public DiskBasedCache(File rootDirectory, int maxCacheSizeInBytes) {
mRootDirectory = rootDirectory;
mMaxCacheSizeInBytes = maxCacheSizeInBytes;
}
}
可以看到,Volley的缓存目录最大5M。
之前我们跳过了initialize()方法,现在我们再看看:
@Override
public synchronized void initialize() {
if (!mRootDirectory.exists()) {
if (!mRootDirectory.mkdirs()) {
VolleyLog.e("Unable to create cache dir %s", mRootDirectory.getAbsolutePath());
}
return;
}
File[] files = mRootDirectory.listFiles();
if (files == null) {
return;
}
for (File file : files) {
try {
long entrySize = file.length();
CountingInputStream cis = new CountingInputStream(
new BufferedInputStream(createInputStream(file)), entrySize);
try {
CacheHeader entry = CacheHeader.readHeader(cis);
// NOTE: When this entry was put, its size was recorded as data.length, but
// when the entry is initialized below, its size is recorded as file.length()
entry.size = entrySize;
putEntry(entry.key, entry);
} finally {
// Any IOException thrown here is handled by the below catch block by design.
//noinspection ThrowFromFinallyBlock
cis.close();
}
} catch (IOException e) {
//noinspection ResultOfMethodCallIgnored
file.delete();
}
}
}
private void putEntry(String key, CacheHeader entry) {
if (!mEntries.containsKey(key)) {
mTotalSize += entry.size;
} else {
CacheHeader oldEntry = mEntries.get(key);
mTotalSize += (entry.size - oldEntry.size);
}
mEntries.put(key, entry);
}
- 它会从自己的缓存目录中读取出所有的缓存文件,然后遍历
- 遍历时,会对每个缓存文件创建一个CacheHeader,这个代表一个请求,并且包含文件大小,但是不包括请求体。
- 由此,也就得到了当前缓存目录的总大小
好了, 再看看它的put()方法:
@Override
public synchronized void put(String key, Entry entry) {
pruneIfNeeded(entry.data.length);
File file = getFileForKey(key);
try {
BufferedOutputStream fos = new BufferedOutputStream(createOutputStream(file));
CacheHeader e = new CacheHeader(key, entry);
boolean success = e.writeHeader(fos);
if (!success) {
fos.close();
VolleyLog.d("Failed to write header for %s", file.getAbsolutePath());
throw new IOException();
}
fos.write(entry.data);
fos.close();
putEntry(key, e);
return;
} catch (IOException e) {
}
boolean deleted = file.delete();
if (!deleted) {
VolleyLog.d("Could not clean up file %s", file.getAbsolutePath());
}
}
一开始就来了个pruneIfNeeded()方法。它干嘛的?看名字,删除修剪,应该是在缓存到文件的时候,检查本地缓存的大小,如果太大了,要删除一部分,否则用户的手机sd卡爆掉还是有可能的。看看:
private void pruneIfNeeded(int neededSpace) {
if ((mTotalSize + neededSpace) < mMaxCacheSizeInBytes) {
return;
}
long before = mTotalSize;
int prunedFiles = 0;
Iterator<Map.Entry<String, CacheHeader>> iterator = mEntries.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, CacheHeader> entry = iterator.next();
CacheHeader e = entry.getValue();
// 删除缓存文件
boolean deleted = getFileForKey(e.key).delete();
if (deleted) {
mTotalSize -= e.size;
} else {
// Log...
}
iterator.remove();
prunedFiles++;
// 当如果存进去的大小不超过最大的大小时,停止删除文件。
if ((mTotalSize + neededSpace) < mMaxCacheSizeInBytes * HYSTERESIS_FACTOR) {
break;
}
}
}
好了,接下来看看,Volley真正是怎么缓存请求到文件的:
- 先创建一个File,代表这个请求
- 创建BufferedOutputStream,用于写入文件
- 创建CacheHeader,保存请求的头部等信息
- 使用CacheHeader的writeHeader()方法,写入header
- 然后写入数据
- 再将此请求的header存入内存中。
如此,Volley的缓存机制也就分析完了。
7. Parsed response delivered on main thread
这里我们分析的是NetworkDispatcher的processRequest()方法的
mDelivery.postResponse(request, response);
这一行。
mDelivery是一个接口:
public interface ResponseDelivery {
/**
* Parses a response from the network or cache and delivers it.
*/
void postResponse(Request<?> request, Response<?> response);
/**
* Parses a response from the network or cache and delivers it. The provided
* Runnable will be executed after delivery.
*/
void postResponse(Request<?> request, Response<?> response, Runnable runnable);
/**
* Posts an error for the given request.
*/
void postError(Request<?> request, VolleyError error);
}
它的实现类是ExecutorDelivery。在我们之前构造RequestQueue的时候:
public RequestQueue(Cache cache, Network network, int threadPoolSize) {
this(cache, network, threadPoolSize,
new ExecutorDelivery(new Handler(Looper.getMainLooper())));
}
在ExecutorDelivery构造器中传入的是主线程的一个Handler。
再看看它的具体实现:
public class ExecutorDelivery implements ResponseDelivery {
/** Used for posting responses, typically to the main thread. */
private final Executor mResponsePoster;
public ExecutorDelivery(final Handler handler) {
// Make an Executor that just wraps the handler.
mResponsePoster = new Executor() {
@Override
public void execute(Runnable command) {
handler.post(command);
}
};
}
@Override
public void postResponse(Request<?> request, Response<?> response) {
postResponse(request, response, null);
}
@Override
public void postResponse(Request<?> request, Response<?> response, Runnable runnable) {
request.markDelivered();
request.addMarker("post-response");
mResponsePoster.execute(new ResponseDeliveryRunnable(request, response, runnable));
}
private class ResponseDeliveryRunnable implements Runnable {
private final Request mRequest;
private final Response mResponse;
private final Runnable mRunnable;
public ResponseDeliveryRunnable(Request request, Response response, Runnable runnable) {
mRequest = request;
mResponse = response;
mRunnable = runnable;
}
@SuppressWarnings("unchecked")
@Override
public void run() {
// NOTE: If cancel() is called off the thread that we're currently running in (by
// default, the main thread), we cannot guarantee that deliverResponse()/deliverError()
// won't be called, since it may be canceled after we check isCanceled() but before we
// deliver the response. Apps concerned about this guarantee must either call cancel()
// from the same thread or implement their own guarantee about not invoking their
// listener after cancel() has been called.
// If this request has canceled, finish it and don't deliver.
if (mRequest.isCanceled()) {
mRequest.finish("canceled-at-delivery");
return;
}
// Deliver a normal response or error, depending.
if (mResponse.isSuccess()) {
// 成功响应
mRequest.deliverResponse(mResponse.result);
} else {
// 失败响应
mRequest.deliverError(mResponse.error);
}
// If this is an intermediate response, add a marker, otherwise we're done
// and the request can be finished.
if (mResponse.intermediate) {
mRequest.addMarker("intermediate-response");
} else {
mRequest.finish("done");
}
// If we have been provided a post-delivery runnable, run it.
if (mRunnable != null) {
mRunnable.run();
}
}
}
}
代码还是很简单的,它就是创建了一个ResponseDeliveryRunnable,run方法中调用request的deliverResponse/deliverError等,然后放入线程池中执行,而这个线程池执行到它的时候,会将它使用handler.post到主线程执行。所以,request.deliverResponse/deliverError会在主线程执行。
我们看看StringRequest的deliverResponse():
@Override
protected void deliverResponse(String response) {
Response.Listener<String> listener;
synchronized (mLock) {
listener = mListener;
}
if (listener != null) {
listener.onResponse(response);
}
}
它就是调用了我们在创建StringRequest的时候创建的一个监听。
至此,一次全新的没有缓存的请求就执行完了。
我们之前没有分析缓存命中的情况,代码如下:
Response<?> response = request.parseNetworkResponse(
new NetworkResponse(entry.data, entry.responseHeaders));
其实这就跟有网络时,使用Request的parseNetworkResponse()是一样的处理。
类图总结
图片来自codeKK。
总结起来,Volley中重要的角色包括:
角色 | 作用 |
---|---|
Volley | 负责创建RequestQueue |
Request | 请求 |
RequestQueue | 请求的管理者,内部包含两个PriorityBlockingQueue,缓存请求队列和网络请求队列 |
CacheDispatcher | 缓存请求的分发器,负责判断本地缓存是否存在,如存在,则解析取出的结果并分发给调用者。如不存在,则将请求分发到网络请求队列中 |
Cache | 缓存接口,提供缓存的基本操作 |
DiskBasedCache | 缓存的具体实现 |
Network | 网络请求接口 |
BasicNetwork | 网络请求的实现,具体作用是调起真正的HttpURLConnection,并处理请求的返回数据 |
HttpResponse | HttpURLConnection请求后的数据 |
NetworkResponse | 处理完网络返回结果后封装的响应实体,存入缓存也是此数据 |
Response | 解析之后用于返回给请求发起者 |
ResponseDelivery | 响应分发器接口 |
ExecutorDelivery | 具体的响应分发器实现者 |
设计模式
一款优秀的开源库,仅从设计的角度来说,它必定符合以下几个特点:
- 结构清晰,职责分明
- 易扩展
- 易维护
那么Volley有哪些地方是做到了这些的?
- 整个库的设计,从网络请求开始到返回结果,基本每一步都做了封装:
Request、RequestQueue、CacheDispatcher、Cache、NetworkDispatcher、Network、HttpStack、HttpResponse、NetworkResponse、Response、ResponseDelivery,就看这些都能看出它的执行流程了 - 看上面列出的那些,很多都是接口或者抽象,用户具备很高的自由度去定制化
- 当具备以上两个特点之后,维护起来肯定更加容易
那么,Volley中用到了哪些设计模式呢?
- 策略模式
对于Request,它可以有多个具体的实现,根据不同的需求,作出不同的请求。 - 观察者模式
对于Request,有各种监听,如成功、错误
分析完了之后,有几个问题:
很多人都说,Volley的设计目标就是去进行 数据量不大,但 通信频繁 的网络操作,而对于大数据量的网络操作,比如下载文件等,Volley 的表现就会非常糟糕。那么问题来了:
- 它为什么就适合数据量小,请求频繁的网络操作?
因为它的网络分发线程有4个,所以可以处理较频繁的请求。 - 它为什么不适合大数据量的操作?
因为在request和response阶段,Volley会将body数据都放入到byte[]中,如果一次性请求多而且数据量大,那么内存吃紧是一方面,另一方面,其他网络请求将陷入等待状态。 - 有什么第三方库是没有它的这个问题的?
- 那个库是如何做到的?
这两个问题等我再去看了OkHttp的处理方式之后再来回答。
并且Developer官网介绍上是这么写的:
Volley offers the following benefits:
- Automatic scheduling of network requests.
内部有CatchDispatcher和NetworkDispatcher,按照优先级调度请求。 - Multiple concurrent network connections.
网络请求有4个线程可同时进行 - Transparent disk and memory response caching with standard HTTP cache coherence.
Volley的缓存机制跟Http非常符合。 - Support for request prioritization.
通过在添加Request的时候,设置了他的mSequence值,然后它实现了Comparator接口,添加到PriorityBlockingQueue,完成优先级的调度。 - Cancellation request API. You can cancel a single request, or you can set blocks or scopes of requests to cancel.
RequestQueue提供cancelAll()和stop()方法,前者可以取消指定Tag的请求,后者将停掉所有的网络请求。 - Ease of customization, for example, for retry and backoff.
这是本库一个非常优秀的地方!扩展性太强了,很多都可以自定义,如Request、Cache、Network、BaseHttpStack等等,面向接口和抽象编程的魅力! - Strong ordering that makes it easy to correctly populate your UI with data fetched asynchronously from the network.
自动切换线程,请求网络异步,返回结果时又切换回主线程 - Debugging and tracing tools.
这个有待我进一步去发现