R-CNN, Fast R-CNN, Faster R-CNN

今年四月份的时候，在一个研究院实习时学习了RCNN, Fast RCNN, Faster RCNN系列Object Detection框架，现在总结一下。

一. R-CNN(Regions with CNN features)

1.1 框架结构

rcnn

论文中提到：

Our object detection system consists of three modules. 
The first generates category-independent region proposals. These proposals define the set of candidate detections available to our detector. 
The second module is a large convolutional neural network that extracts a fixed-length feature vector from each region.
The third module is a set of class specific linear SVMs.

Bounding-box Regression
Based on the error analysis, we implemented a simple method to reduce localization errors. Inspired by the bounding-box regression employed in DPM, we train a linear regression model to predict a new detection window given the pool5features for a selective search region proposal.

我们便知道R-CNN由三个部分组成:

1. 提取Region Proposals的模块;
2. 提取特征向量的卷积神经网络;
3. 线性SVM分类器, bouding-box回归(用于物体的定位).

1.2 神经网络结构

rcnn-net

神经网络的输入为依靠selective search方法提取region proposal后经过warped region调整大小, 然后经过5层卷积和2层全连接层，输出结果一方面送入SVM分类，另一方面送去Bounding-box回归.

二. SPP-net(Spatial Pyramid Pooling, 空间金字塔池化层)

2.1 提出背景

spp-1

Existing deep convolutional neural networks (CNNs) require a fixed-size (e.g., 224*224) input image. This requirement is “artificial” and may reduce the recognition accuracy for the images or sub-images of an arbitrary size/scale. In this work, we equip the networks with another pooling strategy, “spatial pyramid pooling”, to eliminate the above requirement.

在说到Fast R-CNN之前, 先提一下SPP-net. 如论文所说, 由于其他网络比如R-CNN的region proposal需要先经过warped region调整成固定大小, 适用性不是很好, 因此SPP-net提出了一种不限制输入大小的网络.

2.2 实现方式

spp-2

输入的图像(无论大小), 先经过卷积神经网络, 网络结果经过选择的3个filter(此处选取的是16, 4, 1)做pooling, 三个输出首尾连接形成固定长度的输出. 至此解决了输入图像大小限制的问题.

三. Fast R-CNN

3.1 框架结构

fast-rcnn-1

A Fast R-CNN network takes as input an entire image and a set of object proposals. 
The network first processes the whole image with several convolutional (conv) and max pooling layers to produce a conv feature map. Then, for each object proposal a region of interest (RoI) pooling layer extracts a fixed-length feature vector from the feature map. Each feature vector is fed into a sequence of fully connected (fc) layers that finally branch into two sibling output layers: one that produces softmax probability estimates over K object classes plus a catch-all “background” class and another layer that outputs four real-valued numbers for each of the K object classes. Each set of 4 values encodes refined bounding-box positions for one of the K classes.

首先selective search提取出的region proposal输入卷积神经网络, 得到的feature map输入RoI pooling层, 提取出一段固定长度的特征向量, 一方面输入softmax层估计物体概率, 另一方面输入Bounding-box回归层.

The RoI layer is simply the special-case of the spatial pyramid pooling layer used in SPPnets in which there is only one pyramid level.

如论文中所提及, RoI层实际上就是SPP-net的一种情况.

3.2 网络结构(VGG16为例)

fast-rcnn-2

四. Faster R-CNN

4.1 框架结构

faster-rcnn-1

Our object detection system, called Faster R-CNN, is composed of two modules.
The first module is a deep fully convolutional network that proposes regions, and the second module is the Fast R-CNN detector that uses the proposed regions.
The entire system is a single, unified network for object detection (Figure 2).

图像输入神经网络后得到feature map, 在进入RoI pooling之前, 先经过一个RPN层(下一点提到), 然后将得到的region proposal和feature map一起输入RoI pooling层, 后续与Fast R-CNN一致.

４.2 RPN层(Region Proposal Network)

faster-rcnn-2

A Region Proposal Network (RPN) takes an image (of any size) as input and outputs a set of rectangular object proposals, each with an objectness score.

RPN层替代了原来的selective search方法来提取region proposal, 提出anchor box的概念, 通过sliding window的移动和选取k个不同比例的anchor box, 最后得到2k个是否为target的分数和4k个物体坐标.(这里的2指的是target/not a target, ４指的是坐标x/y/w/h)

4.3 网络结构

faster-rcnn-net

五. 总结

conclusion

• Fast R-CNN 通过RoI pooling层将R-CNN后面SVM分类与Bounding-box回归做入到神经网络中;
• Faster R-CNN 通过RPN层将Fast R-CNN前面的region proposal提取层整合入神经网络中, 实现End-to-End.

References

[1] Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation.

[2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. (2014). Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition.

[3] Ross Girshick. (2015). Fast R-CNN.

[4] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.

• 我的个人主页：http://www.techping.cn/
• 我的CSDN博客：http://blog.csdn.net/techping
• 我的简书：http://www.jianshu.com/users/b2a36e431d5e/
• 我的GitHub：https://github.com/techping