2015年,微软亚洲研究院的何凯明团队发布了一种特殊的卷积神经网络——残差神经网络(ResNet)。在残差神经网络出现之前,最深的深度神经网络只有二三十层左右,这该神经网络却可以在实验中轻松达到上百层甚至上千层,另外不会占用过多训练时间,也正因如此,图像识别准确率有了显著增强。此模型更是在同年的ImageNet大赛中,获得图像分类、定位、检测三个项目的冠军。在国际大赛上取得如此优异的成绩,证明了残差神经网络是个实用性强且优异的模型。在本研究中的猫狗二分类的实验中,也是基于残差神经网络来构建分类模型的。
在本文中我们将把kaggle猫狗数据集应用于ResNet-18和ResNet-50网络模型。使用Resnet来探究当前使用卷积神经网络的准确率。如图4-1为ResNet的经典网络结构图——ResNet-18。
ResNet-18都是由BasicBlock组成,从图4-2也可得知50层及以上的ResNet网络模型由BottleBlock组成。在我们就需要将我们预处理过的数据集放入现有的Resnet-18和ResNet-50模型中去训练,首先我们通过前面提到的图像预处理把训练图像裁剪成一个96x96的正方形尺寸,然后输入到我们的模型中,这里就介绍一下ResNet-18的网络模型的结构,因为ResNet50与第五章的ResNet-34模型结构相仿。
ResNet-18的模型结构为:首先第一层是一个7×7的卷积核,输入特征矩阵为[112,112,64],经过卷积核64,stride为2得到出入特征矩阵[56,56,64]。第二层一开始是由一个3×3的池化层组成的,接着是2个残差结构,一开始的输入的特征矩阵为[56,56,64],需要输出的特征矩阵shape为[28,28,128], 然而主分支与shortcut的输出特征矩阵shape必须相同,所以[56,56,64]这个特征矩阵的高和宽从56通过主分支的stride为2来缩减为原来的一半即为28,再通过128个卷积核来改变特征矩阵的深度。然而这里的shortcut加上了一个1x1的卷积核,stride也为2,通过这个stride,输入的特征矩阵的宽和高也缩减为原有的一半,同时通过128个卷积核将输入的特征矩阵的深度也变为了128。第三层,有2个残差结构,输入的特征矩阵shape是[28,28,128],输出特征矩阵shape是[14,14,256], 然而主分支与shortcut的输出特征矩阵shape必须相同,所以[14,14,256]这个特征矩阵的高和宽从14通过主分支的stride为2来缩减为原来的一半即为7,再通过128个卷积核来改变特征矩阵的深度。然而这里的shortcut加上了一个1×1的卷积核,stride也为2,通过这个stride,输入的特征矩阵的宽和高也缩减为原有的一半,同时通过256个卷积核将输入的特征矩阵的深度也变为了256。第四层,有2个残差结构,经过上述的相同的变化过程得到输出的特征矩阵为[7,7,512]。第五层,有2个残差结构, 经过上述的相同的变化过程得到输出的特征矩阵为[1,1,512]。接着是平均池化和全连接层。
import torch.nn as nn
class BasicBlock(nn.Module):
"""Basic Block for resnet 18 and resnet 34
"""
#BasicBlock and BottleNeck block
#have different output size
expansion = 1
def __init__(self, in_channels, out_channels, stride=1):
super().__init__()
#residual function
self.residual_function = nn.Sequential(
nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels, out_channels * BasicBlock.expansion, kernel_size=3, padding=1, bias=False),
nn.BatchNorm2d(out_channels * BasicBlock.expansion)
)
#shortcut
self.shortcut = nn.Sequential()
#the shortcut output dimension is not the same with residual function
#use 1*1 convolution to match the dimension
if stride != 1 or in_channels != BasicBlock.expansion * out_channels:
self.shortcut = nn.Sequential(
nn.Conv2d(in_channels, out_channels * BasicBlock.expansion, kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(out_channels * BasicBlock.expansion)
)
def forward(self, x):
return nn.ReLU(inplace=True)(self.residual_function(x) + self.shortcut(x))
class BottleNeck(nn.Module):
"""Residual block for resnet over 50 layers
"""
expansion = 4
def __init__(self, in_channels, out_channels, stride=1):
super().__init__()
self.residual_function = nn.Sequential(
nn.Conv2d(in_channels, out_channels, kernel_size=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels, out_channels, stride=stride, kernel_size=3, padding=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels, out_channels * BottleNeck.expansion, kernel_size=1, bias=False),
nn.BatchNorm2d(out_channels * BottleNeck.expansion),
)
self.shortcut = nn.Sequential()
if stride != 1 or in_channels != out_channels * BottleNeck.expansion:
self.shortcut = nn.Sequential(
nn.Conv2d(in_channels, out_channels * BottleNeck.expansion, stride=stride, kernel_size=1, bias=False),
nn.BatchNorm2d(out_channels * BottleNeck.expansion)
)
def forward(self, x):
return nn.ReLU(inplace=True)(self.residual_function(x) + self.shortcut(x)) #激活
class ResNet(nn.Module):
def __init__(self, block, num_block, num_classes=2):
super().__init__()
self.in_channels = 64
self.conv1 = nn.Sequential(
nn.Conv2d(3, 64, kernel_size=3, padding=1, bias=False), # 第一个卷积层,输入3通道,输出64通道,卷积核大小3 x 3,padding1
nn.BatchNorm2d(64),
nn.ReLU(inplace=True))
#we use a different inputsize than the original paper
#so conv2_x's stride is 1
# 以下构建残差块, 具体参数可以查看resnet参数表
self.conv2_x = self._make_layer(block, 64, num_block[0], 1)
self.conv3_x = self._make_layer(block, 128, num_block[1], 2)
self.conv4_x = self._make_layer(block, 256, num_block[2], 2)
self.conv5_x = self._make_layer(block, 512, num_block[3], 2)
self.avg_pool = nn.AdaptiveAvgPool2d((1, 1))
self.fc = nn.Linear(512 * block.expansion, num_classes) #fully connected layer
def _make_layer(self, block, out_channels, num_blocks, stride):
"""make resnet layers(by layer i didnt mean this 'layer' was the
same as a neuron netowork layer, ex. conv layer), one layer may
contain more than one residual block
Args:
block: block type, basic block or bottle neck block
out_channels: output depth channel number of this layer
num_blocks: how many blocks per layer
stride: the stride of the first block of this layer
Return:
return a resnet layer
"""
# 扩维
# we have num_block blocks per layer, the first block
# could be 1 or 2, other blocks would always be 1
strides = [stride] + [1] * (num_blocks - 1)#减少特征图尺寸
layers = []
# 特判第一残差块
for stride in strides:
layers.append(block(self.in_channels, out_channels, stride))#不减少特征图尺寸
self.in_channels = out_channels * block.expansion
return nn.Sequential(*layers)
def forward(self, x): #forward方法,即向前计算,通过该方法获取网络输入数据后的输出值
output = self.conv1(x) #第一次卷积
output = self.conv2_x(output)
output = self.conv3_x(output)
output = self.conv4_x(output)
output = self.conv5_x(output)
output = self.avg_pool(output)
output = output.view(output.size(0), -1)# resize batch-size output H
output = self.fc(output)
return output
def resnet18():
""" return a ResNet 18 object
"""
return ResNet(BasicBlock, [2, 2, 2, 2])
def resnet34():
""" return a ResNet 34 object
"""
return ResNet(BasicBlock, [3, 4, 6, 3])
def resnet50():
""" return a ResNet 50 object
"""
return ResNet(BottleNeck, [3, 4, 6, 3])
def resnet101():
""" return a ResNet 101 object
"""
return ResNet(BottleNeck, [3, 4, 23, 3])
def resnet152():
""" return a ResNet 152 object
"""
return ResNet(BottleNeck, [3, 8, 36, 3])