Pytorch基础篇二
定义网络
网络的定义的网络均需要通过torch.nn.Module实现，Net类继承torch.nn.Module类。用户定义的类包含__init__()和forward()
__init__()函数中定义卷积层、全连接层等，forward()函数中定义relu、池化等操作
torch.nn仅支持mini-batch,维度的含义是nSamples x nChannels x Height x Width
对于仅有single sample的情况，则input.unsqueeze(x)可以在指定位置处添加一个假的batch维度
回顾
torch.Tensor - A multi-dimensional array.
autograd.Variable - Wraps a Tensor and records the history of operations applied to it. Has the same API as a Tensor, with some additions like backward(). Also holds the gradient w.r.t. the tensor.
nn.Module - Neural network module. Convenient way of encapsulating parameters, with helpers for moving them to GPU, exporting, loading, etc.
nn.Parameter - A kind of Variable, that is automatically registered as a parameter when assigned as an attribute to a Module.
autograd.Function - Implements forward and backward definitions of an autograd operation. Every Variable operation, creates at least a single Function node, that connects to functions that created a Variable and encodes its history.
快速定义网络
torch.nn.Sequential()实现，此时使用的均是nn.Module中的类，而不是像在nn.functional中的函数
损失函数和反向传播
欲得到损失先计算，先获得损失函数criterion=nn.MSELoss(),然后计算具体的损失loss=criterion(output,target)。
反向传播过程中需要先将网络中参数的buffer置空net.zero_grad(),然后执行output.backward()进行反向传播。
更新权重
直接更新
learning_rate = 0.01
for f in net.parameters():
f.data.sub_(f.grad.data * learning_rate)
使用torch.optim更新
import torch.optim as optim
optimizer = optim.SGD(net.parameters(), lr=0.01)
optimizer.zero_grad()
output = net(input)
loss = criterion(output, target)
loss.backward()
optimizer.step()
注意此时是对optimizer进行
zero_grad()
Feb 23th,2018
