本文是DEEP LEARNING WITH PYTORCH: A 60 MINUTE BLITZ的学习笔记
教程主要分为以下4个部分：
- TENSORS (张量)
- A GENTLE INTRODUCTION TO TORCH.AUTOGRAD (Autograd自动求导)
- NEURAL NETWORKS (神经网络)
- TRAINING A CLASSIFIER (训练一个分类器)
下面开始介绍第一部分：
张量（Tensors）是一种特殊的数据结构，类似于数组（arrays）和矩阵（matrices）。在PyTorch中，我们使用张量来编码模型的输入和输出，以及模型的参数。
除Tensors可以在GPU或其他专门的硬件上运行以加速计算外，其他非常类似于NumPy的ndarrays。
import torch
import numpy as np
1. Tensor Initialization
1.1 Directly from data
Tensors can be created directly from data. The data type is automatically inferred.
data = [[1, 2], [3, 4]]
x_data = torch.tensor(data)
1.2 From a NumPy array
Tensors can be created from NumPy arrays.
np_array = np.array(data)
x_np = torch.from_numpy(np_array)
1.3 From another tensor
The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.
x_ones = torch.ones_like(x_data) # retains the properties of x_data
print(f"Ones Tensor: \n {x_ones} \n")
x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data
print(f"Random Tensor: \n {x_rand} \n")
我们发现，默认情况下新变量x_ones与参数张量x_data的数据类型确实相同，除非对参数dtype进行显式指定。
1.4 With random or constant values
shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor
shape = (2, 3,)
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)
print(f"Random Tensor: \n {rand_tensor} \n")
print(f"Ones Tensor: \n {ones_tensor} \n")
print(f"Zeros Tensor: \n {zeros_tensor}")
2. Tensor Attributes
Tensor attributes describe their shape, datatype, and the device on which they are stored.
tensor = torch.rand(3, 4)
print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device tensor is stored on: {tensor.device}")
3. Tensor Operations
Over 100 tensor operations, including transposing, indexing, slicing, mathematical operations, linear algebra, random sampling...
# We move our tensor to the GPU if available
if torch.cuda.is_available():
tensor = tensor.to('cuda')
print(f"Device tensor is stored on: {tensor.device}")
3.1 Standard numpy-like indexing and slicing:
tensor = torch.ones(4, 4)
tensor[:,1] = 0
print(tensor)
3.2 Joining tensors
use torch.cat to concatenate a sequence of tensors along a given dimension
t1 = torch.cat([tensor, tensor, tensor], dim=1)
print(t1)
我们发现张量t1确实是3个张量tensor按第1轴即列（参数dim=1）拼接在一起
3.3 Multiplying tensors
# This computes the element-wise product
print(f"tensor.mul(tensor) \n {tensor.mul(tensor)} \n")
# Alternative syntax:
print(f"tensor * tensor \n {tensor * tensor}")
This computes the matrix multiplication between two tensors:
print(f"tensor.matmul(tensor.T) \n {tensor.matmul(tensor.T)} \n")
# Alternative syntax:
print(f"tensor @ tensor.T \n {tensor @ tensor.T}")
上图的前者相当于常规的向量化（两个矩阵之间的对应相同位置元素相乘），后者是矩阵乘法，相当于矩阵与其转置
相乘，在这里即是
。
3.4 In-place operations
Operations that have a _ suffix are in-place. For example: x.copy_(y), x.t_(), will change x.
In-place operations save some memory, but can be problematic when computing derivatives because of an immediate loss of history. Hence, their use is discouraged.
4. Bridge with NumPy
Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.
4.1 Tensor to NumPy array
t = torch.ones(5)
print(f"t: {t}")
n = t.numpy()
print(f"n: {n}")
t.add_(1)
print(f"t: {t}")
print(f"n: {n}")
A change in the tensor reflects in the NumPy array.
4.2 NumPy array to Tensor
n = np.ones(5)
t = torch.from_numpy(n)
np.add(n, 1, out=n)
print(f"t: {t}")
print(f"n: {n}")
很明显，vice versa。
总结一下用到的API：
函数：tensor(), from_numpy(), ones_like(), rand_like(), rand(), ones(), zeros(), cuda.is_available(), cat(), stack()
tensor的属性和方法：shape, dtype, device, T, to(), mul(), matmul(), add_(), numpy(), copy_(), t_()
特殊符号：*, @
参考：
- https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
- https://pytorchbook.cn/chapter2/2.1.1.pytorch-basics-tensor/
- https://www.youtube.com/watch?v=u7x8RXwLKcA
- https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/3dbbd6931d76adb0dc37d4e88b328852/tensor_tutorial.ipynb
