Pytorch 在处理数据集方面,有一个Dataset(torch.utils.data.Dataset)的抽象类,英文叫Input Pipeline,直译为输入管道,可以自定义这个的子类来适应自己的数据集,然后可以输入到DataLoader(torch.utils.data.DataLoader),方便遍历整个数据集。
需要重载的几个方法
class CustomDataset(torch.utils.data.Dataset):
def __init__(self):
# TODO
# 1. Initialize file paths or a list of file names.
def __getitem__(self, index):
# TODO
# 1. Read one data from file (e.g. using numpy.fromfile, PIL.Image.open).
# 2. Preprocess the data (e.g. torchvision.Transform).
# 3. Return a data pair (e.g. image and label).
def __len__(self):
# You should change 0 to the total size of your dataset.