1. import Libraries
2. Prepare Dataset
- We use MNIST dataset.
- There are 28*28 images and 10 labels from 0 to 9
- Data is not normalized so we divide each image to 255 that is basic normalization for images.
- In order to split data, we use train_test_split method from sklearn library
Size of train data is 80% and size of test data is 20%. - Create feature and target tensors. At the next parts we create variable from these tensors. As you remember we need to define variable for accumulation of gradients.
- batch_size = batch size means is that for example we have data and it includes 1000 sample. We can train 1000 sample in a same time or we can divide it 10 groups which include 100 sample and train 10 groups in order. Batch size is the group size. For example, I choose batch_size = 100, that means in order to train all data only once we have 336 groups. We train each groups(336) that have batch_size(quota) 100. Finally we train 33600 sample one time.
-
epoch: 1 epoch means training all samples one time.
In our example: we have 33600 sample to train and we decide our batch_size is 100. Also we decide epoch is 29(accuracy achieves almost highest value when epoch is 29). Data is trained 29 times. Question is that how many iteration do I need? Lets calculate:- training data 1 times = training 33600 sample (because data includes 33600 sample)
- But we split our data 336 groups(group_size = batch_size = 100) our data
- Therefore, 1 epoch(training data only once) takes 336 iteration
- We have 29 epoch, so total iterarion is 9744(that is almost 10000 which I used)
- TensorDataset(): Data set wrapping tensors. Each sample is retrieved by indexing tensors along the first dimension.
- DataLoader(): It combines dataset and sample. It also provides multi process iterators over the dataset.
- Visualize one of the images in dataset
3.Convolutional layer:
- Create feature maps with filters(kernels).
- Padding: After applying filter, dimensions of original image decreases. However, we want to preserve as much as information about the original image. We can apply padding to increase dimension of feature map after convolutional layer.
- We use 2 convolutional layer.
- Number of feature map is out_channels = 16
- Filter(kernel) size is 5*5
4.Pooling layer:
- Prepares a condensed feature map from output of convolutional layer(feature map)
- 2 pooling layer that we will use max pooling.
- Pooling size is 2*2
5.Flattening: Flats the features map
6.Fully Connected Layer:
- Artificial Neural Network that we learnt at previous part.
- Or it can be only linear like logistic regression but at the end there is always softmax function.
- We will not use activation function in fully connected layer.
- You can think that our fully connected layer is logistic regression.
- We combine convolutional part and logistic regression to create our CNN model.
7. Instantiate Model Class
- create model
8. Instantiate Loss
- Cross entropy loss
- It also has softmax(logistic function) in it.
9. Instantiate Optimizer
- SGD Optimizer
