Lecture 1 Neutral Network
1. Learning is looking for a function
Three questions for Training procedure
Model Architecture
classification task binary Classification and Multi-class Classification, but some cases are not easy to be formulated as classification problems.
A single layer of Neutrons (Perceptron)
This bias term is an "always on" feature.
A single neuron can only handle binary classification.
Limitation of Perceptron
Neuron Network Model (Multi-layer Perceptron)
non-linear activation function is usually used in neural network.
What does the "Good" Function mean?
Loss Function design
Function = model parameters
Model Parameter Measurement
Optimization
How to find out the best function
Gradient Descent for Optimization
Stochastic gradient descent (SGD)
Mini-batch SGD
Mini-batch is more effective.
Practical tips
different initialization parameters, set them randomly
Tips for mini-batch training
shuffle training samples before every epoch
Possible solutions
more training samples
some tips:dropout, etc
