Deep Learning Study
- Understand it math model
- Understand it implementation
- Understand it application
============ Theory =============
Convolution Net
Image
List of Good paper ?
Recurrent Net
- basic sturcture
- understand how the LSTM cell working
- understand how to use Torch to implement LSTM cell (and more generally, how to generally program LSTM cell)
- How to encode for RNN (1 of N hot vector encoding?)
- How to train RNN? (BPTT?) -- and how to implement this training algorithm in Torch ?
Optimization
- Gradient Descent
- Newton's method for optimization
- SGD
- How to implement these method in torch ?
NLP
Preprocessing
Study Road Map
first understand the traditional word embedding technology and new word embedding technology
then explore the application that can be done with these word embedding technology
Word embedding (feature extraction)
application
what application does NLP have that related to NN ??
explore the following topic (collect survey paper)
- email span detection
- credit card fraud detection
- automatically generate article
Useful reference
- Word Embeddings for Lexical Substitution (from Israel)
http://u.cs.biu.ac.il/~nlp/resources/downloads/lexsub_embeddings/
=============== Tools ================
Caffe
Torch
list of torch example or pakcage:
- Don't dig into the implementation of package method too deep at this moment. Focus on understand the method of important class, and connect the purpose of these methods with the math/optimization of NN model.
- Also spend time to review the Lua's table/object oriented programming ...
- In next stage dig into the implementation of the package and method
Theano
list of theano package:
Scikit-Learn
learn how to create machine learning by read Scikitlearn source code
