摘录
This wouldn’t be possible without the amazing breakthroughs that led us to the current state of Deep Learning. Here I name some of them:
- Idea: Back Propagation.
Learning representations by back-propagating errors by David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams.
**A theoretical framework for Back-Propagation **by Yann Lecun.
- **Idea: **Better initialization of the parameters of the nets. Something to remember: The initialization strategy should be selected according to the activation function used (next).
Weight Initialization for Deep Networks - Practical aspects of Deep Learning | Coursera
This course will teach you the "magic" of getting deep learning to work well. Rather than the deep learning process…www.coursera.org
How to train your Deep Neural Network
There are certain practices in Deep Learning that are highly recommended, in order to efficiently train Deep Neural…rishy.github.io
CS231n Convolutional Neural Networks for Visual Recognition
Course materials and notes for Stanford class CS231n: Convolutional Neural Networks for Visual Recognition.cs231n.github.io
- **Idea: **Better activation functions. This mean, better ways of approximating the functions faster leading to faster training process.
Understanding Activation Functions in Neural Networks
Recently, a colleague of mine asked me a few questions like “why do we have so many activation functions?”, “why is…medium.com
Activation Functions: Neural Networks
Sigmoid, tanh, Softmax, ReLU, Leaky ReLU EXPLAINED !!!towardsdatascience.com
- **Idea: **Dropout. Better ways of preventing overfitting and more.
Learning Less to Learn Better — Dropout in (Deep) Machine learning
In this post, I will primarily discuss the concept of dropout in neural networks, specifically deep nets, followed by…medium.com
Dropout: A Simple Way to Prevent Neural Networks from Overfitting, a great paper by Srivastava, Hinton and others.
- **Idea: **Convolutional Neural Nets (CNNs).
Gradient based learning applied to document recognition by Lecun and others
ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky and others.
- **Idea: **Residual Nets (ResNets).
[1512.03385v1] Deep Residual Learning for Image Recognition
Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the…arxiv.org
[1608.02908] Residual Networks of Residual Networks: Multilevel Residual Networks
Abstract: A residual-networks family with hundreds or even thousands of layers dominates major image recognition tasks…arxiv.org
- **Idea: **Region Based CNNs. Used for object detection and more.
[1311.2524v5] Rich feature hierarchies for accurate object detection and semantic segmentation
Abstract: Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few…arxiv.org
[1703.06870] Mask R-CNN
Abstract: We present a conceptually simple, flexible, and general framework for object instance segmentation. Our…arxiv.org
facebookresearch/Detectron
Detectron - FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and…github.com
- **Idea: **Recurrent Neural Networks (RNNs) and LSTMs.
A Beginner's Guide to Recurrent Networks and LSTMs
Contents The purpose of this post is to give students of neural networks an intuition about the functioning of…deeplearning4j.org
Understanding LSTM Networks -- colah's blog
These loops make recurrent neural networks seem kind of mysterious. However, if you think a bit more, it turns out that…colah.github.io
Recurrent Layers - Keras Documentation
input_length: Length of input sequences, to be specified when it is constant. This argument is required if you are…keras.io
BTW: It was shown by Liao and Poggio (2016) that ResNets == RNNs, arXiv:1604.03640v1.
- **Idea: **Generative Adversarial Networks (GANs).
[1406.2661v1] Generative Adversarial Networks
Abstract: We propose a new framework for estimating generative models via an adversarial process, in which we…arxiv.org
nashory/gans-awesome-applications
gans-awesome-applications - Curated list of awesome GAN applications and demogithub.com
**10. Idea: **Capsule Networks.
What is a CapsNet or Capsule Network?
What is a Capsule Network? What is Capsule? Is CapsNet better than Convolutional Neural Network (CNN)? This article is…hackernoon.com
Understanding Hinton’s Capsule Networks. Part I: Intuition.
Part of Understanding Hinton’s Capsule Networks Series:medium.com
