EECS 598 Unsupervised Feature Learning

Instructor: Prof. Honglak Lee

Instructor webpage:http://www.eecs.umich.edu/~honglak/

Office hours: Th 5pm-6pm, 3773 CSE

Classroom: 1690 CSE

Time: M W 10:30am-12pm

Course Schedule

(Note: this schedule is subject to change.)

DateTopicPapersPresenter

9/8IntroductionHonglak

9/13Sparse codingB. Olshausen, D. Field. Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images. Nature, 1996.

H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. NIPS, 2007.Honglak

9/15Self-taught learning

Application: computer visionR. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML, 2007.

H. Lee, R. Raina, A. Teichman, and A. Y. Ng. Exponential Family Sparse Coding with Application to Self-taught Learning. IJCAI, 2009.

J. Yang, K. Yu, Y. Gong, and T. Huang. Linear Spatial Pyramid Matching Using Sparse Coding for Image Classification. CVPR, 2009.Honglak

9/20Neural networks and deep architectures IY. Bengio. Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, 2009.Chapter 4.

Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. NIPS, 2007.Deepak

9/22Restricted Boltzmann machineY. Bengio. Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, 2009.Chapter 5.Byung-soo

9/27Variants of RBMs and AutoencodersP. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol. Extracting and composing robust features with denoising autoencoders. ICML, 2008.

H. Lee, C. Ekanadham, and A. Y. Ng. Sparse deep belief net model for visual area V2. NIPS, 2008.Chun-Yuen

9/29Deep belief networksY. Bengio. Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, 2009.Chapter 6.

R. Salakhutdinov, PhD Thesis.Chapter 2Anna

10/4Convolutional deep belief networksH. Lee, R. Grosse, R. Ranganath, and A. Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. ICML, 2009.Min-Yian

10/6Application: audioH. Lee, Y. Largman, P. Pham, and A. Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. NIPS, 2009.

A. R. Mohamed, G. Dahl, and G. E. Hinton, Deep belief networks for phone recognition. NIPS 2009 workshop on deep learning for speech recognition.Yash

10/11Factorized models IM. Ranzato, A. Krizhevsky, G. E. Hinton, Factored 3-Way Restricted Boltzmann Machines for Modeling Natural Images. AISTATS, 2010.Chun

10/13Factorized models IIM. Ranzato, G. E. Hinton. Modeling Pixel Means and Covariances Using Factorized Third-Order Boltzmann Machines. CVPR, 2010.Soonam

10/18No class - study break

10/20Project proposal presentations

10/25Temporal modeling IG. Taylor, G. E. Hinton, and S. Roweis. Modeling Human Motion Using Binary Latent Variables. NIPS, 2007.

G. Taylor and G. E. Hinton. Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style. ICML, 2009.Jeshua

10/27Temporal modeling IIG. Taylor, R. Fergus, Y. LeCun and C. Bregler. Convolutional Learning of Spatio-temporal Features. ECCV, 2010.Robert

11/1Energy-based modelsK. Kavukcuoglu, M. Ranzato, R. Fergus, and Y. LeCun, Learning Invariant Features through Topographic Filter Maps. CVPR, 2009.

K. Kavukcuoglu, M. Ranzato, and Y. LeCun, Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition. CBLL-TR-2008-12-01, 2008.Ryan

11/3Pooling and invarianceK. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, What is the Best Multi-Stage Architecture for Object Recognition? ICML, 2009.Min-Yian

11/8Evaluating RBMsR. Salakhutdinov and I. Murray. On the Quantitative Analysis of Deep Belief Networks. ICML, 2008.

R. Salakhutdinov, PhD Thesis.Chapter 4Jeshua

11/10Deep Boltzmann machinesR. Salakhutdinov and G. E. Hinton. Deep Boltzmann machines. AISTATS, 2009.Dae Yon

11/15Local coordinate codingK. Yu, T. Zhang, and Y. Gong. Nonlinear Learning using Local Coordinate Coding, NIPS, 2009.

J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong. Learning Locality-constrained Linear Coding for Image Classification. CVPR, 2010.Robert

11/17Deep architectures IIH. Larochelle, Y. Bengio, J. Louradour and P. Lamblin, Exploring Strategies for Training Deep Neural Networks, JMLR, 2009.Soonam

11/22Deep architectures IIID. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent and S. Bengio, Why Does Unsupervised Pre-training Help Deep Learning? JMLR, 2010.Chun

11/24Application: computer vision IIJ. Yang, K. Yu, and T. Huang. Supervised Translation-Invariant Sparse Coding. CVPR, 2010.

Y. Boureau, F. Bach, Y. LeCun and J. Ponce: Learning Mid-Level Features for Recognition. CVPR, 2010.Dae Yon

11/29Pooling and invariance III. J. Goodfellow, Q. V. Le, A. M. Saxe, H. Lee, and A. Y. Ng. Measuring invariances in deep networks. NIPS, 2009.

Y. Boureau, J. Ponce, Y. LeCun, A theoretical analysis of feature pooling in vision algorithms. ICML, 2010.Anna

12/1Application: natural language processingR. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. ICML, 2009.Guanyu

12/13Project presentations I

12/15Project presentations II

12/19Final project report due