原始GAN

Goodfellow和Bengio等人发表在NIPS 2014年的文章Generative adversary network，是生成对抗网络的开创文章，论文思想启发自博弈论中的二人零和博弈。在二人零和博弈中，两位博弈方的利益之和为零或一个常数，即一方有所得，另一方必有所失。GAN模型中的两位博弈方分别由生成式模型(generative model)和判别式模型(discriminative model)充当。生成模型G捕捉样本数据的分布，判别模型D是一个二分类器，估计一个样本来自于训练数据（而非生成数据）的概率。G和D一般都是非线性映射函数，例如多层感知机、卷积神经网络等。

如图所示，左图是一个判别式模型，当输入训练数据x时，期待输出高概率（接近1）；右图下半部分是生成模型，输入是一些服从某一简单分布（例如高斯分布）的随机噪声z，输出是与训练图像相同尺寸的生成图像。向判别模型D输入生成样本，对于D来说期望输出低概率（判断为生成样本），对于生成模型G来说要尽量欺骗D，使判别模型输出高概率（误判为真实样本），从而形成竞争与对抗。

GAN.png

GAN优势很多：根据实际的结果，看上去产生了更好的样本；GAN能训练任何一种生成器网络；GAN不需要设计遵循任何种类的因式分解的模型，任何生成器网络和任何鉴别器都会有用；GAN无需利用马尔科夫链反复采样，无需在学习过程中进行推断，回避了近似计算棘手的概率的难题。

GAN主要存在的以下问题：网络难以收敛，目前所有的理论都认为GAN应该在纳什均衡上有很好的表现，但梯度下降只有在凸函数的情况下才能保证实现纳什均衡。

GAN发展

一方面GAN的发展很快，这里只是简单粗略将相关论文分了几类，欢迎反馈，持续更新。此外最近ICLR 2017 在进行Open Review，可以关注下ICLR 2017 Conference Track，也有相应论文笔记分享ICLR 2017 | GAN Missing Modes 和 GAN

GAN从2014年到现在发展很快，特别是最近ICLR 2016/2017关于GAN的论文很多，GAN现在有很多问题还有到解决，潜力很大。总体可以将已有的GANs论文分为以下几类

1. GAN Theory
2. GAN in Semi-supervised
3. Muti-GAN
4. GAN with other Generative model
5. GAN with RNN
6. GAN in Application

GAN Theory

此类关注与无监督GAN本身原理的研究：比较两个分布的距离；用DL的一些方法让GAN快速收敛等等。相关论文有：

• GAN: Goodfellow, Ian, et al. "Generative adversarial nets." Advances in Neural Information Processing Systems. 2014.
• LAPGAN: Denton, Emily L., Soumith Chintala, and Rob Fergus. "Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks." Advances in neural information processing systems. 2015.
• DCGAN: Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).
• Improved GAN: Salimans, Tim, et al. "Improved techniques for training gans." arXiv preprint arXiv:1606.03498 (2016).
• InfoGAN: Chen, Xi, et al. "Infogan: Interpretable representation learning by information maximizing generative adversarial nets." arXiv preprint arXiv:1606.03657(2016).**
• EnergyGAN: Zhao, Junbo, Michael Mathieu, and Yann LeCun. "Energy-based Generative Adversarial Network." arXiv preprint arXiv:1609.03126 (2016).
• Creswell, Antonia, and Anil A. Bharath. "Task Specific Adversarial Cost Function." arXiv preprint arXiv:1609.08661 (2016).
• f-GAN: Nowozin, Sebastian, Botond Cseke, and Ryota Tomioka. "f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization." arXiv preprint arXiv:1606.00709 (2016).
• Unrolled Generative Adversarial Networks, ICLR 2017 Open Review
• Improving Generative Adversarial Networks with Denoising Feature Matching, ICLR 2017 Open Review
• Mode Regularized Generative Adversarial Networks, ICLR 2017 Open Review
• b-GAN: Unified Framework of Generative Adversarial Networks, ICLR 2017 Open Review
• Mohamed, Shakir, and Balaji Lakshminarayanan. "Learning in Implicit Generative Models." arXiv preprint arXiv:1610.03483 (2016).

GAN in Semi-supervised

此类研究将GAN用于半监督学习，相关论文有：

• Springenberg, Jost Tobias. "Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks." arXiv preprint arXiv:1511.06390 (2015).
• Odena, Augustus. "Semi-Supervised Learning with Generative Adversarial Networks." arXiv preprint arXiv:1606.01583 (2016).

Muti-GAN

此类研究将多个GAN进行组合，相关论文有：

• CoupledGAN: Liu, Ming-Yu, and Oncel Tuzel. "Coupled Generative Adversarial Networks." arXiv preprint arXiv:1606.07536 (2016).
• Wang, Xiaolong, and Abhinav Gupta. "Generative Image Modeling using Style and Structure Adversarial Networks." arXiv preprint arXiv:1603.05631(2016).
• Generative Adversarial Parallelization, ICLR 2017 Open Review
• LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation, ICLR 2017 Open Review

GAN with other Generative model

此类研究将GAN与其他生成模型组合，相关论文有：

• Dosovitskiy, Alexey, and Thomas Brox. "Generating images with perceptual similarity metrics based on deep networks." arXiv preprint arXiv:1602.02644(2016).
• Larsen, Anders Boesen Lindbo, Søren Kaae Sønderby, and Ole Winther. "Autoencoding beyond pixels using a learned similarity metric." arXiv preprint arXiv:1512.09300 (2015).
• Theis, Lucas, and Matthias Bethge. "Generative image modeling using spatial lstms." Advances in Neural Information Processing Systems. 2015.

GAN with RNN

此类研究将GAN与RNN结合（也以参考Pixel RNN），相关论文有：

• Im, Daniel Jiwoong, et al. "Generating images with recurrent adversarial networks." arXiv preprint arXiv:1602.05110 (2016).
• Kwak, Hanock, and Byoung-Tak Zhang. "Generating Images Part by Part with Composite Generative Adversarial Networks." arXiv preprint arXiv:1607.05387 (2016).
• Yu, Lantao, et al. "SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient." arXiv preprint arXiv:1609.05473 (2016).

GAN in Application

此类研究将GAN的实际运用（不包括图像生成），相关论文有：

• Zhu, Jun-Yan, et al. "Generative visual manipulation on the natural image manifold." European Conference on Computer Vision. Springer International Publishing, 2016.
• Creswell, Antonia, and Anil Anthony Bharath. "Adversarial Training For Sketch Retrieval." European Conference on Computer Vision. Springer International Publishing, 2016.
• Reed, Scott, et al. "Generative adversarial text to image synthesis." arXiv preprint arXiv:1605.05396 (2016).
• Ravanbakhsh, Siamak, et al. "Enabling Dark Energy Science with Deep Generative Models of Galaxy Images." arXiv preprint arXiv:1609.05796(2016).
• Abadi, Martín, and David G. Andersen. "Learning to Protect Communications with Adversarial Neural Cryptography." arXiv preprint arXiv:1610.06918(2016).
• Odena, Augustus, Christopher Olah, and Jonathon Shlens. "Conditional Image Synthesis With Auxiliary Classifier GANs." arXiv preprint arXiv:1610.09585 (2016).
• Ledig, Christian, et al. "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network." arXiv preprint arXiv:1609.04802 (2016).
• Nguyen, Anh, et al. "Synthesizing the preferred inputs for neurons in neural networks via deep generator networks." arXiv preprint arXiv:1605.09304(2016).