HIERARCHICAL REPRESENTATIONS FOR EFFICIENT ARCHITECTURE SEARCH

Hanxiao Liu∗
Carnegie Mellon University
hanxiaol@cs.cmu.edu
Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu
DeepMind
{simonyan,vinyals,chrisantha,korayk}@google.com
ABSTRACT
We explore efficient neural architecture search methods and present a simple yet
powerful evolutionary algorithm that can discover new architectures achieving
state of the art results. Our approach combines a novel hierarchical genetic representation
scheme that imitates the modularized design pattern commonly adopted
by human experts, and an expressive search space that supports complex topologies.
Our algorithm efficiently discovers architectures that outperform a large
number of manually designed models for image classification, obtaining top-1 error
of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive
with the best existing neural architecture search approaches and represents
the new state of the art for evolutionary strategies on this task. We also present
results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10
and 0.1% less on ImageNet whilst reducing the architecture search time from 36
hours down to 1 hour.