Rui Ponte Costa∗
Centre for Neural Circuits and Behaviour
Dept. of Physiology, Anatomy and Genetics
University of Oxford, Oxford, UK
rui.costa@cncb.ox.ac.uk
Yannis M. Assael∗
Dept. of Computer Science
University of Oxford, Oxford, UK
and DeepMind, London, UK
yannis.assael@cs.ox.ac.uk
Brendan Shillingford∗
Dept. of Computer Science
University of Oxford, Oxford, UK
and DeepMind, London, UK
brendan.shillingford@cs.ox.ac.uk
Nando de Freitas
DeepMind
London, UK
nandodefreitas@google.com
Tim P. Vogels
Centre for Neural Circuits and Behaviour
Dept. of Physiology, Anatomy and Genetics
University of Oxford, Oxford, UK
tim.vogels@cncb.ox.ac.uk
Abstract
Cortical circuits exhibit intricate recurrent architectures that are remarkably similar
across different brain areas. Such stereotyped structure suggests the existence of
common computational principles. However, such principles have remained largely
elusive. Inspired by gated-memory networks, namely long short-term memory
networks (LSTMs), we introduce a recurrent neural network in which information
is gated through inhibitory cells that are subtractive (subLSTM). We propose a
natural mapping of subLSTMs onto known canonical excitatory-inhibitory cortical
microcircuits. Our empirical evaluation across sequential image classification
and language modelling tasks shows that subLSTM units can achieve similar
performance to LSTM units. These results suggest that cortical circuits can be
optimised to solve complex contextual problems and proposes a novel view on
their computational function. Overall our work provides a step towards unifying
recurrent networks as used in machine learning with their biological counterparts.
Cortical microcircuits as gated-recurrent neural networks
最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
- 文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
- 文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
- 文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
推荐阅读更多精彩内容
- 自从公元492年印度僧人跋陀在嵩山建立少林寺以来，经达摩祖师的苦心孤诣修道传法，禅宗一花开五叶，从此开始走出少林，...
