Learning to store information over extended time intervals by recurrent backpropagation takes
a very long time, mostly because of insufficient, decaying error backflow. We briefly review …

Recurrent nets are in principle capable to store past inputs to produce the currently desired
output. Because of this property recurrent nets are used in time series prediction and process …

Generative Adversarial Networks (GANs) excel at creating realistic images with complex
models for which maximum likelihood is infeasible. However, the convergence of GAN training …

We introduce the "exponential linear unit" (ELU) which speeds up learning in deep neural
networks and leads to higher classification accuracies. Like rectified linear units (ReLUs), …

Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural
language processing via recurrent neural networks (RNNs). However, success stories of …

Recurrent networks (crossreference Chapter 12) can, in principle, use their feedback
connections to store representations of recent input events in the form of activations. The most …

The Tox21 Data Challenge has been the largest effort of the scientific community to
compare computational methods for toxicity prediction. This challenge comprised 12,000 …

Standard recurrent nets cannot deal with long minimal time lags between relevant signals.
Several recent NIPS papers propose alter (cid: 173) native methods. We first show: problems …

We present a new algorithm for finding low-complexity neural networks with high
generalization capability. The algorithm searches for a “flat” minimum of the error function. A flat …

This paper introduces the application of gradient descent methods to meta-learning. The
concept of “meta-learning”, ie of a system that improves or discovers a learning algorithm, has …