We study the complexity of functions computable by deep feedforward neural networks with
piecewise linear activations in terms of the symmetries and the number of linear regions that …

This paper explores the complexity of deep feedforward networks with linear pre-synaptic
couplings and rectified linear activations. This is a contribution to the growing body of work …

We study a natural Wasserstein gradient flow on manifolds of probability distributions with
discrete sample spaces. We derive the Riemannian structure for the probability simplex from …

The current paradigm of deep neural networks has been successful in part due to the use of
normalization layers. Normalization layers like Batch Normalization, Layer Normalization …

Graph Neural Networks (GNNs) are limited in their expressive power, struggle with long-range
interactions and lack a principled way to model higher-order structures. These problems …

The pairwise interaction paradigm of graph machine learning has predominantly governed
the modelling of relational systems. However, graphs alone cannot capture the multi-level …

We improve recently published results about resources of restricted Boltzmann machines (RBM)
and deep belief networks (DBN) required to make them universal approximators. We …

… Part of this research was performed while Guido Montúfar and Yu Guang Wang were at
the Institute for Pure and Applied Mathematics (IPAM), which is supported by the National …

Graph neural networks (GNNs) are able to leverage the structure of graph data by passing
messages along the edges of the graph. While this allows GNNs to learn features depending …

The quality of signal propagation in message-passing graph neural networks (GNNs)
strongly influences their expressivity as has been observed in recent works. In particular, for …