Advances in ML: Theory Meets Practice

Dan Alistarh

Assistant Professor, IST Austria
Machine Learning Research Lead at Neural Magic

Compressing Deep Neural Networks for Fun and Profit

Deep learning continues to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently executing deep models efficiently on resource-constrained environments, such as mobile or end-user devices. This talk focuses on this question, and will overview some of the model compression techniques we have developed over the past couple of years, and applied to practice at NeuralMagic, a Boston-based startup. In particular, I will talk about tools for inducing high weight (kernel) sparsity in convolutional neural networks, as well as techniques for exploiting and enhancing activation sparsity in deep networks with ReLU-based networks. The lecture will also include a demo, showcasing some of the practical speedups we can achieve on real deployments.

Julie Josse

Professor, CMAP, École Polytechnique

A Missing Value Tour

In many application settings, the data have missing features which make data analysis challenging. An abundant literature addresses missing data as well as more than 150 R packages. In this presentation, I will give an overview on different topics/methods to handle missing values. We will discuss about the inferential framework where the aim is to estimate at best the parameters and their variance in the presence of missing data, matrix completion methods where the aim is to impute as well as possible and also recent results in a supervised-learning setting.

Nathanael Perraudin

Senior Data Scientist, Swiss Data Science Center (SDSC)

Deepsphere: an almost equivariant graph-based spherical CNN

In the last 5 years, the field of Machine Learning has been revolutionized by the success of deep learning. Thanks to the increasing availability of data and computations, we now are able to train very complex/deep models and hence solve challenging tasks better than we ever did.
Nevertheless, Deep Learning is successful when the network architecture exploits properties of the data, allowing efficient and principled learning. For example, convolutional neural networks (CNNs) revolutionized computer vision because the network architecture has been specifically designed to deal with images. CNNs have important advantages, but their main strength is that they are built for translation equivariance, meaning that if the input is translated, so will the output, which allows spatial weight sharing using few parameters compared to a full input parametrization, and therefore they are computationally efficient.
Unfortunately, not all datasets are images and we need architectures that adapt to other types of data, encoding both domain specific knowledge and data specific characteristics.

In this talk, we will show how graph neural networks can be used to adapt convolution to irregular data domains. In particular we focus on data lying on a sphere and discuss how rotational equivariance can be achieved. Eventually, through the example of the sphere, this talk aims at providing intuitions on how graph neural networks behave in geometrical framework. Hopefully, by the end, you should be able to judge if they have a place in your next deep architecture.

Nima Riahi

Senior Data Scientist, SBB CFF FFS

Complex Problems in Rail Transport

The planning, maintenance, and daily operation of a rail system are complex, interrelated problems involving rolling stock, crew, infrastructure, passengers, and other factors. In the past and today, some of this complexity is addressed by conventional analytics or operations research techniques. But new developments in the field of AI and computer hardware allow for new approaches to tackle the hard problems faced by the industry. We will introduce a few problems (from the fields of delay propagation, routing/planning and sensor-based environmental awareness of locomotives) and discuss what currently is or may be attempted to address them.