The theorey meets practice workshop series aims to bring practitioners and theoreticians together and to stimulate the exchange between experts from industry and academia.
For practitioners, the workshop should give an idea of exciting new developments which they can use in their work. For theorists, it should provide a forum to frame the practicality of assumptions and recent work, as well as potentially interesting open questions.
|13:30-14:00||Dan Alistarh Compressing Deep Neural Networks for Fun and Profit|
|14:00-14:30||Nathanael Perraudin [Slides] Deepsphere: an almost equivariant graph-based spherical CNN|
|14:30-14:45||Short Break (coffee served on the upper level)|
|14:45-15:15||Nima Riahi Complex Problems in Rail Transport|
|15:15-15:45||Julie Josse [Slides] A Missing Value Tour|
|15:45--16h50||Hands-on Session: Julie Josse [Slides] Review on Missing Values Methods with Demos Participants should bring their own laptop. Lab materials (zip) To install R and Rstudio, follow the steps described here. (you only need to download R and Rstudio, and install both -- you do not need to install git, Rtools, etc.)|
Compressing Deep Neural Networks for Fun and Profit
Deep learning continues to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently executing deep models efficiently on resource-constrained environments, such as mobile or end-user devices. This talk focuses on this question, and will overview some of the model compression techniques we have developed over the past couple of years, and applied to practice at NeuralMagic, a Boston-based startup. In particular, I will talk about tools for inducing high weight (kernel) sparsity in convolutional neural networks, as well as techniques for exploiting and enhancing activation sparsity in deep networks with ReLU-based networks. The lecture will also include a demo, showcasing some of the practical speedups we can achieve on real deployments.
A Missing Value Tour
In many application settings, the data have missing features which make data analysis challenging. An abundant literature addresses missing data as well as more than 150 R packages. In this presentation, I will give an overview on different topics/methods to handle missing values. We will discuss about the inferential framework where the aim is to estimate at best the parameters and their variance in the presence of missing data, matrix completion methods where the aim is to impute as well as possible and also recent results in a supervised-learning setting.
Deepsphere: an almost equivariant graph-based spherical CNN
In the last 5 years, the field of Machine Learning has been revolutionized by the success of deep learning. Thanks to the increasing availability of data and computations, we now are able to train very complex/deep models and hence solve challenging tasks better than we ever did. Nevertheless, Deep Learning is successful when the network architecture exploits properties of the data, allowing efficient and principled learning. For example, convolutional neural networks (CNNs) revolutionized computer vision because the network architecture has been specifically designed to deal with images. CNNs have important advantages, but their main strength is that they are built for translation equivariance, meaning that if the input is translated, so will the output, which allows spatial weight sharing using few parameters compared to a full input parametrization, and therefore they are computationally efficient. Unfortunately, not all datasets are images and we need architectures that adapt to other types of data, encoding both domain specific knowledge and data specific characteristics.
In this talk, we will show how graph neural networks can be used to adapt convolution to irregular data domains. In particular we focus on data lying on a sphere and discuss how rotational equivariance can be achieved. Eventually, through the example of the sphere, this talk aims at providing intuitions on how graph neural networks behave in geometrical framework. Hopefully, by the end, you should be able to judge if they have a place in your next deep architecture.
Complex Problems in Rail Transport
The planning, maintenance, and daily operation of a rail system are complex, interrelated problems involving rolling stock, crew, infrastructure, passengers, and other factors. In the past and today, some of this complexity is addressed by conventional analytics or operations research techniques. But new developments in the field of AI and computer hardware allow for new approaches to tackle the hard problems faced by the industry. We will introduce a few problems (from the fields of delay propagation, routing/planning and sensor-based environmental awareness of locomotives) and discuss what currently is or may be attempted to address them.