publications

2021

1. ICML

   Dataset Dynamics via Gradient Flows in Probability Space

   Alvarez-Melis, David, and Fusi, Nicolò

   In Proceedings of the 38th International Conference on Machine Learning 18–24 jul 2021

   Abs PDF

   Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify —with high accuracy— previously unseen datasets.

2020

1. NeurIPS

   Geometric Dataset Distances via Optimal Transport

   Alvarez-Melis, David, and Fusi, Nicolo

   In Neural Information Processing Systems (NeurIPS) 18–24 jul 2020

2019

1. AISTATS

   Towards Optimal Transport with Global Invariances

   Alvarez-Melis, David, Jegelka, Stefanie, and Jaakkola, Tommi S.

   In Proceedings of Machine Learning Research 16–18 apr 2019

   Abs PDF

   Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can be directly evaluated. Unfortunately neither requirement is likely to hold when object representations are learned from data. Indeed, automatically derived representations such as word embeddings are typically fixed only up to some global transformations, for example, reflection or rotation. As a result, pairwise distances across two such instances are ill-defined without specifying their relative transformation. In this work, we propose a general framework for optimal transport in the presence of latent global transformations. We cast the problem as a joint optimization over transport couplings and transformations chosen from a flexible class of invariances, propose algorithms to solve it, and show promising results in various tasks, including a popular unsupervised word translation benchmark.

2018

1. NeurIPS

   Towards Robust Interpretability with Self-Explaining Neural Networks

   Alvarez Melis, David, and Jaakkola, Tommi

   16–18 apr 2018