Spectral methods - the computation of eigenvalues, eigenvectors, and eigenfunctions of matrices and linear operators - are foundational to much of applied math and computational science. In machine learning, these methods are the basis for principal component analysis, and were central to the development of nonparametric ML algorithms like Laplacian eigenmaps and spectral clustering. In the modern deep learning era of ML, with its focus on fitting large parametric models, many of these methods have been left behind. In this talk, I will discuss three different areas where spectral methods intersect with current research in ML. Spectral Inference Networks (SpIN) are a contrastive learning method for deep neural networks that are equivalent to finding the smallest eigenfunctions of a given linear operator on high-dimensional functions. Fermionic Neural Networks (FermiNets) are a class of deep neural networks designed to be used for problems in quantum chemistry and condensed matter physics, greatly improving the accuracy of quantum Monte Carlo calculations over classic methods. The Geometric Manifold Component Estimator (GeoManCEr) is a nonparametric spectral method for unsupervised disentangling of independent data manifolds, which is radically different from probabilistic approaches to disentangling in that it uses curvature as a learning signal. Together, these methods illustrate how classical numerical methods and cutting-edge machine learning techniques can inform one another to solve challenging applied problems and provide new insights.
Bio: David Pfau is a senior research scientist at DeepMind, whose research interests span artificial intelligence, machine learning and computational science. As a PhD student at the Center for Theoretical Neuroscience at Columbia, he worked on algorithms for analyzing and understanding high-dimensional data from neural recordings with Liam Paninski and nonparametric Bayesian methods for predicting time series data with Frank Wood. Current research interests include applications of machine learning to computational physics and connections between group theory and "disentangling" in unsupervised learning.