The No-U-Turn sampler (NUTS) is arguably one of the most important Markov chain Monte Carlo methods out there, but its recursive architecture has long made it challenging to understand. This talk will provide a clearer picture of how NUTS operates and why it performs well in high-dimensional problems. Specifically, I will present a concise proof of NUTS’ reversibility by interpreting it as an auxiliary variable method, in the sense of Section 2.1 of arxiv.org/abs/2404.15253. This novel auxiliary variable approach offers significant flexibility for constructing transition kernels that are reversible with respect to a given target distribution, and includes as special cases Metropolis-Hastings, slice samplers, proximal samplers, and existing locally adaptive HMC methods. Next, I will present the first mixing time guarantee for NUTS, based on couplings and concentration of measure, which aligns well with empirical observations. Specifically, the mixing time of NUTS, when initialized in the concentration region of the canonical Gaussian measure, scales as d^{1/4}, up to log factors, where d is the dimension (see arxiv.org/abs/2410.06978). This scaling is expected to be sharp (see arxiv.org/abs/1001.4460). A key insight in our analysis is that concentration of measure leads to uniformity in NUTS’ locally adapted transitions. We formalize this uniformity by using an extension of a recent coupling framework (see arxiv.org/abs/2308.04634) to a broader class of accept/reject chains. NUTS is then interpreted as one such chain, with the accept chain showing more uniform behavior. This is joint work with Bob Carpenter (Flatiron), Milo Marsden (Stanford), and Stefan Oberdörster (Bonn).
Probability and Combinatorics
Tuesday, December 3, 2024 - 3:30pm
Nawaf Bou-Rabee
Rutgers University
Other Events on This Day
-
Homogenization with critical disorder
Probability and Combinatorics
3:30pm