An Introduction to Stein's Method for Machine Learning: Foundations, Applications, and Geometry
Abstract
This mini-course introduces Stein's method, connecting its foundations in probability with modern methods for sampling and inference. We will begin with Charles Stein's original paper on normal approximation, then explore Barbour's generator approach to deriving Stein operators for other distributions. Along the way, we will discuss theoretical applications, including derivation or improvement of concentration inequalities and mention other interesting results like the Fourth Moment Theorem. In the second part, we will use the Langevin Stein operator to derive Stein variational gradient descent (SVGD), a nonparametric approach to variational inference that can also be viewed as a deterministic sampling method. Finally, we will interpret SVGD through the framework of gradient flows, revealing the geometric structure behind its particle dynamics.