Erfan Mirzaei · Italian Institute of Technology
PAC-Bayesian Mini-Course
Abstract
This mini-course offers a gentle introduction to PAC-Bayes theory. We begin with classical PAC bounds for a single predictor from a finite set, then extend to PAC-Bayes, which handles infinite hypothesis spaces and randomized predictors via a KL divergence complexity penalty. Through a foundational bound, we illustrate the core ideas and discuss why PAC-Bayes is especially useful for understanding the generalization of averaged and randomized predictors.
