I've recently came across topic known as PAC-Bayesian, but I cannot find a source to read about it. Any article that I came across are talking about its application in a specific area but there is no introduction to what it exactly is.
Asked
Active
Viewed 4,771 times
7
-
1In addition to the aforementioned materials, I recommend recent materials: 1) ICML2019 tutorial A Primer on PAC-Bayesian Learning; 2) A Primer on PAC-Bayesian Learning; 3) NIPS 2017 Workshop (Almost) 50 Shades of Bayesian Learning: PAC-Bayesian trends and insights. – nzw0301 Aug 07 '19 at 16:32
-
I'm lately interested in this topic myself, and have been looking for some good sources as well. The most interesting one I found so far is the overview/tutorial paper by David McAllester titled A PAC-Bayesian Tutorial with A Dropout Bound. – sjm.majewski Jan 06 '14 at 09:07
3 Answers
6
Here are a few quick Google hits...
- PAC-Bayes Analysis: Background and Applications
- Probably Approximately Correct Learning and Vapnik-Chervonenkis Dimension
- Probably approximately correct learning on Wikipedia
- Overview of the Probably Approximately Correct (PAC) Learning Framework
From this last one, a quote:
A more refined, Bayesian extension of the PAC model is explored in [26]. Using the Bayesian approach involves assuming a prior distribution over possible target concepts as well as training instances. Given these distributions, the average error of the hypothesis as a function of training sample size, and even as a function of the particular training sample, can be defined. Also, $1 - \delta$ confidence intervals like those in the PAC model can be defined as well.
[26] $=$ W. Buntine, A Theory of Learning Classification Rules. PhD thesis, University of Technology, Sydney, 1990.
Nick Stauner
- 12,510
2
This paper is a good way to start : https://arxiv.org/pdf/1901.05353.pdf
-
3Please don't provide link-only answers; eventually, the link will break, and then the answer will be useless. Providing a link is fine, but you should at least summarize the essentials - and provide the paper's title and author(s) - in the body of the answer. – jbowman Dec 13 '19 at 14:45
0
A more recent elementary introduction to PAC-Bayes
User-friendly introduction to PAC-Bayes bounds
by Pierre Alquier. It is an 80 page study of this topic.
Cochon
- 1