Safety and Luckiness

Peter’s research currently (2023) mainly focuses on testing and confidence intervals based on e-values. But this is just one aspect of what he calls  safety and Luckiness. The basic idea is to make sure that inference from data is done in – indeed – a safer way. The replicability crisis in the applied sciences provides ample evidence that we often jump to conclusions which simply aren’t justified. The goal is to improve this situation!  The resulting procedures point towards a unified view of statistical inference, in which specific Bayesian, frequentist, likelihoodist and even ‘fiducial’ methods arise as special cases; “prior distributions” become just one aspect of luckiness. Subtopics include:

      • Safe Testing: Hypothesis Testing and Model Choice under Optional Stopping and Optional Continuation – this is the work on e-values and the like cited on the front page. The “luckiness” idea comes out most forcefully in the  E-Posterior (G., Phil. Trans. Roy. Soc. London A). Citing from the abstract: we develop a representation of a decision maker’s uncertainty based on e-variables. Like the Bayesian posterior, this e-posterior allows for making predictions against arbitrary loss functions that may not be specified ex ante. Unlike the Bayesian posterior, it provides risk bounds that have frequentist validity irrespective of prior adequacy: if the e-collection (which plays a role analogous to the Bayesian prior) is chosen badly, the bounds get loose rather than wrong [this encapsulates “luckiness”], making e-posterior-based decision rules safer than Bayesian ones
      • Safe Bayesian Inference: Reparing Bayesian inference under misspecification (when the model is wrong, but useful) – see G. & Van Ommen, 2017G. 2011, G. 2012
      • Safe Probability: working with probability distributions that only capture parts, not all of your domain of interest – see Safe Probability, G. 2018., and Van Ommen, Koolen and G 2016.
      • Luckiness in Learning: quantifying how many data are needed to reach conclusions of a desired quality in machine learning and sequential prediction, with generalized Bayesian and PAC-Bayesian methods that automatically adopt to the inherent ‘easiness’ of the learning task – see De Rooij et al. 2014; Van Erven et al., 2015; G. and Mehta 2017b, Koolen, G. and Van Erven, 2016, G. and Mehta, 2017a and more recently, G., Thomas Steinke, Lydia Zakynthinou. PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes, Proceedings COLT (Conference on Learning Theory), 2021 and Z. Mhammedi, G. and B. Guedj, 2019. PAC-Bayes Unexpected Bernstein Inequality (NeurIPS 2019)