2

Are there any contemporary monographs on minimax theory$^1$, and if so, what are they?

So far, I have seen minimax theory given some treatment in Theory of Point Estimation by Lehmann and Casella (1998), and a more extensive treatment in Introduction to Nonparametric Estimation by Tsybakov (2009).

With the reservation that I only have terse introductory knowledge of this subject, and that I am at a stage where I don't know what I don't know; it feels like there is a gulf between what is covered on this from the statistical decision theory angle in Lehmann and Casella, compared with more 'modern' papers in statistical machine learning.

To elaborate a little further on what I mean by more 'modern', many of the papers I have had a passing glance at seem to use techniques and results in their arguments which seem alien to what I would call 'classical statistical decision theory', and often these results are seemingly obscure inequalities or theorems that can only be found in other papers, but not monographs I know of.

$^1$ In the sense of 'minimax estimator' and 'minimaxity' in statistical decision theory and statistical machine learning rather than the referent in artifical intelligence and game theory.

microhaus
  • 2,186
  • 1
  • 4
  • 17

1 Answers1

1

I found these notes by Larry Wasserman really helpful for learning the basic theory.

I used these lecture notes by Yihong Wu in a class also.

Wainwright also has a good chapter on minimax theory.

There is still definitely a rift between these resources and some modern papers. I tend to notice Tsybakov (2009) being cited in modern papers.

Edit: I was recently reminded of this book by Gine and Nickl it is very similar to Tsbakov (2009) although slightly more modern.

I think it really stands out for the chapter on empirical processes theory which is extensive (like not far off from Van der Vaart and Wellner level extensive). However, it also has a chapter devoted to studying the minimax properties of infinite-dimensional estimators. The nice thing is that a lot of the minimax results for nonparametric tests and estimators rely on the empirical process and functional approximation results covered earlier so it is nice to have all that together like in Tsybakov (2009).

References in the event that the hyperlinks are no longer active:

Tibshirani, R., Wasserman, L. (2017). Lecture notes: Minimax theory. Statistical Machine Learning 10/36-702 Spring 2017, Carnegie Mellon University. Retrieved from https://www.stat.cmu.edu/~ryantibs/statml/lectures/minimax.pdf

Wu, Y. (2020). Lecture notes: Information-theoretic methods for high-dimensional statistics. Information-theoretic methods for high-dimensional statistics ECE598YW Spring 2016, Yale University. Retrieved from http://www.stat.yale.edu/~yw562/teaching/it-stats.pdf

Wainwright, M. (2019). High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge University Press. https://doi.org/10.1017/9781108627771

Tsybakov, T. (2009). Introduction to Nonparametric Estimation. Springer, New York, NY. https://doi.org/10.1007/b13794

Giné, E., & Nickl, R. (2015). Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge University Press. https://doi.org/10.1017/CBO9781107337862

microhaus
  • 2,186
  • 1
  • 4
  • 17
Ariel
  • 2,273
  • 2
  • 23
  • +1. Thank you for this, and for the instructive edits. Looking through the Wu notes has allowed me to better isolate what I feel is one component of how the more modern papers in statistical machine learning have evolved, and that is the use of information theoretic tools. I have come across some of the tools in empirical processes through introductory statistical learning theory, but the van der Vaart and Wellner book is beyond me, at least until I'm more fluent in analysis and functional analysis. – microhaus Jun 05 '21 at 21:29
  • Glad I could help! Yeah, the Wu notes were really helpful for me also and as someone who took functional analysis, I still find Van der Vaart and Wellner very challenging so I completely understand that! – Ariel Jun 05 '21 at 21:42
  • 1
    As you've already done quite a bit of work in posting hyperlinks, would you mind if I additionally edited your answer only to include additional AP style references to each of the references you have linked, in the event that these links go dead? – microhaus Jun 05 '21 at 21:44
  • 1
    Oh yeah absolutely, that is a good call! – Ariel Jun 05 '21 at 21:45