S&DS677: Topics in High-Dimensional Statistics and Information Theory
Lecture schedule
Except otherwise noted, the reading materials below is from
| topics | reading | notes and references |
| Jan 16 | Introduction (slides), basics of statistical decision theory | Sec 28.1-28.3 | |
| Jan 23 | Minimax theorem and duality, tensorization of experiments, Anderson's lemma | Sec 28.4-28.6 | |
| Jan 30 | f-divergence I: definitions, properties, examples, hypothesis testing | Sec 7.1-7.3 | |
| Feb 6 | f-divergence II: inequalties and joint range, variational representation, GAN, information bound | Sec 7.4-7.5, 29.1-29.2 | |
| Feb 13 | f_divergence III: chain rule and tensorization, Donsker-Varadhan and Gibbs | Sec 4.3-4.4 | Jayram's chainrule for squared Hellinger |
| Feb 20 | f_divergence IV: PAC Bayes and applications, variational inference | Sec 4.8, 7.13 | Applications to smallest singular value: Oliveira and Moutada;
dim-free concentration on sample covariance matrix: Zhivotovskiy |
| Feb 27 | Mutual information method | Sec 30 | |
| Mar 5 | Le Cam's method (two-point), Assoud's lemma (hypercube), Fano's method (packing) | Sec 31 | |
| Mar 12 | Spring break I 😄 | | |
| Mar 19 | Spring break II 😂 | | |
| Mar 26 | Metric entropy I | Sec 27.1-2, 27.4 | |
| Apr 2 | Metric entropy II | Sec 27.3, 27.5 | |
| Apr 9 | Entropic upper bound for statistical estimation (Yang-Barron, Le Cam-Birgé, Yatracos) | Sec 32 | |
| Apr 16 | functional estimation and HT I: goodness-of-fit testing, second moment method and truncation | Lec 23 of these lecture notes | See (3.7) and the surrounding discussion in this paper by Gerber-Polyanskiy for a tentalizing conjecture of the sample complexity of goodness of fit testing versus estimation and many examples |
| Apr 23 | functional estimation and HT II: sparse detection, uniformity testing | Lec 24 of these lecture notes | Survey by Cannone |
|
|
|