Course Description:
Study of the pivotal role that information theory plays in illuminating modern statistics.
Topics include the equivalence of data compression and statistical modeling
(from the Shannon, universal coding, and Kolmogorov viewpoints), and its
relationship to the minimum description length principle;
and clean risk bounds for complex estimation scenarios based on an index of
resolvability.
Other possible topics include various aspects of
hypothesis testing (fixed and sequential tests, error exponents, multiple testing);
and the arbitrary sequence approach to on-line learning with
applications to prediction, data compression and portfolio selection.
Last modified on September 5, 2006