Course Description:
Study of several key results in probability using ideas and methods from information theory. Topics include entropy and its relationship to Fisher information, the law of large numbers, central limit theorem (normal approximation), law of small numbers (Poisson approximation), large deviations, martingales, Markov chains, and information projection. The approach we take quantitifies the increase in entropy or more generally the drop in information distance from an approximating distribution. Interpretations from statistics, physics, and finance.
Last modified on September 15, 2005