Information Inequalities.

Entropy inequalities for joint distributions.

There is a long history of entropy inequalities for joint distributions, going back to Shannon's development of his chain rule and subadditivity, Fujishige's discovery of its submodularity, and Han's study of entropic "correlation measures". The closely related Shearer's lemma, first developed for combinatorial applications, was improved and put to powerful uses by Kahn, Friedgut and others. In joint work with Prasad Tetali [MT08], we synthesize, generalize and provide a clear and unified development of all of these inequalities. As applications, we give new bounds for determinants of positive-definite matrices using minors, and also new bounds for counting graph homomorphisms.

Entropy inequalities for sums.

The first lower bound for the entropy of a sum goes back to Shannon's foundational 1948 paper on information theory, although a full proof was only given by Stam. Stam's 1959 paper [Sta59], which was based on his Ph.D. thesis, is still a beautiful testament to the depth of this inequality (now famous as the "entropy power inequality"); it contains, among other things, implications for uncertainty principles, and the first proof of the logarithmic Sobolev inequality for the Gaussian (albeit in a non-canonical form), whose importance would be deeply understood by Gross in the 1970's. Much more recently, Artstein, Ball, Barthe and Naor [ABBN04] proved using sophisticated functional analytic tools an elegant generalization of the entropy power inequality that affirmatively answered a long-open conjecture about the monotonicity of entropy in the central limit theorem. In joint work with Andrew Barron [MB07], we develop an even more general inequality called the "subset-sum entropy power inequality", using a rather transparent proof.

Logarithmic Sobolev inequalities.

Logarithmic Sobolev inequalities are powerful inequalities closely related to concentration of measure, convergence of Markov processes, PDE's, and geometry. In joint work with Ioannis Kontoyiannis [KM06], we gave an alternative and elementary proof of a LSI for discrete compound Poisson distributions that was first proved by Wu [Wu00] in a more general setting. Our main motivation was to extend the use of these inequalities from their typical use in proving exponential or superexponential deviation inequalities, to proving polynomial concentration, which is often all one can expect in the compound Poisson setting. (See here for more.)

References

[ABBN04]    S. Artstein, K. M. Ball, F. Barthe, and A. Naor. Solution of Shannon’s problem on the monotonicity of entropy. J. Amer. Math. Soc., 17(4):975–982, 2004.
[KM06]    I. Kontoyiannis and M. Madiman. Measure concentration for compound Poisson distributions. Elect. Comm. in Probab., 11:45–57, 2006.
[MB07]    M. Madiman and A.R. Barron. Generalized entropy power inequalities and monotonicity properties of information. IEEE Trans. Inform. Theory, 53(7):2317–2329, July 2007.
[MT08]    M. Madiman and P. Tetali. Information Inequalities for Joint Distributions, with Interpretations and Applications. IEEE Trans. Inform. Theory, 2008 (in press).
[Sta59]    A. J. Stam. Some inequalities satisfied by the quantities of information of Fisher and Shannon. Information and Control, 2:101–112, 1959.
[Wu00]    L. Wu. A new modified logarithmic Sobolev inequality for Poisson point processes and several applications. Probab. Theory Related Fields, 118(3):427–438, 2000.

Read the relevant publications for more details, or go back to research summaries.