Publications of Andrew Barron

Ph.D. Dissertation:

    A. R. Barron (1985). Logically smooth density estimation. Stanford Univ., Stanford, CA.

Journal Publications

  1. D. Cleveland, A. R. Barron, A. N. Mucciardi (1980). Methods for determining the depth of near-surface defects. Journal of Nondestructive Evaluation, Vol.1, pp.21-36.

  2. A. R. Barron (1985). The strong ergodic theorem for densities: generalized Shannon-McMillan-Breiman theorem. Annals of Probability, Vol.13, pp.1292-1303. (Finalist for the best paper prize by the IEEE Information Theory Society.)

  3. A. R. Barron (1986). Entropy and the central limit theorem. Annals of Probability, Vol.14, pp.336-342. (Finalist for the best paper prize by the IEEE Information Theory Society.)

  4. A. R. Barron (1986). Discussion on Diaconis and Freedman: the consistency of Bayes estimates. Annals of Statistics, Vol.14, pp.26-30.

  5. A. R. Barron and T. M. Cover (1988). A bound on the financial value of information. IEEE Transactions on Information Theory, Vol.34, pp.1097-1100.

  6. A. R. Barron (1989). Uniformly powerful goodness of fit tests. Annals of Statistics, Vol.17, pp.107-124.

  7. B. Clarke and A. R. Barron (1990). Information-theoretic asymptotics of Bayes methods. IEEE Transactions on Information Theory, Vol.IT-38, pp.453-471. (Winner 1992 Browder J. Thompson Memorial Prize award for the best paper in all IEEE journals for authors of age 30 or under at time of submission).

  8. A. R. Barron and X. Xiao (1991). Discussion on Friedman's multivariate adaptive regression. Annals of Statistics, Vol.19, pp.67-82.

  9. A. R. Barron and C. Sheu (1991). Approximation of density functions by sequences of exponential families. Annals of Statistics, Vol.19, pp.1347-1369.

  10. A. R. Barron and T. M. Cover (1991). Minimum complexity density estimation. IEEE Transactions on Information Theory, Vol.IT-37, pp.1034-1054.

  11. A. R. Barron, L. Gyorfi, and E. C. van der Meulen (1992). Distribution estimation consistent in total variation and in two types of information divergence. IEEE Transactions on Information Theory, Vol.IT-38, pp.1437-1454.

  12. A. R. Barron (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, Vol.IT-39, pp.930-944.

  13. A. R. Barron (1994). Approximation and estimation bounds for artificial neural networks. Machine Learning, Vol.14, pp.113-143.

  14. A. R. Barron (1994). Comment on Cheng and Titterington: Neural Networks, A Review from a Statistical Perspective. Statistical Science, Vol.9, No. 1, pp.33-35.

  15. B. Clarke and A. R. Barron (1994). Jeffreys' prior is asymptotically least favorable under entropy risk. Journal of Statistical Planning and Inference, Vol.41, pp.37-60.

  16. Q. Xie and A. R. Barron (1997). Minimax redundancy for the class of memoryless sources. IEEE Transactions on Information Theory, Vol.43, pp.646-657.

  17. Y. Yang and A. R. Barron (1998). An asymptotic property of model selection criteria. IEEE Transactions on Information Theory, Vol.44, pp.117-133.

  18. A. R. Barron, J. Rissanen and B. Yu (1998). The minimum description length principle in coding and modeling. (Invited Paper. Special issue in honor of 50 years since Claude Shannon's seminal work.) IEEE Transactions on Information Theory, Vol.44, pp.2734-2760.

  19. A. R. Barron and N. Hengartner (1998). Information theory and superefficiency. Annals of Statistics, Vol.26, pp.1800-1825.

  20. A. R. Barron, L. Birge and P. Massart (1999). Risk bounds for model selection by penalization. Probability Theory and Related Fields, Vol.113, pp.301-413.

  21. A. R. Barron, M. Schervish, and L. Wasserman (1999). The consistency of posterior distributions in nonparametric problems. Annals of Statistics, Vol.27, pp.536-651.

  22. Y. Yang and A. R. Barron (1999). Information-theoretic determination of minimax rates of convergence. Annals of Statistics, Vol.27, pp.1564-1599.

  23. Q. Xie and A. R. Barron (2000). Asymptotic minimax regret for data compression, gambling, and prediction. IEEE Transaction on Information Theory, Vol.46, pp.431-445.

  24. G. Cheang and A. R. Barron (2000). A Better Approximation for Balls. Journal of Approximation Theory, Vol.104, pp. 183-203.

  25. J. E. Cross and A. R. Barron (2003). Efficient Universal Portfolios for Past Dependent Target Classes. Mathematical Finance, Volume 13, Issue 2, Page 245-276.

  26. O. Johnson and A. R. Barron (2004). Fisher Information Inequalities and the Central Limit Theorem. Probability Theory Related Field , 129, Page 391-409.

  27. F. Liang and A. R. Barron (2004). Exact Minimax Strategies for Predictive Density Estimation, Data Compression, and Model Selection. IEEE Transactions on Information Theory, Vol. 50, Page 2708-2726 .

  28. G. Leung and A. R. Barron (2006). Information theory and mixing least-squares regressions. IEEE Transactions on Information Theory, Vol. 52. no.8, pp.3396-3410.

  29. M. Madiman and A. R. Barron (2007). Generalized entropy power inequalities and monotonicity properties of information. IEEE Transactions on Information Theory, Vol. 53. no.7, pp.2317-2329.

  30. A. R. Barron, A. Cohen, W. Dahmen and R. DeVore (2008). Approximation and learning by greedy algorithms. Annals of Statistics, Vol. 36, pp. 64-94.

  31. C. Huang, G.L.H. Cheang and A. R. Barron. Risk of Penalized Least Squares, Greedy Selection and L1 Penalization for Flexible Function Libraries. Submitted to Annals of Statistics, 2008.

  32. A. M. Kagan, T. Yu, A. R. Barron and M. Madiman. Contributions to the theory of Pitman estimators . Submitted 2008.

  33. J. Takeuchi, T. Kawabata and A. R. Barron (2012). Properties of Jeffreys Mixture for Markov Sources. IEEE Transactions on Information Theory , To Appear, vol.58, no.12.

  34. A. Joseph and A. R. Barron (2012). Least Squares Superposition Codes of Moderate Dictionary Size Are Reliable at Rates up to Capacity. IEEE Transactions on Information Theory , vol.58, no.5. May, pp.2541-2557.

  35. A. Joseph and A. R. Barron. Fast Sparse Superposition Codes have Exponentially Small Error Probability for R < C. Submitted to the IEEE Transactions on Information Theory, July 6, 2012.

Book Chapters and Articles:

  1. A. R. Barron (1984). Predicted squared error: a criterion for automatic model selection. Chapter 4 in Self-Organizing Methods in Modeling, S. J. Farlow (Editor), Marcel Dekker, New York, pp.87-103.

  2. R. L. Barron, A. N. Mucciardi, F. J. Cook, J. N. Craig, and A. R. Barron (1984). Adaptive learning networks. Chapter 2 in Self-Organizing Methods in Modeling, S. J. Farlow (Editor), Marcel Dekker, New York, pp.25-65.

  3. A. R. Barron (1987). Are Bayes rules consistent in information? In Open Problems in Communication and Computation, T. M. Cover and B. Gopinath (Editors), Springer-Verlag, New York, pp.85-91.

  4. A. R. Barron (1991). Complexity regularization with application to artificial neural networks. In Nonparametric Functional Estimation and Related Topics, G. Roussas (Editor), Kluwer Academic Publishers, Boston, MA and Dordrecht, The Netherlands, pp.561-576.

  5. A. R. Barron (1998). Information-theoretic Characterization of Bayes Performance and the Choice of Priors in Parametric and Nonparametric Problems. In Bayesian Statistics 6, J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith (Editors). Oxford University Press. pp.27-52.

  6. J.Q. Li and A.R. Barron (2000). Mixture Density Estimation. In Advances in Neural Information Processing Systems, Vol.12, S.A. Solla, T.K. Leen and K-R. Mueller (Editors). MIT Press, Cambridge, Massachusetts, pp. 279-285.

  7. F. Liang and A.R. Barron (2005). Exact minimax predictive density estimation and MDL. In Advances in Minimum Description Length: Theory and Applications, P. Grunwald, I.J. Myung and M. Pitt (Editors). MIT Press, Cambridge, Massachusetts.

  8. A.R. Barron, C. Huang, J. Q. Li and Xi Luo (2008). MDL Principle, Penalized Likelihood, and Statistical Risk. In Feschrift for Jorma Rissanen. Tampere University Press, Tampere, Finland.

Publications in Conference Proceedings:

  1. A. R. Barron, F. W. van Straten, and R. L. Barron (1977). Adaptive learning network approach to weather forcasting: a summary. Proceedings of the IEEE International Conference on Cybernetics and Society, Washington, DC, September 19-21. Published by IEEE, New York, pp.724-727.

  2. A. R. Barron and R. L. Barron (1988). Statistical learning networks: a unifying view. In Computing Science and Statistics: Proceedings of the 20th Symposium on the Interface, Reston, Virginia, April 20-23. E. Wegman, Ed., Published by the American Statistical Association, Alexandria, Virginia, pp.192-203. (Invited presentation).

  3. A. R. Barron (1989). Statistical properties of artificial neural networks. Proceedings of the IEEE International Conference on Decision and Control, Tampa, Florida, Dec.13-15. pp.280-285,vol.1, Published by IEEE, New York. (Invited presentation).

  4. R. L. Barron, R. L. Cellucci, P. R. Jordan, N. E. Beam, P. Hess, and A. R. Barron (1990). Applications of polynomial neural networks to fault detection, isolation, and estimation (FDIE) and reconfigurable flight control. Proceedings of the National Aerospace Electronics Conference, Dayton, Ohio, May 23-25, pp.507-519, vol.2 (Winner of the best paper prize, 1990 NAECON). Republished in Proceedings 1998 NAECON, pp. 348-360. IEEE

  5. A. R. Barron (1991). Approximation and estimation bounds for artificial neural networks. In Computational Learning Theory: Proceedings of the Fourth Annual ACM Workshop, Santa Cruz, CA, August 5-7. L. Valiant, Ed., Morgan Kaufmann Publishers, Inc., San Mateo, California, pp.243-249. (Honored as one of the four papers invited to appear in expanded form in a special issue of Machine Learning, representing the top presentations at the workshop.)

  6. A. R. Barron (1992). Neural Net Approximation. Proceedings of the 7th Yale Workshop on Adaptive and Learning Systems, May 20-22, K. S. Narendra, Ed., Center for Systems Science, Yale University, pp.69-72.

  7. D. Haussler and A. R. Barron (1993). How well do Bayes methods work for on-line prediction of + or -1 values? Computational Learning and Cognition: Proc. Third NEC Research Symposium, SIAM, Philadelphia, pp.74-100.

  8. J. Takeuchi and A. R. Barron (1998). Asymptotic Minimax Regret by Bayes Mixtures. International Symposium on Information Theory. Cambridge, Ma, August 16-21, 1998.

  9. J. Takeuchi and A. R. Barron (1998). Robustly Minimax Codes for Universal Data Compression. 21st Symposium on Information Theory and Its Applications. Gifu, Japan, December 2-5.

  10. G.H.L. Cheang and A. R. Barron (1999). Estimation with Two Hidden Layer Neural Nets. Proceedings of the 1999 International Joint Conference on Neural Networks (IJCNN), pp.375-378, vol.1, IEEE

  11. G.H.L. Cheang and A. R. Barron (2001). Penalized Least Squares, Model Selection, Convex Hull Classes, and Neural Nets. Proceedings of the 9th European Symposium on Artificial Neural Networks. M. Verleysen, Ed. pp.371-376.

  12. A. R. Barron (2000). Limits of Information, Markov Chains and Projection. IEEE International Symposium on Information Theory. Sorrento, Italy, June 25-30.

  13. M. Madiman and A. R. Barron (2006). The Monotonicity of Information in the Central Limit Theorem and Entropy Power Inequalities. Proceedings of the 2006 IEEE International Symposium on Information Theory. Seattle, Washington, July 2006. pp.1021-1025.

  14. A. R. Barron and Xi Luo (2007). Adaptive Annealing. Proceedings 45th Annual Allerton Conference on Communication, Control, and Computing. Allerton House, UIUC, Illinois. September 26-28. pp.665-673.

  15. A. R. Barron, C. Huang, J. Q. Li, and Xi Luo (2008). MDL, Penalized Likelihood and Statistical Risk. IEEE Information Theory Workshop. Porto, Portugal, May 4-9. pp.247-257.

  16. A. R. Barron and Xi Luo (2008). MDL procedures with l_1 penalty and their statistical risk. First Workshop on Information Theoretic Methods in Science and Engineering. Tampere, Finland, August 18-20, 2008.

  17. M. Madiman, A. R. Barron, A. Kagan, T. Yu (2009). A Model for Pricing Data Bundles by Minimax Risks for Estimation of a Location Parameter. Proceedings of the IEEE Workshop on Information Theory. Volos, Greece, June 10-12,2009, pp.106-109.

  18. A. R. Barron, A. Joseph (2010). Least Squares Superposition Codes of Moderate Dictionary Size, Reliable at Rates up to Capacity. Proc. IEEE International Symposium on Information Theory . Austin, Texas, June 13-18, 2010. pp.275-279.

  19. A. R. Barron, A. Joseph (2010). Towards fast reliable communication at rates near capacity with Gaussian noise. Proc. IEEE International Symposium on Information Theory. Austin, Texas, June 13-18, 2010. pp.315-319

  20. A. R. Barron, A. Joseph (2011). Analysis of fast sparse superposition codes. Proc. IEEE International Symposium on Information Theory. St Petersburg, Russia, August 1-6, 2011. pp.1772-1776.

  21. E. Abbe, A. R. Barron (2011). Polar coding schemes for the AWGN channel. Proc. IEEE International Symposium on Information Theory. St Petersburg, Russia, August 1-6, 2011. pp.194-198.

  22. A. R. Barron and Sanghee Cho (2012). High-rate sparse superposition codes with iteratively optimal estimates. Proc. IEEE International Symposium on Information Theory. Cambridge, MA, July, 2012, pp.120-124.

Technical Reports:

  1. A. R. Barron (1984). Monotonic central limit theorem for densities. Department of Statistics Technical Report #50, Stanford University, Stanford, California.

  2. A. R. Barron (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Department of Statistics Technical Report #7, University of Illinois, Champaign, Illinois.

  3. B. Clarke and A. R. Barron (1990). Entropy risk and the Bayesian central limit theorem. Department of Statistics Technical Report, Purdue University, West Lafayette, Indiana.

  4. A. R. Barron (1991). Information Theory and Martingales. Presented at 1991 IEEE International Symposium on Information Theory (recent results session), Budapest, Hungary, June23-29.

  5. A. R. Barron (1997). Information Theory in Probability, Statistics, Learning, and Neural Nets. Department of Statistics. Yale University. Working paper distributed at plenary presentation of the Tenth Annual ACM Workshop on Computational Learning Theory.

  6. J. Takeuchi and A. R. Barron (1997). Asymptotically minimax regret for exponential and curved exponential families. Fourteen page summary for presentation at the 1998 International Symposium on Information Theory, Cambridge, Massachusetts.

  7. A. R. Barron (1999). Limits of Information, Markov Chains, and Projection. Eight page summary of presentation at the 2000 IEEE International Symposium on Information Theory, Sorrento, Italy.

  8. F. Liang and A. R. Barron (2001). Exact Minimax Strategies for Predictive Density Estimation, Data Compression and Model Selection. Seven page summary of presentation at the 2002 IEEE International Symposium on Information Theory, Lausanne, Switzerland.

  9. A. R. Barron and A. Joseph (2010). Least Squares Superposition Codes of Moderate Dictionary Size, Reliable at Rates up to Capacity. This is the version submitted to the IEEE Transactions on Information Theory, June 2010. Final version available in publication list above.

  10. A. R. Barron and A. Joseph (2011). Sparse Superposition Codes are Fast and Reliable at Rates Approaching Capacity with Gaussian Noise. June 10, 2011. This is an expanded version of the paper. Later in 2012 a shorter version was submitted to the IEEE Transactions on Information Theory, see the publication list above.

A Selection of Seminar Presentation Files (pdf format); to view on your computer or to project on a screen):

  1. Information Inequalities and the Central Limit Theorem. Presented at Boston University, March 1, 2007. Similar presentations at IBM T.J. Watson Research Center, Sept 22, 2006 and the IEEE International Symposium on Information Theory, June 2006, The Monotonicity of Information in the Central Limit Theorem and Entropy Power Inequalities.

  2. Fast and Accurate L1 Penalized Estimations. Presented at Rugters University, December 12, 2007. Similar presentation: A Simple Algorithm for L1-Penalized Least Squares, U.C. Berkeley, October 4, 2007.

  3. Principles of Information Theory in Probability and Statistics. Presented at the Elements of Information Theory Workshop: CoverFest. On the Occasion of the 70th birthday of Tom Cover. Stanford University, May 16, 2008.

  4. MDL, Penalized Likelihood and Statistical Risk. Presented at the Information Theory Workshop, Porto, Portugal, May 8. Festschrift on the occasion of the 75th birthday of Jorma Rissanen. Similar presentations with updates for the regression case at the Workshop on Information Theory Methods in Science and Engineering, Tampere Finland, August 19, 2008 and the Information and Communication Conference, Renyi Institute, Budapest, August 25-28, 2008, on the occasion of the 70th birthday of Imre Csiszar: MDL Procedures with L_1 Penalty and their Statistical Risk

  5. Adaptive Annealing. Presentation at the Allerton Conference on Communication, Control, and Computing. September 27, 2007.

  6. Information Theory and Flexible High-Dimensional Non-Linear Function Estimation. Presented at the Info-Metrics Institute Workshop, American University, Wash, DC, November 12, 2011. Similar presentation at Harvard Univ, Dept Statistics, Oct.2011. Overview of several useful results for high-dimensional function estimation. Disclaimer: The proposed solution on page 19 to the differential equation for Adaptive Annealing is problematic due to discontinuity of the gradient at the origin.

  7. Least Squares Superposition Codes of Moderate Dictionary Size, Reliable at Rates up to Capacity (presented by Antony Joseph). Presentation at the IEEE International Symposium on Information Theory, Austin, TX, June 13-18,2010.

  8. Information and Statistics and Practical Achievement of Shannon Capacity. Three-part Invited Tutorial Presentation at the Workshop on Information Theory and its Applications, U.C. San Diego, February 9, 2011.

  9. Analysis of Fast Sparse Superposition Codes. Presentation at the IEEE International Symposium on Information Theory, St. Petersburg, Russia, August 5, 2011. Adds details of distribution analysis not in the earlier presentation "Toward Fast Reliable Communication, at Rates Near Capacity with Gaussian Noise," IEEE International Symposium on Information Theory, Austin, TX, June 18, 2010. Similar Presentations: "Communication by Regression: Practical Achievement of Shannon Capacity," at Workshop Infusing Statistics and Engineering, Harvard University, June 5-6, 2011. "Sparse Superposition Codes: low complexity and exponentially small error probability at all rates below capacity," Workshop on Information Theory Methods in Science and Engineering, Helsinki, Finland, August 8, 2011.

  10. High-Rate Sparse Superposition Codes with Iteratively Optimal Estimates Presentation at the IEEE International Symposium on Information Theory, Cambridge, MA, July 2, 2012. Similar presentation: Communication by Regression. Rice University, March 12, 2012. Also at University of Michigan March 9, 2012.