
Publications and Presentations of Andrew Barron
Ph.D. Dissertation:
Journal Publications:
 D. Cleveland, A. R. Barron, A. N. Mucciardi (1980). Methods for determining the depth of nearsurface defects. Journal of Nondestructive Evaluation, Vol.1, pp.2136.






B. Clarke and A. R. Barron (1990). Informationtheoretic asymptotics of Bayes methods. IEEE Transactions on Information Theory, Vol.IT38, pp.453471. (Winner 1992 Browder J. Thompson Memorial Prize award for the best paper in all IEEE journals for authors of age 30 or under at time of submission).


































Book Chapters:


R. L. Barron, A. N. Mucciardi, F. J. Cook, J. N. Craig, and A. R. Barron (1984). Adaptive learning networks. Chapter 2 in SelfOrganizing Methods in Modeling, S. J. Farlow (Editor), Marcel Dekker, New York, pp.2565.




J.Q. Li and A.R. Barron (2000). Mixture Density Estimation. In Advances in Neural Information Processing Systems, Vol.12, S.A. Solla, T.K. Leen and KR. Mueller (Editors). MIT Press, Cambridge, Massachusetts, pp. 279285.


A.R. Barron, C. Huang, J. Q. Li and Xi Luo (2008). MDL Principle, Penalized Likelihood, and Statistical Risk. In Festschrift for Jorma Rissanen.
Peter Grunwald, Petri Myllymaki, Ioan Tabus, Marcelo Weinberger & Bin Yu (Editors). Tampere International Center for Signal Processing. TICSP series, #38. Tampere University of Technology, Tampere, Finland.
Publications in Conference Proceedings: (4 to 27 pages)
 A. R. Barron, F. W. van Straten, and R. L. Barron (1977). Adaptive learning network approach to weather forcasting: a summary. Proceedings of the IEEE International Conference on Cybernetics and Society, Washington, DC, September 1921. Published by IEEE, New York, pp.724727.

A. R. Barron and R. L. Barron (1988). Statistical learning networks: a unifying view. In Computing Science and Statistics: Proceedings of the 20th Symposium on the Interface, Reston, Virginia, April 2023. E. Wegman, Ed., Published by the American Statistical Association, Alexandria, Virginia, pp.192203. (Invited presentation).


R. L. Barron, R. L. Cellucci, P. R. Jordan, N. E. Beam, P. Hess, and A. R. Barron (1990). Applications of polynomial neural networks to fault detection, isolation, and estimation (FDIE) and reconfigurable flight control. Proceedings of the National Aerospace Electronics Conference, Dayton, Ohio, May 2325, pp.507519, vol.2 (Winner of the best paper prize, 1990 NAECON). Republished in Proceedings 1998 NAECON, pp. 348360. IEEE

A. R. Barron (1991). Approximation and estimation bounds for artificial neural networks. In Computational Learning Theory: Proceedings of the Fourth Annual ACM Workshop, Santa Cruz, CA, August 57. L. Valiant, Ed., Morgan Kaufmann Publishers, Inc., San Mateo, California, pp.243249. (Honored as one of the four papers that appeared by invitation in expanded form in a special issue of Machine Learning, representing the top presentations at the workshop.)

A. R. Barron (1992). Neural Net Approximation. Proceedings of the 7th Yale Workshop on Adaptive and Learning Systems, May 2022, K. S. Narendra (Editor), Center for Systems Science, Yale University, pp. 6972.

D. Haussler and A. R. Barron (1993). How well do Bayes methods work for online prediction of + or 1 values? Computational Learning and Cognition: Proc. Third NEC Research Symposium, SIAM, Philadelphia, pp. 74101.






J. Takeuchi and A. R. Barron (2001). Properties of Jeffreys mixture for Markov sources. Proc. Workshop on Information Based Induction Sciences (IBIS), pp. 327333.




A. R. Barron and Xi Luo (2007). Adaptive Annealing. Proceedings 45th Annual Allerton Conference on Communication, Control, and Computing. Allerton House, UIUC, Illinois. September 2628. pp.665673.


















Newsletter Article:

R. Venkataramanan, S. Tatikonda, A. Barron (2016). Sparse Regression Codes.
IEEE Information Theory Society Newsletter.December 2016, pp. 715. [Based on ISIT Tutorial by R. Venkataramanan and A. Barron, Barcelona, July 2016.]
Patents:
Technical Reports: (with details not in subsequent publications)
 A. R. Barron (1984). Monotonic central limit theorem for densities. Department of Statistics Technical Report #50, Stanford University, Stanford, California.

A. R. Barron (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Department of Statistics Technical Report #7, University of Illinois, Champaign, Illinois.

B. Clarke and A. R. Barron (1990). Entropy risk and the Bayesian central limit theorem. Department of Statistics Technical Report, Purdue University, West Lafayette, Indiana.

A. R. Barron (1991). Information theory and martingales. Presented at 1991 IEEE International Symposium on Information Theory (recent results session), Budapest, Hungary, June 2329.

A. R. Barron, Y. Yang and B. Yu (1994). Asymptotically optimal function estimation by minimum complexity criteria. Seven page
original submission. Presented at the IEEE International Symposium on Information Theory, Trondheim, Norway, June 27  July 1.

A. R. Barron (1997). Information theory in probability, statistics, learning, and neural nets. Department of Statistics. Yale University. Working paper distributed at plenary presentation of the Tenth Annual ACM Workshop on Computational Learning Theory.

J.I. Takeuchi and A. R. Barron (1997). Asymptotically minimax regret for exponential and curved exponential families. Fourteen page original. Presentation at the 1998 International Symposium on Information Theory, Cambridge, Massachusetts.

A. R. Barron (1999). Limits of information, Markov chains, and projection. Eight page original submission.Ppresentation at the 2000 IEEE International Symposium on Information Theory, Sorrento, Italy.

J. Yu and A. R. Barron (2003). Maximal compounded wealth for portfolios of stocks and options. Working paper, some of which was presented at the Workshop on Complexity and Inference, DIMACS, Rutgers University, June 25.

W. Qiu and A. R. Barron (2007). A maximum wealth asset index and mixture strategies for universal portfolios on subsets of stocks. See also the Yale Dissertation of Wei (David) Qiu

C. Huang, G.L.H. Cheang and A. R. Barron (2008). Risk of Penalized Least Squares, Greedy Selection and L1 Penalization for Flexible Function Libraries. [Too long for journal publication, yet still has some of our best results.]

A. R. Barron and A. Joseph (2011). Sparse Superposition Codes are Fast and Reliable at Rates Approaching Capacity with Gaussian Noise. June 10, 2011. [This is an expanded version. A shorter version was completed in 2012, appearing 2014 in the IEEE Transactions on Information Theory, per the publication list above.]

S. Chatterjee and A. R. Barron (2014). Information Theory of Penalized Likelihoods and its Statistical Implications. arXiv:1401.6714v2, April 27. [A shorter version is in ISIT 2014.]
A Selection of Seminar Presentation Files (pdf format); to view on a computer or to project on a screen):




MDL, Penalized Likelihood and Statistical Risk. Presented at the Information Theory Workshop, Porto, Portugal, May 8. Festschrift on the occasion of the 75th birthday of Jorma Rissanen. Similar presentations with updates for the regression case at the Workshop on Information Theory Methods in Science and Engineering, Tampere Finland, August 19, 2008 and the Information and Communication Conference, Renyi Institute, Budapest, August 2528, 2008, on the occasion of the 70th birthday of Imre Csiszar:
MDL Procedures with L_1 Penalty and their Statistical Risk

Adaptive Annealing. Presentation at the Allerton Conference on Communication, Control, and Computing. September 27, 2007.

Information Theory and Flexible HighDimensional NonLinear Function Estimation. Presented at the InfoMetrics Institute Workshop, American University, Wash, DC, November 12, 2011. Similar presentation at Harvard Univ, Dept Statistics, Oct.2011. Overview of several useful results for highdimensional function estimation. Disclaimer: The proposed solution on page 19 to the differential equation for Adaptive Annealing is problematic due to discontinuity of the gradient at the origin.



Analysis of Fast Sparse Superposition Codes. Presentation at the IEEE International Symposium on Information Theory, St. Petersburg, Russia, August 5, 2011. Adds details of distribution analysis not in the earlier presentation "Toward Fast Reliable Communication, at Rates Near Capacity with Gaussian Noise," IEEE International Symposium on Information Theory, Austin, TX, June 18, 2010. Similar Presentations: "Communication by Regression: Practical Achievement of Shannon Capacity," at Workshop Infusing Statistics and Engineering, Harvard University, June 56, 2011. "Sparse Superposition Codes: low complexity and exponentially small error probability at all rates below capacity," Workshop on Information Theory Methods in Science and Engineering, Helsinki, Finland, August 8, 2011.








Other Conference Presentations (19832008): (proceedings containing not more than 1 page abstracts). A number of subsequent presentations are described in the links above.]
Invited Departmental Seminar Presentations (19852007): (These had short abstract announcements).
[Some subsequent presentations in links above.]
 Purdue University, Joint Statistics Colloquium, October 3, 1985. Topic: Entropy and the central limit theorem.
 Michigan State University, Department of Statistics and Probability, January 28, 1986. Topic: Generalized ShannonMcMillanBreiman theorem.
 University of Chicago, Department of Statistics, October 20, 1986. Topic: Uniformly powerful tests.
 University of Virginia, Department of Mathematics, March 5, 1987. Topic: Convergence of Bayes estimators of probability density functions.
 Stanford University, Department of Statistics, October 20, 1987. Topic: Convergence of Bayes estimators of probability density functions.
 University of Chicago, Department of Statistics, March 7, 1988. Topic: Convergence of Bayes estimators of probability density functions.
 McGill University, Joint Statistics Seminar for Montreal universities, March 31, 1988. Topic: Approximation of densities by sequences of exponential families.
 Dupont Research Center, Dover, Delaware, April 26, 1988. Topic: Statistical learning networks.
 IBM T. J. Watson Research Center, Yorktown Heights, New York, August 10, 1988. Topic: Statistical learning networks.
 Stanford University, Information Systems Laboratory, November 3, 1988. Topic: Minimum complexity density estimation.
 IBM Technical Education Center, Thornwood, New York, January 1112, 1989. Statistical learning networks. In the short course on Knowledge Acquisition from Data.
 Cornell University, Department of Economics and Program of Statistics (Cohosts), February 1, 1989. Topic: Convergence of Bayes estimators of probability density functions.
 Purdue University, Department of Statistics, September 7, 1989. Topic: Minimum complexity density estimation.
 University of Lowell, Massachusetts, Joint Seminar, Department of Mathematics and Department of Electrical Engineering, March14, 1990. Topic: Statistical properties of polynomial networks and other artificial neural networks.
 Carnegie Mellon University, Department of Statistics, April 4, 1990. Topic: Statistical properties of polynomial networks and other artificial neural networks.
 University of Chicago, Department of Statistics, October 15, 1990. Topic: Statistical properties of artificial neural networks.
 University of California, San Diego, Department of Mathematics, January 7, 1991. Topic: Approximation bounds for artificial neural networks.
 University of California, San Diego, Department of Mathematics, January 8, 1991. Topic: Complexity regularization for nonlinear model selection.
 Siemens Corporation, Princeton, New Jersey, February 28, 1991. Topic: Universal approximation bounds for superpositions of a sigmoidal function.
 University of Wisconsin, Department of Statistics, April 3, 1991. Topic: Complexity regularization for nonlinear model selection.
 University of Wisconsin, Department of Mathematics, April 4, 1991. Topic: Approximation bounds for artificial neural networks.
 Technical University of Budapest, Department of Electrical Engineering, July 2, 1991. Topic: Universal approximation bounds for superpositions of a sigmoidal function.
 Mathematical Sciences Research Institute, Berkeley, California, September 25, 1991. Topic: Empirical process bounds for artificial neural networks.
 Stanford University, Department of Statistics, October 15, 1991. Topic: Approximation and estimation bounds for artificial neural networks.
 University of California, Santa Cruz, Department of Computer and Information Sciences, October 17, 1991. Topic: Computationally efficient approximation and estimation of functions using artificial neural networks.
 Yale University, Department of Statistics, January 13, 1992. Topic: Neural network estimation.
 University of Virginia, Department of Electrical Engineering, Eminent Speaker Series, February 21, 1992. Topic: Estimation of functions of several variables  neural networks, Fourier decomposition, and Bayes methods.
 North Carolina State University, Department of Statistics, February 29, 1992. Topic: Estimation of functions of several variables  neural networks, Fourier decomposition, and Bayes methods.
 Cornell University, Center for Applied Mathematics, March 6, 1992. Topic: Estimation of functions of several variables  neural networks, Fourier decomposition, and Bayes methods.
 University of North Carolina, Department of Statistics, March 30, 1992. Topic: Estimation of functions of several variables  neural networks, Fourier decomposition, and Bayes methods.
 University of Joenesu, Finland, Department of Statistics, April 9, 1992. Topic: Introduction to artificial neural networks.
 University of Paris VI, Department of Statistics, April 15, 1992, and University of Paris, Orsay, Department of Statistics, April16, 1992. Topic: Estimation of functions of several variables  neural networks, Fourier decomposition, and Bayes methods.
 University of Paris VI, Department of Statistics, April 22, 1992, and University of Paris, Orsay, Department of Statistics, April23, 1992. Topic: Performance bounds for complexitybased model selection.
 Princeton University, Department of Electrical Engineering, May 14, 1992. Topic: Overview of approximation results for sigmoidal networks.
 University of Massachusetts at Lowell, Joint Seminar, Department of Mathematics and Department of Electrical Engineering, October 21, 1992. Topic: Statistical accuracy of neural nets.
 University of Tokyo, Japan, Department of Information, Physics and Engineering, March 1993. Topic: Information theory and model selection.
 University of Paris VI, Department of Statistics, May 1993. Topic: Optimal rate properties of minimum complexity estimation.
 University of Paris VI, Department of Statistics, May 1993. Topic: Informationtheoretic proof of martingale convergence.
 University of Pennsylvania, Wharton School, October 21, 1993. Topic: Neural networks and statistics.
 Massachusetts Institute of Technology, Center for Biological and Computational Learning, October 27, 1993. Topic: Neural networks and statistics.
 Rutgers University, Department of Statistics, October 5, 1994, Topic: Statistical accuracy of neural nets.
 University of South Carolina, Department of Mathematics, Spring 1995. Topic: Neural net approximation.
 Carnegie Mellon University, Department of Statistics, Fall 1995. Topic: Information risk and superefficiency.
 Massachusetts Institute of Technology, Department of Applied Mathematics, March 1996. Topic: Consistent and uniformly consistent classification.
 Columbia University, Department of Statistics, Fall 1996. Topic: Consistency of posterior distributions in nonparametric problems.
 Northeastern University, Joint Mathematics Colloquium with MIT, Harvard, and Brandiess, February 27, 1997. Topic: Information theory in probability and statistics.
 Iowa State University, Department of Statistics, March 28, 1997. Topic: Information theory in probability and statistics: The fundamental role of Kullback divergence.
 Washington University, St. Louis, Department of Electrical Engineering, Center for Imaging Systems, April 16, 1997. Topic: Universal data compression, prediction, and gambling.
 Massachusetts Institute of Technology, LIDS Colloquium, May 5, 1998. Topic: Simple universal portfolio selection.
 University of California, Santa Cruz, Baskin Center for Computer Engineering, October 1998. Topic: Approximation bounds for Gaussian mixtures.
 Lucent, Bell Laboratories, Murray Hill, New Jersey, March 1999. Topic: Approximation and estimation bounds for mixture density estimation.
 Stanford University, Department of Statistics, Probability Seminar, May 24, 1999. Topic: Information, martingales, Markov chains, convex projections, and the CLT.
 Stanford University, Department of Statistics, Statistics Seminar, May 25, 1999. Topic: Mixture density estimation.
 Rice University, Departments of Statistics and Electrical Engineering, November 4, 2000. Topics: Information theory and statistics  best invariant predictive density estimators.
 University of Chicago, Department of Statistics, November 21, 2000. Topics: Information theory and statistics  best invariant predictive density estimators.
 Yale University, Department of Computer Science, Alan J. Perlis Seminar, April 26, 2001. Topic: Neural nets, Gaussian mixtures, and statistical information theory.
 Brown University, Department of Applied Mathematics, May 9, 2001. Topic: I do not recall.
 University of Massachusetts at Lowell, Department of Mathematics, September 19, 2001. Topic: Mixture density estimation.
 University of California at Los Angeles, Department of Statistics, May 21, 2002. Topic: Nonlinear approximation, estimation, and neural nets (I do not recall the specific title).
 Columbia University, Department of Statistics, October 28, 2002. Topic: Information inequalities in probability and statistics.
 University of Georgia, Department of Statistics, November 26, 2002. Topic: Information inequalities in probability and statistics.
 University of Pennsylvania, Wharton School, October 29, 2003. Topic: Portfolio estimation for compounding wealth.
 University of North Carolina (in conjunction with Duke University), Departments of Statistics, November 3, 2004. Topic: Risk assessment and advantages of model mixing for regression.
 South Carolina, Department of Mathematics, April 7, 2005. IMI Distinquished Lecture. Topic: Statistical theory for nonlinear function approximation: neural nets, mixture models, and adaptive kernel machines.
 Helsinki University and Helsinki Institute of Information Technology, Helsinki, Finland. August 2225, 2005. Two talks: (1) Statistical foundations and analysis of the minimum description length principle. (2) Consequences of MDL for neural nets and Gaussian mixtures.
 Princeton University, Department of Operations Research and Financial Engineering, October 4, 2005. Topic: Statistical perspectives on growth rate optimal portfolio estimation.
 IBM Research Laboratories, Yorktown Heights, September 22, 2006. Topic: Generalized entropy power inequalities and the central limit theorem.
 Purdue University, Department of Computer Science, February 26, 2007. Prestige Lecture Series on the Science of Information. Topic: The interplay of information theory, probability, and statistics.
 University of Illinois, Joint Seminar, Department of Statistics and Department of Electrical and Computer Engineering, February 27, 2007. Prestige Conference Series. Topic: The interplay of information theory and probability.
 Boston University, Department of Statistics, March 1, 2007. Prestige Conference Series. Topic: Information inequalities and the central limit theorem.
 University of California at Berkeley, Department of Computer Science, October 4, 2007. Topic: Fast and accurate greedy algorithm for L1 penalized least squares. Primarily presented by Cong Huang.
 Rutgers University, Department of Statistics, December 12, 2007. Topic: Fast and accurate L1 penalized least squares. Copresented with Cong Huang.
Ph.D. Dissertations Supervised:



Yuhong Yang (1996). Minimax
Optimal Density Estimation. Yale University. [Was Assistant Professor at Iowa State University;
Now Professor at University of Minnesota, Department of Statistics.]

Qun (Trent) Xie (1997). Minimax Coding
and Prediction. Yale University. [Was at GE Capital, Inc., Fairfield, CT. Then an Assistant
Professor at Tsinghua Univ.]




Feng Liang (2002). Exact
Minimax Predictive Density Estimation. Yale University. [Was Assistant Professor Department of Statistics,
Duke University. Now Associate Professor, Department of Statistics, University of Illinois at UrbanaChampaign]









