Statistics 680b


Instructor: Marten Wegkamp

Description:   We discuss recent theoretical developments in nonparametric  regression, density estimation and classification.  We introduce some basic empirical process theory and related tricks. Emphasis will be put on universal consistency and model selection. There is no required textbook, although I will use various sources: monographs by Devroye and Lugosi (2001), Van de Geer (2000), combined with articles, and ongoing research.
Below you find an outline of the course.

 Prerequisites:  Stat330/Stat600 or equivalent.


    I. Empirical process theory
        A. Uniform laws of large numbers.
        B. Rates of convergence.
        C. VC classes.

    II. Universal bandwidth selection
        A. Density estimation: Devroye and Lugosi (1996,1997), Wegkamp (1999).
        B. Regression estimation: Hengartner and Wegkamp (2001).

    III. Nonparametric regression
        A. Least squares regression: consistency, rates of convergence.
        B. Universal consistency: Kohler.
        C. Monotone regression: Groeneboom.
        D. Model selection: Baraud (1999) and Wegkamp (2001)

    IV.  Nonparametric classification
        A. Stone’s result: universal consistency.
        B. Structural risk minimization: penalties related to VC-dimension.
        C. Random penalties: Koltchinskii and Panchenko (2001) and Lugosi and Wegkamp (2002).

    V.  Special models: data reduction
        A. Partial linear models.
        B. Direct estimation of the index coefficient in a single index model (Hristache, Juditsky and Spokoiny (2001).