Contents of Joe Chang's Stochastic Processes notes

• Chapter 1: Markov chains
1. What is a Markov chain? How to simulate one.
2. The Markov property.
3. How matrix multiplication gets into the picture.
4. Statement of the Basic Limit Theorem about convergence to stationarity. A motivating example shows how complicated random objects can be generated using Markov chains.
5. Stationary distributions, with examples. Probability flux.
6. Other concepts from the Basic Limit Theorem: irreducibility, periodicity, and recurrence. An interesting classical example: recurrence or transience of random walks.
7. Introduces the idea of coupling.
8. Uses coupling to prove the Basic Limit Theorem.
9. A Strong Law of Large Numbers for Markov chains.
• Chapter 2: More on Markov chains, Examples and Applications
1. Branching processes.
2. Time reversibility.
3. Application of time reversibility: a tandem queue model.
4. The Metropolis method.
5. Simulated annealing.
6. Ergodicity concepts for time-inhomogeneous Markov chains.
7. Proof of the main theorem of simulated annealing.
8. Card shuffling: speed of convergence to stationarity.
• Chapter 3: Markov Random Fields and Hidden Markov Models
1. MRF's on graphs and HMM's.
2. Bayesian framework
3. The Hammersley-Clifford Theorem and Gibbs Distributions
4. Phase transitions in the Ising model.
5. Likelihood and data analysis in hidden Markov chains.
6. Simulated MRF's, the Gibbs' sampler.
• Chapter 4: Martingales
1. Where did the name come from?
2. Definition and examples.
3. Optional sampling.
4. Stochastic integrals and option pricing in discrete time.
5. Martingale convergence.
6. Stochastic approximation.
• Chapter 5: Brownian motion
1. The definition and some simple properties.
2. Visualizing Brownian motion. Discussion and demystification of some strange and scary pathologies.
3. The reflection principle.
4. Conditional distribution of Brownian motion at some point in time, given observed values at some other times.
5. Existence of Brownian motion. How to construct Brownian motion from familiar objects.
6. Brownian bridge. Application to testing for unformity.
7. A boundary crossing problem solved in two ways: differential equations and martingales.
8. Discussion of some issues about probability spaces and modeling.
• Chapter 6: Diffusions and Stochastic Calculus