1.1 Probability Spaces . . . .. . . . . . . 1

1.2 Distributions . . . . . . . .. . . . . . . . . 8

1.3 Random Variables . . . . . . . . . . . . 12

1.4 Integration . . . . . . . . .. . . . . . 15

1.5 Properties of the Integral . . .. . . . . . 21

1.6 Expected Value . . . . . . .. . . . . 24

1.6.1 Inequalities . . . . . . . . .. . . . 24

1.6.2 Integration to the Limit . . . . . . 25

1.6.3 Computing Expected Values . . . . . . 27

1.7 Product Measures, Fubini’s Theorem . . . .. 31

2.1 Independence . . . . . . . . .. . . . 37

2.1.1 Sufficient Conditions for Independence . . . .. . . 38

2.1.2 Independence, Distribution, and Expectation . . .. . . 41

2.1.3 Sums of Independent Random Variables . . . . . . . . 42

2.1.4 Constructing Independent Random Variables . . . . 45

2.2 Weak Laws of Large Numbers . . . . . . . . . . 47

3.1 The De Moivre-Laplace Theorem . . .. . . . 81

3.2 Weak Convergence . . . . .. . . . . 83

3.2.1 Examples . . . . . . . . . . . 83

3.2.2 Theory . . . . . . . . . . . . . . 86

3.3 Characteristic Functions . . . . .. . . . . 91

3.3.1 Definition, Inversion Formula . . . . . . . . . 91

3.3.2 Weak Convergence . . . . . . . . . . . 97

3.3.3 Moments and Derivatives . . . . . . . 98

3.3.4 Polya’s Criterion* . . . . .. . . 101

iii

3.3.5 The Moment Problem* . . . . .. . . . 103

3.4 Central Limit Theorems . . . . 106

3.4.1 i.i.d. Sequences . . .. 106

3.4.2 Triangular Arrays . . . . .. . . 110

3.4.3 Prime Divisors (Erd¨os-Kac)* . . . .. . 114

3.4.4 Rates of Convergence (Berry-Esseen)* . . .118

3.5 Local Limit Theorems* . . . . 121

3.6 Poisson Convergence . . . .. 126

3.6.1 The Basic Limit Theorem . . . . . . 126

3.6.2 Two Examples with Dependence . . .. 130

3.6.3 Poisson Processes . . .. 132

3.7 Stable Laws* . . . . . . . 135

3.8 Infinitely Divisible Distributions* . . .. 144

4.1 Stopping Times . . . . . 153

4.2 Recurrence . . . . . . . . 162

4.3 Visits to 0, Arcsine Laws* . . .. 172

4.4 Renewal Theory* . . . .. . 177

5.1 Conditional Expectation .. . 189

5.1.1 Examples . . . . . 191

5.1.2 Properties . . . 193

5.1.3 Regular Conditional Probabilities* . . . .197

5.2 Martingales, Almost Sure Convergence . . . . 198

5.3 Examples . . .. . . . 204

5.3.1 Bounded Increments . . . . . 204

5.3.2 Polya’s Urn Scheme . . . . . . 205

5.3.3 Radon-Nikodym Derivatives . . . . . 206

5.3.4 Branching Processes . . .. . . 209

6.1 Definitions . . .. . . . 233

6.2 Examples . . . . . . . . 236

6.3 Extensions of the Markov Property . . . . . 240

6.4 Recurrence and Transience .. . . 245

6.5 Stationary Measures . . . .. . . 252

6.6 Asymptotic Behavior . . .. . 261

7.1 Definitions and Examples .. . . 279

7.2 Birkhoff’s Ergodic Theorem . .. 283

7.3 Recurrence . . . . . . . 287

7.4 A Subadditive Ergodic Theorem* . 290

7.5 Applications* . . . . . 294

8.1 Definition and Construction . . .. 301

8.2 Markov Property, Blumenthal’s 0-1 Law . . 307

8.3 Stopping Times, Strong Markov Property . 312

8.4 Path Properites . . . .. 315

8.4.1 Zeros of Brownian Motion . . . . . 316

8.4.2 Hitting times . . . .. . 316

8.4.3 L´evy’s Modulus of Continuity . . . . . 319

8.5 Martingales . . . . . 320

8.5.1 Multidimensional Brownian Motion . . . . 324

8.6 Itˆo’s formula* . . . .. . 327

8.7 Donsker’s Theorem . . . . .333

8.8 CLT’s for Martingales* . . .340

8.9 Empirical Distributions, Brownian Bridge . . . . 346

8.10 Weak convergence* . . .. 351

A.1 Carathe

’eodory’s Extension Theorem . . . 359

A.2 Which Sets Are Measurable? . . 364

A.3 Kolmogorov’s Extension Theorem . . . 366

A.4 Radon-Nikodym Theorem . .. . 368