content

1 Introduction

2 Introduction to Programming 17
3 Mathematical Preliminaries 37
4 Analysis in Metric Space 54
5 Introduction to Dynamics 88
6 Introduction to Markov Chains 110
7 Measure and Integral 165
8 General State Markov Chains 201
9 Dynamics of General State MCs 237
10 Stochastic Dynamic Programming 263
11 More Stochastic Dynamic Programming 293
12 Numerical Methods for Dynamic Programming 308
A Appendix 323
A.1 Appendix to Chapter 4 . . . . . . . . . . . . . . . . . . . . . . 323
A.2 Appendix to Chapter 6 . . . . . . . . . . . . . . . . . . . . . . 324
A.3 Appendix to Chapter 11 . . . . . . . . . . . . . . . . . . . . . 325
July 21, 2006 Page 6 of 333