TABLE OF CONTENTS
PREFACE
SYMBOLS AND ABBREVIATIONS
1 PROBABILITY THEORY
1.1 RANDOM EVENTS AND THEIR PROBABILITIES 1
1.2 RANDOM VARIABLES 6
1.2.1 Basic Concepts 6
1.2.2 Discrete Random Variables 9
1.2.2.1 Numerical Parameter 9
1.2.2.2 Important Discrete Probability Distributions 10
1.2.3 Continuous Random Variables 14
1.2.3.1 Probability Density and Numerical Parameter 14
1.2.3.2 Important Continuous Probability Distributions 16
1.2.4 Mixtures of Random Variables 22
1.2.5 Functions of Random Variables 26
1.3 TRANSFORMATION OF PROBABILITY DISTRIBUTIONS 28
1.3.1 z-Transformation 29
1.3.2 Laplace-Transformation 31
1.4 CLASSES OF PROBABILITY DISTRIBUTIONS BASED ON AGING 35
BEHAVIOUR
1.5 ORDER RELATIONS BETWEEN RANDOM VARIABLES 43
1.6 MULTIDIMENSIONAL RANDOM VARIABLES 46
1.6.1 Basic Concepts 46
1.6.2 Two-Dimensional Random Variables 47
1.6.2.1 Discrete Components 47
1.6.2.2 Continuous Components 48
1.6.3 n-Dimensional Random Variables 57
1.7 SUMS OF RANDOM VARIABLES 62
1.7.1 Sums of Discrete Random Variables 62
1.7.2 Sums of Continuous Random Variables 63
1.7.3 Sums of a Random Number of Random Variables 68
1.8 INEQUALITIES IN PROBABILITY THEORY 70
1.8.1 Inequalities for Probabilities 70
1.8.2 Inequalities for Moments 72
1.9 LIMIT THEOREMS 73
1.9.1 Convergence Criteria for Sequences of Random Variables 73
1.9.2 Laws of Large Numbers 74
1.9.3 Central Limit Theorem 76
1.10 EXERCISES 81
© 2006 by Taylor & Francis Group, LLC
2 BASICS OF STOCHASTIC PROCESSES
2.1 MOTIVATION AND TERMINOLOGY 91
2.2 CHARACTERISTICS AND EXAMPLES 95
2.3 CLASSIFICATION OF STOCHASTIC PROCESSES 99
2.4 EXERCISES 105
3 RANDOM POINT PROCESSES
3.1 BASIC CONCEPTS 107
3.2 POISSON PROCESSES 113
3.2.1 Homogeneous Poisson Processes 113
3.2.1.1 Definition and Properties 113
3.2.1.2 Homogeneous Poisson Process and Uniform Distribution 119
3.2.2 Nonhomogeneous Poisson Processes 126
3.2.3 Mixed Poisson Processes 130
3.2.4 Superposition and Thinning of Poisson Processes 136
3.2.4.1 Superposition 136
3.2.4.2 Thinning 137
3.2.5 Compound Poisson Processes 140
3.2.6 Applications to Maintenance 142
3.2.6.1 Nonhomogeneous Poisson Process and Minimal Repair 142
3.2.6.2 Standard Replacement Policies with Minimal Repair 144
3.2.6.3 Replacement Policies for Systems with Two Failure Types 147
3.2.6.4 Repair Cost Limit Replacement Policies 149
3.3 RENEWAL PROCESSES 155
3.3.1 Definitions and Examples 155
3.3.2 Renewal Function 158
3.3.2.1 Renewal Equations 158
3.3.2.2 Bounds on the Renewal Function 164
3.3.3 Asymptotic Behaviour 166
3.3.4 Recurrence Times 170
3.3.5 Stationary Renewal Processes 173
3.3.6 Alternating Renewal Processes 175
3.3.7 Compound Renewal Processes 179
3.3.7.1 Definition and Properties 179
3.3.7.2 First Passage Time 183
3.3.8 Regenerative Stochastic Processes 185
3.4 APPLICATIONS TO ACTUARIAL RISK ANALYSIS 188
3.4.1 Basic Concepts 188
3.4.2 Poisson Claim Arrival Process 190
3.4.3 Renewal Claim Arrival Process 196
3.4.4 Normal Approximations for Risk Processes 198
3.5 EXERCISES 200
© 2006 by Taylor & Francis Group, LLC
4 MARKOV CHAINS IN DISCRETE TIME
4.1 FOUNDATIONS AND EXAMPLES 207
4.2 CLASSIFICATION OF STATES 214
4.2.1 Closed Sets of States 214
4.2.2 Equivalence Classes 215
4.2.3 Periodicity 218
4.2.4 Recurrence and Transience 220
4.3 LIMIT THEOREMS AND STATIONARY DISTRIBUTION 226
4.4 BIRTH- AND DEATH PROCESSES 231
4.5 EXERCISES 233
5 MARKOV CHAINS IN CONTINUOS TIME
5.1 BASIC CONCEPTS AND EXAMPLES 239
5.2 TRANSITION PROBABILITIES AND RATES 243
5.3 STATIONARY STATE PROBABILITIES 252
5.4 SOJOURN TIMES IN PROCESS STATES 255
5.5 CONSTRUCTION OF MARKOV SYSTEMS 257
5.6 BIRTH- AND DEATH PROCESSES 261
5.6.1 Birth Processes 261
5.6.2 Death Processes 264
5.6.3 Birth and Death Processes 266
5.6.3.1 Time-Dependent State Probabilities 266
5.6.3.2 Stationary State Probabilities 274
5.6.3.3 Inhomogeneous Birth- and Death Processes 277
5.7 APPLICATIONS TO QUEUEING MODELS 281
5.7.1 Basic Concepts 281
5.7.2 Loss Systems 283
5.7.2.1 -Systems 283 M/M/∞
5.7.2.2 -Systems 284 M/M/s/0
5.7.2.3 Engset's Loss System 286
5.7.3 Waiting Systems 287
5.7.3.1 -Systems 287 M/M/s/∞
5.7.3.2 -Systems 290 M/G/1/∞
5.7.3.3 -Systems 293 G/M/1/∞
5.7.4 Waiting-Loss-Systems 294
5.7.4.1 M/M/s/m-System 294
5.7.4.2 M/M/s/ -System with Impatient Customers 296 ∞
5.7.5 Special Single-Server Systems 298
5.7.5.1 System with Priorities 298
5.7.5.2 M/M/1/m-System with Unreliable Server 300
© 2006 by Taylor & Francis Group, LLC
5.7.6 Networks of Queueing Systems 303
5.7.6.1 Introduction 303
5.7.6.2 Open Queueing Networks 303
5.7.6.3 Closed Queueing Networks 310
5.8 SEMI-MARKOV CHAINS 314
5.9 EXERCISES 321
6 MARTINGALES
6.1 DISCRETE-TIME MARTINGALES 331
6.1.1 Definition and Examples 331
6.1.2 Doob-Type Martingales 336
6.1.3 Martingale Stopping Theorem and Applications 340
6.1.4 Inequalities for Discrete-Time Martingales 344
6.2 CONTINUOUS-TIME MARTINGALES 345
6.3 EXERCISES 349
7 BROWNIAN MOTION
7.1 INTRODUCTION 351
7.2 PROPERTIES OF THE BROWNIAN MOTION 353
7.3 MULTIDIMENSIONAL AND CONDITIONAL DISTRIBUTIONS 357
7.4 FIRST PASSAGE TIMES 359
7.5 TRANSFORMATIONS OF THE BROWNIAN MOTION 366
7.5.1 Identical Transformations 366
7.5.2 Reflected Brownian Motion 367
7.5.3 Geometric Brownian Motion 368
7.5.4 Ornstein-Uhlenbeck Process 369
7.5.5 Brownian Motion with Drift 370
7.5.5.1 Definitions and First Passage Times 370
7.5.5.2 Application to Option Pricing 374
7.5.5.3 Application to Maintenance 379
7.5.5.4 Point Estimation for Brownian Motion with Drift 384
7.5.6 Integral Transformations 387
7.5.6.1 Integrated Brownian Motion 387
7.5.6.2 White Noise 389
7.6 EXERCISES 392
ANSWERS TO SELECTED EXERCISES 397
REFERENCES 405