ESI 6213 Stochastic Decision Models I

Objective: Get exposed to the theory of mathematical probability and stochastic decision processes and build foundations for their applications.

Textbook: Introduction to Probability Models, S. Ross, 9th ed., 2006.

Topics:

I. Review of probability theory
Elements of set theory; events & probability spaces
Elementary Bayesian concepts; statistical independence
Random variables, vectors, & functions; moments
Conditional probability & expectation
Distributions & transformations
II. Markov chains
Markovian property
Chapman-Kolmogorov equations
Classification of states
Limiting probabilities
Applications
III. Poisson processes
Counting processes
Properties of Poisson processes
Non-homogeneous Poisson processes
Compound Poisson processes
IV. Markov processes
Continuous-time Markov chains
Birth & death processes
Transition probability function
Limiting probabilities
Applications
V. Renewal processes
Limit theorems and their applications
Renewal reward processes
Regenerative processes
Semi-Markov processes