EE 512: Stochastic Processes (Spring 2020)

Instructor: Ben Reichardt

EEB 528, 213-740-7229

office hours: TBA

TA: Runzhou (Roger) Zhang

EEB 500, 213-740-1488

office hours: TBA

Lectures: Tuesday and Thursday 12:30-1:50pm, OHE 132

Discussion: F 12-12:50, OHE 132

Summary: The course is an exploration of the theory and applications of stochastic processes with a special focus on computation. This entails a rigorous mastery of the underlying probability theory and statistics as well as familiarity with a programming language (Python or R recommended). There will be two midterms and a final exam.

Prerequisites:EE 503 and (one of 441 or 510 or 518), and EE 441

Textbooks:There is no required textbook for this class. But the following textbooks are highly recommended. They are listed in order of priority for this course:

Gubner, J. A., Probability and Random Processes for Electrical and Computer Engineers, Cambridge University Press, 2006.

Ross, S. M. Stochastic Processes.

Hsu, H. P. Schaum's outline of theory and problems of probability, random variables, and random processes. 2nd Ed. McGraw-Hill, 2014.

Glasserman, P. Monte Carlo methods in financial engineering. Springer, 2013.

Ross, S. M. Simulation. Academic Press, 2013.

Durrett, R., Essentials of Stochastic Processes. Springer, 2016.

Grading: 15% homework, 45% midterms, 40% final

Your lowest homework score will be thrown out before computing final grades.

No late homework will be accepted.  No make-up exams will be given.

You are encouraged to discuss homework problems among yourselves, but each person must do their own work. Copying or turning in identical homework sets is cheating.

You have one week from the date that a graded paper is returned to dispute the scoring of a problem, by submitting a request in writing to me.

Outline: (each item roughly corresponds to one week’s material)

1. Overview of probability: Probability spaces, random variables, distribution functions, moment generating functions, expectation, conditional probability and expectation, probability inequalities, examples

2. Stochastic processes: Examples, notions of convergence, definition of a stochastic process, independence, zero-one laws, laws of large numbers, central limit theorems, stable laws

3. The Poisson process: Definition, conditional distribution of the arrival times, non-homogeneous Poisson process, compound Poisson random variables and processes

4. Renewal theory: Limit theorems, Wald’s identity, key renewal theorem, branching processes, regenerative processes, stationary point processes

5. Discrete-time Markov chains: Examples in communication systems, Chapman-Kolmogorov equations, limit theorems, time-reversible Markov chains, semi-Markov processes

AE 512: Stochastic Processes (Spring 2020)

6. Continuous-time Markov chains: Examples, birth-death processes, Kolmogorov differential equations, limiting probabilities, time reversibility, uniformization, application to queueing theory, hidden Markov models and the Baum-Welch algorithm

Midterm exam 1 In class (date TBA — February 20??)

7. Martingales: Definition, martingale differences, level crossings, stopping times, Azuma’s maximal inequality, sub-martingales, super-martingales, and the martingale convergence theorem

8. Random walks: Definition, duality in random walks, exchangeable random variables, analysis using martingales, ruin problems, application in queueing systems

9. Brownian motion and other Markov processes: Definition, continuity and non-differentiability of paths, hitting times, maximum variable and arc sin laws

10. Variations on Brownian motion: Examples of diffusions, backward and forward diffusion equations, Markov shot noise process, scale functions, speed measures, calculation of functionals of measures

Midterm exam 2 In class (date TBA — March 26??)

11. Stochastic integration: Definition of Itô integral, Itô lemma, Chain rule of differentiation, Stratonovich integral, connection to Riemann and Riemann-Stieltjes integral

12. * Stochastic differential equations and finance applications: Itô stochastic differential equations, solution by the Itô lemma and the Stratonovich calculus, Girsanov’s change of measure technique, Black-Scholes formula

13. ** Simulation: General techniques for simulating continuous random variables, simulating stochastic point processes, variance reduction techniques, sample complexity bounds, generating from the stationary distribution of a Markov chain, Markov Chain Monte Carlo

Final exam Wednesday, May 14, 2-4pm 

Important:

Any form of cheating or plagiarism will lead to a severe penalty which is an F in the class. Assisting or facilitating cheating will also lead to an F in the class.

If you find the course difficult with a high risk to a grade you would not like to get, you may consider dropping the class or see me early in the semester to review your progress and help you out with understanding the material better.

After the class is over nothing can be done to change grades based on personal constraints or any other excuse. If you encounter an emergency that prevents you from studying and doing well, let me know right away.

EE 512: Stochastic Processes (Spring 2020)

Statement for Students with Disabilities:

Any student requesting academic accommodations based on a disability is required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accommodations can be obtained from DSP. Please be sure the letter is delivered to me (or to TA) as early in the semester as possible. DSP is located in STU 301 and is open 8:30 a.m.–5:00 p.m., Monday through Friday. The phone number for DSP is (213) 740-0776.