EE 491D: Stochastic Models

Course Description

EE491D is an undergraduate course on applied probability, stochastic processes, and  Markov processes. Many UH EE students complete EE 342, but do not have confidence  in their ability to apply probability to real-world problems involving uncertainty. We  first do an extensive review of the results and techniques in EE 342 by using examples  that are related to applications. The applications in EE 491D will include  communications, computer networks, systems reliability, Google search, software  reliability, data structures, probabilistic analysis of algorithms, system performability,  wireless networks, financial engineering, and so forth. We introduce additional topics on  elementary stochastic processes, Markov chains, and queuing theory. We will also  consider obtaining numerical solutions to difficult problems by using Monte Carlo  simulation techniques. This course can be used as a Systems Elective or a TE.

Course Objectives:

• To strengthen the student’s background in solving probability problems • To introduce the student to stochastic processes
• To introduce the student to additional applications of probability and stochastic  processes

Pre-requisite: EE 342

Textbooks:

1. Introduction to Probability Models, tenth or eleventh edition, by Sheldon Ross.  This is a follow-up textbook to the previous EE 342 textbook (A First Course in  Probability by Ross).  
2. Probability by Dimitri Bertsekas and John Tsitsiklis (BT) (current EE 342  textbook).

Grading: 

The approximate weights for grades are 1 midterm (25%), 2 quizzes (15% each), homework (10%) and the Final Exam (35%).

Course Outline

• Extensive review of EE 342 – includes coverage of topics that students might not  have learned well (for whatever reason) in EE 342. BT chapters 1-5 and Ross  chapters 1-3. 4 to 5 weeks.
  
• Bernoulli and Poisson Processes – BT chapter 6 and Ross chapter 5. 2.5 weeks
  
• Markov Processes – BT chapter 7 and Ross chapters 4 and 6. 3 weeks.
  
• Other Topics – possibly queueing theory, renewal processes, statistics. 3 weeks.
  
• Applications of probability and stochastic processes.