Lecture 1: (8/24) Introduction. Why random processes? Engineering Applications: communication system (matched filter), ATM switch, information theory.
Lecture 2: (8/26) Probability triple, sample space, events, set operations, fields, sigma fields, probability measure, probability axioms, properties, conditional probability, Theorem of total probability, Bayes rule, examples.
Lecture 3: (8/28) Conditional probability example (BSC), independence, random variable definition, distribution function and properties, continous random variables, pdf, examples.
Lecture 4: (8/31) Review RVs and distribution functions, discrete RVs, Multiple RVs, joint distribution function and properties, joint pdf, marginals, condition density and dist. functions, independence.
Lecture 5: (9/2) Functions of one random variable, transformation of multiple RVs, auxiliary method, distribution function method, examples, simulation of RVs.
Lecture 6: (9/4) Examples of functions of RVs, Expectation, definition, properties, and examples, quiz 1
Lecture 7: (9/9) , Expectation, moments, moment generating function, characteristic function, expectation of multiple random variables, correlation, covariance, joint characteristic function, properties, conditional expectation, examples.
Lecture 8: (9/11) Review representation of RVs, Jointly Gaussian random variables, properties. Expectation, definition, properties.
Lecture 9: (9/14) Linear transformations of Gaussian RVs, similarity transformations, conditional Gaussian RVs, sequences of RVs, convergence, sequence of events and convergence.
Lecture 10: (9/16) Sequence of events, AS, KM, IP, and ID convergence. Relationships between different convergence and proofs.
Lecture 11: (9/18) Proof AS convergence implies convergence IP. examples. quiz 2
Lecture 12: (9/21) Review convergence relations and definitions, mutual convergence, Borel Cantelli lemma, examples, Weak Law of Large Numbers, Chernoff bounds.
Lecture 13: (9/23) (2 hours) Strong Law of Large Numbers (SLLN), Martingales, Martingale Convergence Theorem (MCT), proof of SLLN using MCT, renewal theory application, Central Limit Theorem, Bayesian estimation, minimum mean squared error estimation, properties of conditional expectation, examples.
Lecture 14: (9/25) Linear vector spaces, normed vector spaces, inner product spaces, complete spaces, Banach and Hilbert space properties.
Lecture 15: (9/28) Proof of projection theorem, normal equations, application to linear estimation, examples.
Lecture 16: (9/30) Minimum mean squre error estimation, linear MMSE estimation, examples,
Lecture 17: (10/2) quiz 3
Lecture 18: (10/7) Estimation of Widesense Markov processes, Random processes, definitions, characterizations, joint distribution functions, joint pdf/mass functions.
Lecture 19: (10/9) Separability, mean function, autocorrelation function, examples, Bernoulli process, Binomail process, Sinusoid process with random phase.
Lecture 20: (10/12) Poisson process, process of independent increments, Gaussian processes, Brownian motion.
Lecture 21: (10/14) Gaussian processes, Brownian motion, AR processes. quiz 4
Lecture 22: (10/16) Markov processes and properties, Chapman-Kolmogorov equations, Markov chains, state trasition diagrams, state transition matrix, stationary processes.
Lecture 23: (10/19) Stationary processes, widesense stationary processes, two random processes, orthogonal processes, independent processes, jointly widesense stationary processes, examples.
Lecture 24: (10/21) Self-similar and long range dependent processes, second order self-similar processes and properties, short range dependent processes, second order processes, properties of autocorrelation function.
Lecture 25: (10/23) second order processes, properties of autocorrelation function, continuous process definitions. quiz 5
Lecture 26: (10/26) continuous processes, quadratic mean continuous, second order calculus, quadratic mean derivatives, quadratic mean integrals.
Lecture 27: (10/28) time averages, ergodic processes, mean ergodic, ergodic in correlation, ergodic in distribution, examples.
Lecture 28: (10/30) power spectral density, properties, examples, cross power spectral density, stationary processes and linear time invariant systems.
Lecture 29: (11/4) exam
Lecture 30: (11/6) autocorrelation and power spectral density review, linear time invariant systems, examples, Karhunen Loeve expansion.
Lecture 31: (11/13) Karhunen Loeve expansion, Mercers Theorem, convergence, examples.
Lecture 32: (11/14) (2 hours) KL expansion examples, KL application to waveform detection in additive Gaussian noise, baseband processes, narrowband processes. quiz 6
Lecture 33: (11/16) Hilbert Transform, narrowband processes, quadrature representation, envelope and phase representation, Gaussian narrowband processes, spectral factorization.
Lecture 34: (11/18) Spectral factorization, minimum phase systems, innovations filter, whitening filter, Paley Wiener condition, Discrete process factorization, state space representation.
Lecture 35: (11/20) Matched Filter, unconstrained solution using Schwarz Inequality, optimal filter, signal to noise power ratio, examples: white noise and nonwhite noise.
Lecture 36: (11/23) Realizable Matched Filter, integral equation, frequency domain solution using spectral factorization and additive decompositions, examples.
Lecture 37: (11/25) Matched Filter example, comparison between unconstrained and realizable, Wiener Filter introduction. quiz 7
Lecture 38: (11/30) Wiener Filter, principle of orthogonality, integral equation, unconstrained case, examples, realizable filter derivation.
Lecture 37: (12/2) Realizable Wiener Filter, examples, pure prediction, innovations processes.
Lecture 38: (12/4) Innovations process derivation, estimation using innovations, example, state space model.
Lecture 39: (12/7) Discrete time Kalman filter, measurement update and time update equations, covariance update equation, gain equation, example.
Lecture 40: (12/9) quiz 8