Lecture 1: (1/10) Introduction, course overview, references, slides-1.
Lecture 2: (1/12) Adaptive signal processing, learning algorithms, tap delay line, square error cost function, stochastic gradient algorithms and least square algorithms, applications. slides-2.
Lecture 3: (1/14) Linear vector spaces, closure, norms, inner products. Subspaces, orthogonal vectors, orthonormal basis vectors, linear independent vectors.
Lecture 4: (1/19) Complete set of basis functions, Gram-Schmidt orthogonalization. Matrix definition, addition, linear functions (multiplying a matrix by a vector), composition of linear functions (multiplying two matrices). Square matrices, identity matrix, matrices of full rank and inverses. Matlab introduction.
Lecture 5: (1/21) Orthonormal matrices. Gaussian elimination, pivoting, example. LU decomposition. Least squares problem, quadratic cost function.
Lecture 6: (1/24) Least squares problem, projection theorem, pseudoinverse, mean squared error. Determinants, geometric intuition. Autonomous vector difference equations.
Lecture 7: (1/26) Autonomous vector difference equations. Eigenvalues, eigenvectors, characteristic equation. Similarity transformation. Decomposition of symmetric matrices.
Lecture 8: (1/31) Review of probability and random variables. Random variables, pdfs, CDF, expectation, mean, covariance matrix, moment generating function (MGF). Gaussian random vectors, joint pdfs, properties: joint MGF, linear transformation of jointly Gaussian random vectors, simulating Gaussian random variables.
Lecture 9: (2/2) Simulation of Gaussian random variables. Random processes, discrete time random processes, complete characterization, mean function, autocorrelation function. Gaussian random processes. Stationarity and widesense stationarity.
Lecture 10: (2/4) Review first homework. Widesense stationary processes, properties of autocorrelation function, power spectral density.
Lecture 11: (2/7) Power spectral density and properties. Power of random signals. Passing random signals through LTI systems. Computing output mean, autocorrelation function, and power spectral density. Moving average, Autoregressive, and ARMA processes. Examples.
Lecture 12: (2/9) Optimal linear minimum mean squared error filtering. Model assumptions, energy function, Wiener solution. Projection theorem.
Lecture 13: (2/14) Eigenvalues of correlation matrix and relationship to energy function. System identification problem, selecting model order. Gradient of energy function, steepest descent algorithm. Convergence of steepest descent.
Lecture 14: (2/16) Review of energy surface and steepest descent algorithm. Convergence of steepest descent. Choosing step size based on eigenvalues. Energy surface as a function of eigenvalues. LMS algorithm, noisy stochastic gradient algorithm and properties. PS 2 due. slides-14.
Lecture 15: (2/23) LMS algorithm, convergence of algorithm, mean convergence, energy function behavior, transient effects, convergence speed as a function of step size, excess mean squared error, misadustment. LMS algorithm applications: inverse modeling, system identification, prediction, interference cancellation. slides-15.
Discussion section: (2/25) Review of computing autocorrelation function and power spectral density for output to a LTI system driven by white noise. PS 2 and PS 3 discussion.
Lecture 16: (2/28) Applications of LMS. Inverse modeling: channel equalization problem, ISI and additive noise; deconvolution problem. System identification. Prediction: AR process prediction.
Lecture 17: (3/2) Interference cancellations, echo cancellation. Variations to LMS, LMS with nonzero bias, normalized LMS, block LMS, frequency domain LMS. Least Squares methods, solution. Data presentation using windows. Properties of LS solution, statistical properties of solution. Singular Value Decomposition. slides-17.
Lecture 18: (3/9) Review LS solution, data windowing, and statistical properties of solution. Singular Value Decomposition and pseudoinverse. slides-18.
Lecture 19: (3/11) Review SVD and pseudoinverse. Recursive least squares algorithm. Exponential weighting and intial conditions. Sherman Morrison Woodbury formula. Recursive Least Squares (RLS) algorithm. Gain equation, covariance equation, weight update equation. A priori estimation error. RLS comments.
Individual Meetings: (3/14) Discuss preliminary ideas for projects.
Lecture 20: (3/28) Review RLS algorithm. Exponential weighting and intial conditions. Sherman Morrison Woodbury formula. Gain equation, covariance equation, weight update equation. A priori estimation error.
Lecture 21: (3/30) Homework review. RLS summary. PS 4 due.
Lecture 22: (4/4) Convergence of RLS algorithms: mean convergence, energy function for prior error. RLS properties. Adaptive equalization example. Introduction to innovations processes. Estimation given two sets of training examples that are related via invertible linear transformation.
Lecture 23: (4/6) Innovations processes, forward prediction. Forming the innovations process, whitening input process. Introduction to Kalman Filtering.
Individual meetings: (4/8) Discussion of projects. PS 5 due.
Lecture 23: (4/11) Introduction to Kalman filtering, time and measurment updates. Review of material for exam.
Exam: (4/16)
Lecture 24: (4/18) Exam problems. Review of innovations and Kalman filtering.
Lecture 25: (4/20) Kalman filter prediction equations. Matrix decompositions: LU, SVD, QR, Cholesky.
Individual meetings: (4/22)
Lecture 26: (4/25) Project topics. Cholesky factorization, square root algorithms, QR RLS algorithm. IIR adaptive filters, updating weights. Forward and backward prediction, Levinson Durbin algorithm. Lattice filter predictor.