Course textbook
Boaz Porat, Digital Processing of Random Signals: Theory and Methods, Dover Publications, Inc., 2008.
Course outline
- Lecture 1: Probability space, cumulative distribution and probability density functions, discrete-time random processes, Gaussian random processes, stationarity and ergodicity, means and covariances.
- Lecture 2: Hilbert spaces of stationary processes, inner product, orthogonality of vectors and sets, orthogonal projections, the best linear approximation, innovation processes, linear predictions.
- Lecture 3: The Wold decomposition theorem, power spectral density, spectral factorization, properties of the best linear predictors, the Yule-Walker equation, wide-sense ergodicity of Gaussian processes, moving average (MA), autoregressive (AR) and ARMA models.
- Lecture 4: Principles of parameter estimation, bias, admissibility and the minimax properties, minimum variance unbiased estimates, the Cramer-Rao inequality, sequences of estimates.
- Lecture 5: Maximum likelihood (ML) estimation, the method of moments, least-squares estimation, maximum entropy estimation, model order selection.
- Lecture 6: Estimation of means and covariances, nonparametric spectrum estimation, periodograms, smoothed and windowed periodograms.
- Lecture 7: The Fisher information of stationary Gaussian processes, ML parameter estimation, relative efficiency, parameter estimation from the sample covariance.
- Lecture 8: The AR parameter estimation, the Yule-Walker estimate, the Levinson-Durbin algorithm, maximum entropy estimation, ML estimation, LS estimation, model order selection, estimation of the spectral density.
- Lecture 9: MA parameter estimation, ARMA estimation, approximate ML ARMA parameter estimation, exact ML ARMA estimation.
- Lecture 10: Adaptive estimation, the recursive least squares (RLS) algorithm, the extended LS algorithm, the recursive ML algorithm, stochastic gradient algorithm, lattice algorithms for AR estimation.
- Lecture 11: Estimation of deterministic processes, estimation of harmonic signals and its Cramer-Rao bound, ML estimation of harmonic signals, the Prony method, the trancated SVD method, the MUSIC algorithm.
- Lecture 12: Cumulants, polyspectra of linear stationary processes, the cumulants of ARMA processes, estimation of the cumulants.
- Lecture 13: Linear methods of MA and ARMA parameter estimation, nonlinear methods for MA and ARMA parameter estimation, deconvolution.