Course textbook

Boaz Porat, Digital Processing of Random Signals: Theory and Methods, Dover Publications, Inc., 2008.

Course outline

Lecture 1: Probability space, cumulative distribution and probability density functions, discrete-time random processes, Gaussian random processes, stationarity and ergodicity, means and covariances.
Lecture 2: Hilbert spaces of stationary processes, inner product, orthogonality of vectors and sets, orthogonal projections, the best linear approximation, innovation processes, linear predictions.
Lecture 3: The Wold decomposition theorem, power spectral density, spectral factorization, properties of the best linear predictors, the Yule-Walker equation, wide-sense ergodicity of Gaussian processes, moving average (MA), autoregressive (AR) and ARMA models.
Lecture 4: Principles of parameter estimation, bias, admissibility and the minimax properties, minimum variance unbiased estimates, the Cramer-Rao inequality, sequences of estimates.
Lecture 5: Maximum likelihood (ML) estimation, the method of moments, least-squares estimation, maximum entropy estimation, model order selection.
Lecture 6: Estimation of means and covariances, nonparametric spectrum estimation, periodograms, smoothed and windowed periodograms.
Lecture 7: The Fisher information of stationary Gaussian processes, ML parameter estimation, relative efficiency, parameter estimation from the sample covariance.
Lecture 8: The AR parameter estimation, the Yule-Walker estimate, the Levinson-Durbin algorithm, maximum entropy estimation, ML estimation, LS estimation, model order selection, estimation of the spectral density.
Lecture 9: MA parameter estimation, ARMA estimation, approximate ML ARMA parameter estimation, exact ML ARMA estimation.
Lecture 10: Adaptive estimation, the recursive least squares (RLS) algorithm, the extended LS algorithm, the recursive ML algorithm, stochastic gradient algorithm, lattice algorithms for AR estimation.
Lecture 11: Estimation of deterministic processes, estimation of harmonic signals and its Cramer-Rao bound, ML estimation of harmonic signals, the Prony method, the trancated SVD method, the MUSIC algorithm.
Lecture 12: Cumulants, polyspectra of linear stationary processes, the cumulants of ARMA processes, estimation of the cumulants.
Lecture 13: Linear methods of MA and ARMA parameter estimation, nonlinear methods for MA and ARMA parameter estimation, deconvolution.

Marking scheme

Home assignments (5 × 5% = 25%), final project (25%), final exam (50%).