Course description

This is a course in statistical signal processing. In particular, as opposed to a first course is digital signal processing which deals with deterministic signals, this course aims to present methods by which to design signal processing techniques in the presence of uncertainty. This will usually involve an application of some estimation method, which will form the core of the assocated tool or apparatum. The emphasis will be on teaching how varios estimation frameworks yield different signal processing algorithms as opposed to presenting cook-book solutions.

Recommended reading

  • B. Porat, Digital Processing of Random Signals: Theory and Methods, Dover Publications (Mineola, NY), 1994
  • M. H. Hayes, Statistical Digital Signal Processing and Modeling, Wiley, 1996

Lecture notes

The course material will consist of three principal parts which will cover various aspects of modeling, estimation, and prediction of random signals and events. Specifically, the three parts will cover the follong subjects

Modeling
  • Probability space of random variables and processess
  • Probability measure, cdf, and pdf
  • Stationarity, ergodicity, spectral distribution and density
  • Hilbert spaces of WSS random processes
  • Innovation process, Word decomposition
  • Best linear predictor, Yule-Walker equations
  • AR, MA, and ARMA models
 Estimation
  • Parametric vs non-parametric estimation
  • Sufficient statistics, Neyman-Fisher factorization theorem
  • Fisher's information, the Cramer-Rao inequality, sequences of estimates
  • Maximum-likelihood, least-squares, and maximum entropy estimates
  • Bayesian estimation and model complexity/order selection
  • Non-parametric spectral analysis
 Prediction