EECE 522 Estimation Theory
This Course is Offered Spring of Even Years;
Next Offered Spring 2018
Instructor Information
Course Description
Addresses the theory and practice of estimating parameters for discretetime
signals embedded in noise. Topics include:
 Classical Estimation (Deterministic Parameter)
 CramerRao Lower Bound
 Minimum Variance Unbiased Estimation
 Least Squares Estimation
 Maximum Likelihood Estimation
 Bayesian Estimation (Random Parameter)
 Minimum Mean Square Estimation
 Maximum A Posteriori Estimation
 Optimal Filtering
 Wiener Filtering
 Kalman Filtering
 Applications
 Radar, Sonar, and Emitter Location
 Communication Systems
Background Assumed
This course is not for the
mathematically weak!!
Must Have a Basic Understanding of:

Linear Algebra or Matrix Theory (see textbook appendix
& Reserve Books #1, 4)

Probability Theory and Random Functions (see textbook
appendix & Reserve Books #1, 2, 3)

Probability Density Functions

Joint, Marginal, Conditional Versions

Gaussian/Normal

Mean and Variance of Random Variables

WideSense Stationary Random Processes

Correlation Function & Covariance Matrix

Power Spectral Density

Digital Signal Processing (see Reserve Book #2)
 Fourier Transform for DiscreteTime Signals
 DiscreteTime Filters (Mostly FIR  not design, but
operation via convolution)
Textbook
 Fundamentals of Statistical Signal Processing, Volume I:
Estimation Theory
by Steven Kay (Published by
Prentice Hall)
Other Books of Interest
 Parameter Estimation  H. Sorenson
 Covers same ground as textbook but in a different
order; thus, provides an interesting alternative view.
 Has appendices on Matrices and Probability Theory  a
little more detailed than textbook.
2.
Signal Processing: Discrete Spectral Analysis, Detection, and
Estimation  M. Schwartz and L. Schaw
o
Ch. 2 Reviews Digital Signal Processing
o
Ch. 3 reviews Random DiscreteTime Signals
o
Ch. 6 gives concise coverage of Parameter Estimation (Classical
and Bayesian) as well as Wiener Filter
o
Ch. 7 covers Kalman Filters and has example of Aircraft Tracking
 Introduction to Random Signal Analysis and Kalman
Filtering  R. Brown
 Gives a good overview of probability and random
processes
 Several Chapters on Kalman Filter
 Estimation Theory and Applications  N.
Nahi
 An older book on estimation, but still might have
useful perspectives on parameter estimation
 BUT... mostly focused on stateestimation (e.g.,
Kalman Filter type stuff)
 HOWEVER... has a good section on Matrix Algebra and
Quadratic Forms
 Applied Optimal Estimation  A. Gelb
 "THE BIBLE" for Kalman Filters  on the bookshelf of
virtually everyone working with Kalman Filters!
 Data Analysis: A Bayesian Tutorial  D.
Sivia
 An Excellent, downtoearth book on Bayesian
estimation
 Starts with Bayesian approach and shows how it
"degenerates" into classical methods (ML & LS)
 Mostly deals with problems of the scientific data
analysis sort but still very good for signal processing types
Relevant Papers & Other Material
For most of these
you can find them in the library.
I'll try to post most of them on
Blackboard.
Only by reading papers in the area
can you really get a feeling for how this stuff works!
The following link gives some advice
on how to read technical papers:
How To Read Papers
General Papers
 D. Torrieri, "Statistical Theory of Passive Location
Systems," IEEE Transactions on Aerospace and Electronic Systems, pp.
183  198, March 1984
 W. Gardner, "Likelihood Sensitivity and the CramerRao
Bound," IEEE Transactions on Information Theory, p. 491, July 1979
 J. Cadzow, "Least Squares, Modeling, and Signal
Processing," Digital Signal Processing, pp. 2  20, 1994
 W. Press et al., "Ch. 15 Modeling of Data", in
Numerical Recipes in C, 2nd Edition, Cambridge Press
Application Papers
 S. Stein, "Differential Delay/Doppler ML Estimation with
Unknown Signals," IEEE Transactions on Signal Processing, pp. 2717 
2719, August 1993
 T. Berger and R. Blahut, "Coherent Estimation of
Differential Delay and Differential Doppler," Proceedings of the 1984
Conference on Information Sciences and Systems, Princeton University, pp. 537
 541, 1984
 M. Fowler, “Analysis of Passive Emitter Location using
Terrain Data,” IEEE Transactions on Aerospace and Electronic Systems,
pp. 495 – 507, April 2001.
 K. Becker, "An Efficient Method of Passive Emitter
Location," IEEE Transactions on Aerospace and Electronic Systems, pp.
1019 – 1104, Oct. 1992
 P. Chestnut, "Emitter Location Accuracy using TDOA and
Differential Doppler," IEEE Transactions on Aerospace and Electronic
Systems, pp. 214  218.
 M. Fowler, “Air‑to‑Air Passive Location,” U.S. Patent
#5,870,056 Issued 2/9/1999
 D. Rife and Boorstyn, "SingleTone Parameter Estimation from
DiscreteTime Observations," IEEE Transactions on Information Theory, pp.
591  598, Sept. 1974.
 S. Tretter, "Estimating the Frequency of a Noisy Sinusoid by Linear
Regression," IEEE Transactions on Information Theory, pp. 832  835, Nov.
1985.
 S. Kay, "A Fast Accurate Single Frequency Estimator," IEEE
Transactions on Acoustics, Speech, and Signal Processing , pp. 1987  1990,
Dec. 1989.
Assorted Handouts
Lecture Notes
Please
download, print out, and bring to the relevant class  see Course Schedule above
These notes are
complete versions of my class notes.
 You'll only need to fill in
certain spoken information during class you deem important.
 This will free you up for
inclass thinking (come ready to do some!)
There also a few "reading notes"
that supplement the textbook's coverage... these are now posted on BB.
New PDFs of PPT Charts
Notes #1a Probability Review
Notes #1b Vectors and Matrices Review (See Reading Notes on BB)
Notes #2: Ch 1 Intro to Est
Notes #3: Ch 2 MVUE
Notes #4: Ch 3 Cramer Rao Bound Pt. A
Notes #5: Ch 3 Cramer
Rao Bound Pt. B
Notes #6: Ch 3 Cramer
Rao Bound Pt. C
Notes #7: Ch 3 Cramer
Rao Bound Pt. D
Notes #8: Ch 3 CRLB Examples
Notes #9: CRLB Example for Doppler Location (See Reading Notes on BB)
Notes #10 Ch_4 Linear
Models
Notes #11 Ch_6 BLUE
Notes #12 Ch7A
Notes #13 Ch7B
Notes #14 Ch7C
Notes #15 ML Example  Revised
Notes #16 Ch8A
Notes #17 Ch8B
Notes #18 Ch8C
Notes #19 Ch8D
Notes #20 LS Single Platform (See Reading Notes on BB)
Notes
#21 Doppler Tracking (See Reading Notes on BB)
Notes
#22 Results for 2 RVs (PreCh. 10) (See Reading Notes on BB)
Notes #23 Ch10A
Notes #24 Ch10B
(See Reading Notes on BB)
Notes #24a
Bayesian Example
Notes #25 Ch11A
Notes #26 Ch11B
Notes #26a Recursive Bayesian
Notes #27 Ch12A
Notes #28 Ch12B
Notes #28a Wiener Filter for Deblurring Images
Notes #29 Ch13A
Notes #30 Ch13B
Notes #31 Ch13C
Notes #32 Ch13D
Homework Assignments
 Assignments
 Will be posted on
Blackboard (if you don't know how to get access to it ask me)
 Solutions
 Will be posted on
Blackboard (if you don't know how to get access to it ask me)
Project Information
A significant portion of your grade will be
based on a project. It is important to start early.
Things you can do earlyon are:
 Understand the Signal Model and Project Issues
 Derive/Analyze CramerRao bounds for your problem
 Write simulation code to generate the data
Things you can do by midsemester are:
 Estimator derivation and analysis (most projects will
use classical methods, all of which we will have studied by Spring Break)
 Coding of estimator
 Start your analyses of effects and/or tradeoffs
Things you can do by end of semester are:
 Complete your analyses of effects and/or tradeoffs
 Complete your simulations
 Analyze your results
 Write your report
Here are three files to help you get started.
 The first gives a list of project suggestions.
 The second gives details on how to do your report.
 The third is a MSWord template that will help you format your report to
professional publication standards
Project Files
MATLAB Handouts & Links
Links of Interest
DSP Tutorials and Reference Material
DSP Demos
Some interactive demos of DSP
concepts (e.g., filter design)
DSP Tutorial
Some basics of DSP theory and
implementation.
The Scientist and Engineer's Guide to Digital Signal Processing
A freely downloadable DSP Book!!!!
Provides coverage at the level assumed as a prerequisite for EE522  so it's a
good place to start if you need a refresher.
Signals & Systems Demos (Johns Hopkins University)
A neat set of java applets that
demonstrate continuoustime & discretetime signal processing at the level
assumed as a prerequisite for EE522  so it's a good place to start if you need
a refresher.
Estimation Oriented Material
Frequency Estimation
An overview of many different ways
to estimate frequency.
Blind SNR Estimation
Discusses how to estimate the SNR of
a signal.
Back to Prof. Fowler's Homepage