Statistical and Adaptive Signal Processing : Spectral Estimation, Signal Modeling, Adaptive Filtering and Array Processing

by Manolakis, Dimitris G.; Ingle, Vinay K.; Kogon, Stephen M.

• ISBN13: 9781580536103
• ISBN10: 1580536107
• Format: Hardcover
• Copyright: 2005-04-30
• Publisher: Artech House on Demand

Summary

This authoritative volume on statistical and adaptive signal processing offers you a unified, comprehensive and practical treatment of spectral estimation, signal modeling, adaptive filtering, and array processing. Packed with over 3,000 equations and more than 300 illustrations, this unique resource provides you with balanced coverage of implementation issues, applications, and theory, making it a smart choice for professional engineers and students alike.

Author Biography

Stephen M. Kogon is a member of the technical staff at M.I.T. Lincoln Laboratory, and has been associated with Raytheon Co., Boston College, and Georgia Tech Research Institute. Vinay K. Ingle is an associate professor of electrical and computer engineering at Northeastern University. Dimitris G. Manolakis is a member of the technical staff at M.I.T. Lincoln Laboratory and taught at the University of Athens, Northeastern University, Boston College, and Worcester Polytechnic Institute.

Table of Contents

Preface	xvii
Introduction	1	(32)
Random Signals	1	(7)
Spectral Estimation	8	(3)
Signal Modeling	11	(5)
Rational or Pole-Zero Models
Fractional Pole-Zero Models and Fractal Models
Adaptive Filtering	16	(9)
Applications of Adaptive Filters
Features of Adaptive Filters
Array Processing	25	(4)
Spatial Filtering or Beamforming
Adaptive Interference Mitigation in Radar Systems
Adaptive Sidelobe Canceler
Organization of the Book	29	(4)
Fundamentals of Discrete-Time Signal Processing	33	(42)
Discrete-Time Signals	33	(4)
Continuous-Time, Discrete-Time, and Digital Signals
Mathematical Description of Signals
Real-World Signals
Transform-Domain Representation of Deterministic Signals	37	(10)
Fourier Transforms and Fourier Series
Sampling of Continuous-Time Signals
The Discrete Fourier Transform
The z-Transform
Representation of Narrowband Signals
Discrete-Time Systems	47	(7)
Analysis of Linear, Time-Invariant Systems
Response to Periodic Inputs
Correlation Analysis and Spectral Density
Minimum-Phase and System Invertibility	54	(10)
System Invertibility and Minimum-Phase Systems
All-Pass Systems
Minimum-Phase and All-Pass Decomposition
Spectral Factorization
Lattice Filter Realizations	64	(6)
All-Zero Lattice Structures
All-Pole Lattice Structures
Summary	70	(5)
Problems	70	(5)
Random Variables, Vectors, and Sequences	75	(74)
Random Variables	75	(8)
Distribution and Density Functions
Statistical Averages
Some Useful Random Variables
Random Vectors	83	(14)
Definitions and Second-Order Moments
Linear Transformations of Random Vectors
Normal Random Vectors
Sums of Independent Random Variables
Discrete-Time Stochastic Processes	97	(18)
Description Using Probability Functions
Second-Order Statistical Description
Stationarity
Ergodicity
Random Signal Variability
Frequency-Domain Description of Stationary Processes
Linear Systems with Stationary Random Inputs	115	(10)
Time-Domain Analysis
Frequency-Domain Analysis
Random Signal Memory
General Correlation Matrices
Correlation Matrices from Random Processes
Whitening and Innovations Representation	125	(8)
Transformations Using Eigen-decomposition
Transformations Using Triangular Decomposition
The Discrete Karhunen-Loeve Transform
Principles of Estimation Theory	133	(9)
Properties of Estimators
Estimation of Mean
Estimation of Variance
Summary	142	(7)
Problems	143	(6)
Linear Signal Models	149	(46)
Introduction	149	(7)
Linear Nonparametric Signal Models
Parametric Pole-Zero Signal Models
Mixed Processes and the Wold Decomposition
All-Pole Models	156	(16)
Model Properties
All-Pole Modeling and Linear Prediction
Autoregressive Models
Lower-Order Models
All-Zero Models	172	(5)
Model Properties
Moving-Average Models
Lower-Order Models
Pole-Zero Models	177	(5)
Model Properties
Autoregressive Moving-Average Models
The First-Order Pole-Zero Model 1: PZ (1,1)
Summary and Dualities
Models with Poles on the Unit Circle	182	(2)
Cepstrum of Pole-Zero Models	184	(5)
Pole-Zero Models
All-Pole Models
All-Zero Models
Summary	189	(6)
Problems	189	(6)
Nonparametric Power Spectrum Estimation	195	(66)
Spectral Analysis of Deterministic Signals	196	(13)
Effect of Signal Sampling
Windowing, Periodic Extension, and Extrapolation
Effect of Spectrum Sampling
Effects of Windowing: Leakage and Loss of Resolution
Summary
Estimation of the Autocorrelation of Stationary Random Signals	209	(3)
Estimation of the Power Spectrum of Stationary Random Signals	212	(25)
Power Spectrum Estimation Using the Periodogram
Power Spectrum Estimation by Smoothing a Single Periodogram---The Blackman-Tukey Method
Power Spectrum Estimation by Averaging Multiple Periodograms---The Welch-Bartlett Method
Some Practical Considerations and Examples
Joint Signal Analysis	237	(9)
Estimation of Cross-Power Spectrum
Estimation of Frequency Response Functions
Multitaper Power Spectrum Estimation	246	(8)
Estimation of Auto Power Spectrum
Estimation of Cross Power Spectrum
Summary	254	(7)
Problems	255	(6)
Optimum Linear Filters	261	(72)
Optimum Signal Estimation	261	(3)
Linear Mean Square Error Estimation	264	(10)
Error Performance Surface
Derivation of the Linear MMSE Estimator
Principal-Component Analysis of the Optimum Linear Estimator
Geometric Interpretations and the Principle of Orthogonality
Summary and Further Properties
Solution of the Normal Equations	274	(4)
Optimum Finite Impulse Response Filters	278	(8)
Design and Properties
Optimum FIR Filters for Stationary Processes
Frequency-Domain Interpretations
Linear Prediction	286	(9)
Linear Signal Estimation
Forward Linear Prediction
Backward Linear Prediction
Stationary Processes
Properties
Optimum Infinite Impulse Response Filters	295	(11)
Noncausal IIR Filters
Causal IIR Filters
Filtering of Additive Noise
Linear Prediction Using the Infinite Past---Whitening
Inverse Filtering and Deconvolution	306	(4)
Channel Equalization in Data Transmission Systems	310	(9)
Nyquist's Criterion for Zero ISI
Equivalent Discrete-Time Channel Model
Linear Equalizers
Zero-Forcing Equalizers
Minimum MSE Equalizers
Matched Filters and Eigenfilters	319	(6)
Deterministic Signal in Noise
Random Signal in Noise
Summary	325	(8)
Problems	325	(8)
Algorithms and Structures for Optimum Linear Filters	333	(62)
Fundamentals of Order-Recursive Algorithms	334	(9)
Matrix Partitioning and Optimum Nesting
Inversion of Partitioned Hermitian Matrices
Levinson Recursion for the Optimum Estimator
Order-Recursive Computation of the LDLH Decomposition
Order-Recursive Computation of the Optimum Estimate
Interpretations of Algorithmic Quantities	343	(4)
Innovations and Backward Prediction
Partial Correlation
Order Decomposition of the Optimum Estimate
Gram-Schmidt Orthogonalization
Order-Recursive Algorithms for Optimum FIR Filters	347	(8)
Order-Recursive Computation of the Optimum Filter
Lattice-Ladder Structure
Simplifications for Stationary Stochastic Processes
Algorithms Based on the UDUH Decomposition
Algorithms of Levinson and Levinson-Durbin	355	(6)
Lattice Structures for Optimum FIR Filters and Predictors	361	(7)
Lattice-Ladder Structures
Some Properties and Interpretations
Parameter Conversions
Algorithm of Schur	368	(6)
Direct Schur Algorithm
Implementation Considerations
Inverse Schur Algorithm
Triangularization and Inversion of Toeplitz Matrices	374	(4)
LDLH Decomposition of Inverse of a Toeplitz Matrix
LDLH Decomposition of a Toeplitz Matrix
Inversion of Real Toeplitz Matrices
Kalman Filter Algorithm	378	(9)
Preliminary Development
Development of Kalman Filter
Summary	387	(8)
Problems	389	(6)
Least-Squares Filtering and Prediction	395	(50)
The Principle of Least Squares	395	(1)
Linear Least-Squares Error Estimation	396	(10)
Derivation of the Normal Equations
Statistical Properties of Least-Squares Estimators
Least-Squares FIR Filters	406	(5)
Linear Least-Squares Signal Estimation	411	(5)
Signal Estimation and Linear Prediction
Combined Forward and Backward Linear Prediction (FBLP)
Narrowband Interference Cancelation
LS Computations Using the Normal Equations	416	(6)
Linear LSE Estimation
LSE FIR Filtering and Prediction
LS Computations Using Orthogonalization Techniques	422	(9)
Householder Reflections
The Givens Rotations
Gram-Schmidt Orthogonalization
LS Computations Using the Singular Value Decomposition	431	(7)
Singular Value Decomposition
Solution of the LS Problem
Rank-Deficient LS Problems
Summary	438	(7)
Problems	439	(6)
Signal Modeling and Parametric Spectral Estimation	445	(54)
The Modeling Process: Theory and Practice	445	(4)
Estimation of All-Pole Models	449	(13)
Direct Structures
Lattice Structures
Maximum Entropy Method
Excitations with Line Spectra
Estimation of Pole-Zero Models	462	(5)
Known Excitation
Unknown Excitation
Nonlinear Least-Squares Optimization
Applications	467	(4)
Spectral Estimation
Speech Modeling
Minimum-Variance Spectrum Estimation	471	(7)
Harmonic Models and Frequency Estimation Techniques	478	(15)
Harmonic Model
Pisarenko Harmonic Decomposition
Music Algorithm
Minimum-Norm Method
Esprit Algorithm
Summary	493	(6)
Problems	494	(5)
Adaptive Filters	499	(122)
Typical Applications of Adaptive Filters	500	(6)
Echo Cancelation in Communications
Equalization of Data Communications Channels
Linear Predictive Coding
Noise Cancelation
Principles of Adaptive Filters	506	(10)
Features of Adaptive Filters
Optimum versus Adaptive Filters
Stability and Steady-State Performance of Adaptive Filters
Some Practical Considerations
Method of Steepest Descent	516	(8)
Least-Mean-Square Adaptive Filters	524	(24)
Derivation
Adaptation in a Stationary SOE
Summary and Design Guidelines
Applications of the LMS Algorithm
Some Practical Considerations
Recursive Least-Squares Adaptive Filters	548	(12)
LS Adaptive Filters
Conventional Recursive Least-Squares Algorithm
Some Practical Considerations
Convergence and Performance Analysis
RLS Algorithms for Array Processing	560	(13)
LS Computations Using the Cholesky and QR Decompositions
Two Useful Lemmas
The QR-RLS Algorithm
Extended QR-RLS Algorithm
The Inverse QR-RLS Algorithm
Implementation of QR-RLS Algorithm Using the Givens Rotations
Implementation of Inverse QR-RLS Algorithm Using the Givens Rotations
Classification of RLS Algorithms for Array Processing
Fast RLS Algorithms for FIR Filtering	573	(17)
Fast Fixed-Order RLS FIR Filters
RLS Lattice-Ladder Filters
RLS Lattice-Ladder Filters Using Error Feedback Updatings
Givens Rotation--Based LS Lattice-Ladder Algorithms
Classification of RLS Algorithms for FIR Filtering
Tracking Performance of Adaptive Algorithms	590	(17)
Approaches for Nonstationary SOE
Preliminaries in Performance Analysis
The LMS Algorithm
The RLS Algorithm with Exponential Forgetting
Comparison of Tracking Performance
Summary	607	(14)
Problems	608	(13)
Array Processing	621	(70)
Array Fundamentals	622	(9)
Spatial Signals
Modulation-Demodulation
Array Signal Model
The Sensor Array: Spatial Sampling
Conventional Spatial Filtering: Beamforming	631	(10)
Spatial Matched Filter
Tapered Beamforming
Optimum Array Processing	641	(11)
Optimum Beamforming
Eigenanalysis of the Optimum Beamformer
Interference Cancelation Performance
Tapered Optimum Beamforming
The Generalized Sidelobe Canceler
Performance Considerations for Optimum Beamformers	652	(7)
Effect of Signal Mismatch
Effect of Bandwidth
Adaptive Beamforming	659	(12)
Sample Matrix Inversion
Diagonal Loading with the SMI Beamformer
Implementation of the SMI Beamformer
Sample-by-Sample Adaptive Methods
Other Adaptive Array Processing Methods	671	(7)
Linearly Constrained Minimum-Variance Beamformers
Partially Adaptive Arrays
Sidelobe Cancelers
Angle Estimation	678	(5)
Maximum-Likelihood Angle Estimation
Cramer-Rao Lower Bound on Angle Accuracy
Beamsplitting Algorithms
Model-Based Methods
Space-Time Adaptive Processing	683	(2)
Summary	685	(6)
Problems	686	(5)
Further Topics	691	(54)
Higher-Order Statistics in Signal Processing	691	(6)
Moments, Cumulants, and Polyspectra
Higher-Order Moments and LTI Systems
Higher-Order Moments of Linear Signal Models
Blind Deconvolution	697	(5)
Unsupervised Adaptive Filters---Blind Equalizers	702	(7)
Blind Equalization
Symbol Rate Blind Equalizers
Constant-Modulus Algorithm
Fractionally Spaced Equalizers	709	(7)
Zero-Forcing Fractionally Spaced Equalizers
MMSE Fractionally Spaced Equalizers
Blind Fractionally Spaced Equalizers
Fractional Pole-Zero Signal Models	716	(9)
Fractional Unit-Pole Model
Fractional Pole-Zero Models: FPZ (p, d, q)
Symmetric α-Stable Fractional Pole-Zero Processes
Self-Similar Random Signal Models	725	(16)
Self-Similar Stochastic Processes
Fractional Brownian Motion
Fractional Gaussian Noise
Simulation of Fractional Brownian Motions and Fractional Gaussian Noises
Estimation of Long Memory
Fractional Levy Stable Motion
Summary	741	(4)
Problems	742	(3)
Appendix A Matrix Inversion Lemma	745	(2)
Appendix B Gradients and Optimization in Complex Space	747	(6)
Gradient	747	(2)
Lagrange Multipliers	749	(4)
Appendix C Matlab Functions	753	(2)
Appendix D Useful Results from Matrix Algebra	755	(12)
Complex-Valued Vector Space	755	(1)
Some Definitions
Matrices	756	(4)
Some Definitions
Properties of Square Matrices
Determinant of a Square Matrix	760	(2)
Properties of the Determinant
Condition Number
Unitary Matrices	762	(2)
Hermitian Forms after Unitary Transformations
Significant Integral of Quadratic and Hermitian Forms
Positive Definite Matrices	764	(3)
Appendix E Minimum Phase Test for Polynomials	767	(2)
Bibliography	769	(18)
Index	787