Advanced Digital Signal Processing and Noise Reduction, 3rd Edition

by Saeed V. Vaseghi (Brunel University, UK)

• ISBN13: 9780470094945
• ISBN10: 047009494X
• Edition: 3rd
• Format: Hardcover
• Copyright: 2006-01-01
• Publisher: WILEY

Summary

Signal processing plays an increasingly central role in the development of modern telecommunication and information processing systems, with a wide range of applications in areas such as multimedia technology, audio-visual signal processing, cellular mobile communication, radar systems and financial data forecasting. The theory and application of signal processing deals with the identification, modelling and utilisation of patterns and structures in a signal process. The observation signals are often distorted, incomplete and noisy and hence, noise reduction and the removal of channel distortion is an important part of a signal processing system.

Author Biography

Saeed Vaseghi is currently a Professor of Communications and Signal Processing at Brunel University’s department of Electronics and Computer Engineering and is Group Leader for the Communications & Multimedia Signal Processing Group.

Previously, Saeed obtained a first in Electrical and Electronics Engineering from Newcastle University, and a PhD in Digital Signal Processing from Cambridge University. His Ph.D. in noisy signal restoration led to establishment of CEDAR, the world's leading system for restoration of audio signals. Saeed also held a British Telecom lectureship at UEA Norwich, and a readership at Queen's University of Belfast before his move to Brunel.

Table of Contents

Preface	xvii
Symbols	xxi
Abbreviations	xxv
Introduction	1	(22)
Signals and Information	1	(2)
Signal Processing Methods	3	(2)
Transform-based Signal Processing	3	(1)
Model-based Signal Processing	4	(1)
Bayesian Signal Processing	4	(1)
Neural Networks	5	(1)
Applications of Digital Signal Processing	5	(12)
Adaptive Noise Cancellation	5	(1)
Adaptive Noise Reduction	6	(1)
Blind Channel Equalisation	7	(1)
Signal Classification and Pattern Recognition	8	(1)
Linear Prediction Modelling of Speech	9	(1)
Digital Coding of Audio Signals	10	(2)
Detection of Signals in Noise	12	(1)
Directional Reception of Waves: Beam-forming	13	(2)
Dolby Noise Reduction	15	(1)
Radar Signal Processing: Doppler Frequency Shift	15	(2)
Sampling and Analogue-to-digital Conversion	17	(6)
Sampling and Reconstruction of Analogue Signals	18	(1)
Quantisation	19	(2)
Bibliography	21	(2)
Noise and Distortion	23	(16)
Introduction	24	(1)
White Noise	25	(1)
Band-limited White Noise	26	(1)
Coloured Noise	26	(1)
Impulsive Noise	27	(2)
Transient Noise Pulses	29	(1)
Thermal Noise	30	(1)
Shot Noise	31	(1)
Electromagnetic Noise	31	(1)
Channel Distortions	32	(1)
Echo and Multipath Reflections	33	(1)
Modelling Noise	33	(6)
Additive White Gaussian Noise Model	36	(1)
Hidden Markov Model for Noise	36	(1)
Bibliography	37	(2)
Probability and Information Models	39	(54)
Introduction	40	(1)
Random Signals	41	(3)
Random and Stochastic Processes	43	(1)
The Space of a Random Process	43	(1)
Probability Models	44	(6)
Probability and Random Variables	45	(1)
Probability Mass Function	45	(2)
Probability Density Function	47	(1)
Probability Density Functions of Random Processes	48	(2)
Information Models	50	(9)
Entropy	51	(3)
Mutual Information	54	(2)
Entropy Coding	56	(3)
Stationary and Nonstationary Random Processes	59	(3)
Strict-sense Stationary Processes	61	(1)
Wide-sense Stationary Processes	61	(1)
Nonstationary Processes	62	(1)
Statistics (Expected Values) of a Random Process	62	(11)
The Mean Value	63	(1)
Autocorrelation	63	(3)
Autocovariance	66	(1)
Power Spectral Density	66	(2)
Joint Statistical Averages of Two Random Processes	68	(1)
Cross-correlation and Cross-covariance	68	(2)
Cross-power Spectral Density and Coherence	70	(1)
Ergodic Processes and Time-averaged Statistics	70	(1)
Mean-ergodic Processes	70	(2)
Correlation-ergodic Processes	72	(1)
Some Useful Classes of Random Processes	73	(10)
Gaussian (Normal) Process	73	(1)
Multivariate Gaussian Process	74	(1)
Mixture Gaussian Process	75	(1)
A Binary-state Gaussian Process	76	(1)
Poisson Process	77	(1)
Shot Noise	78	(1)
Poisson--Gaussian Model for Clutters and Impulsive Noise	79	(1)
Markov Processes	80	(1)
Markov Chain Processes	81	(1)
Gamma Probability Distribution	82	(1)
Rayleigh Probability Distribution	83	(1)
Laplacian Probability Distribution	83	(1)
Transformation of a Random Process	83	(7)
Monotonic Transformation of Random Processes	84	(2)
Many-to-one Mapping of Random Signals	86	(4)
Summary	90	(3)
Bibliography	90	(3)
Bayesian Inference	93	(44)
Bayesian Estimation Theory: Basic Definitions	94	(8)
Dynamic and Probability Models in Estimation	95	(1)
Parameter Space and Signal Space	96	(1)
Parameter Estimation and Signal Restoration	97	(1)
Performance Measures and Desirable Properties of Estimators	98	(2)
Prior and Posterior Spaces and Distributions	100	(2)
Bayesian Estimation	102	(14)
Maximum a Posteriori Estimation	103	(1)
Maximum-likelihood Estimation	104	(3)
Minimum Mean Square Error Estimation	107	(1)
Minimum Mean Absolute Value of Error Estimation	108	(1)
Equivalence of the MAP, ML, MMSE and MAVE for Gaussian Processes with Uniform Distributed Parameters	109	(1)
The Influence of the Prior on Estimation Bias and Variance	109	(5)
The Relative Importance of the Prior and the Observation	114	(2)
The Estimate-Maximise Method	116	(3)
Convergence of the EM Algorithm	117	(2)
Cramer-Rao Bound on the Minimum Estimator Variance	119	(2)
Cramer-Rao Bound for Random Parameters	120	(1)
Cramer-Rao Bound for a Vector Parameter	121	(1)
Design of Gaussian Mixture Models	121	(3)
EM Estimation of Gaussian Mixture Model	122	(2)
Bayesian Classification	124	(8)
Binary Classification	125	(2)
Classification Error	127	(1)
Bayesian Classification of Discrete-valued Parameters	128	(1)
Maximum a Posteriori Classification	128	(1)
Maximum-likelihood Classification	129	(1)
Minimum Mean Square Error Classification	129	(1)
Bayesian Classification of Finite State Processes	130	(1)
Bayesian Estimation of the Most Likely State Sequence	131	(1)
Modelling the Space of a Random Process	132	(2)
Vector Quantisation of a Random Process	132	(1)
Vector Quantisation using Gaussian Models	133	(1)
Design of a Vector Quantiser: K-means Clustering	133	(1)
Summary	134	(3)
Bibliography	135	(2)
Hidden Markov Models	137	(28)
Statistical Models for Nonstationary Processes	138	(1)
Hidden Markov Models	139	(6)
Comparison of Markov and Hidden Markov Models	139	(2)
A Physical Interpretation: HMMs of Speech	141	(1)
Hidden Markov Model as a Bayesian Model	142	(1)
Parameters of a Hidden Markov Model	143	(1)
State Observation Probability Models	143	(1)
State Transition Probabilities	144	(1)
State-Time Trellis Diagram	145	(1)
Training Hidden Markov Models	145	(7)
Forward-Backward Probability Computation	147	(1)
Baum-Welch Model Re-estimation	148	(1)
Training HMMs with Discrete Density Observation Models	149	(1)
HMMs with Continuous Density Observation Models	150	(1)
HMMs with Gaussian Mixture pdfs	151	(1)
Decoding of Signals using Hidden Markov Models	152	(3)
Viterbi Decoding Algorithm	154	(1)
HMMs in DNA and Protein Sequence Modelling	155	(1)
HMMs for Modelling Speech and Noise	156	(6)
Modelling Speech with HMMs	156	(1)
HMM-based Estimation of Signals in Noise	156	(2)
Signal and Noise Model Combination and Decomposition	158	(1)
Hidden Markov Model Combination	159	(1)
Decomposition of State Sequences of Signal and Noise	160	(1)
HMM-based Wiener Filters	160	(2)
Modelling Noise Characteristics	162	(1)
Summary	162	(3)
Bibliography	163	(2)
Least Square Error Filters	165	(22)
Least Square Error Estimation: Wiener Filters	166	(4)
Block-data Formulation of the Wiener Filter	170	(2)
QR Decomposition of the Least Square Error Equation	171	(1)
Interpretation of Wiener Filters as Projections in Vector Space	172	(2)
Analysis of the Least Mean Square Error Signal	174	(1)
Formulation of Wiener Filters in the Frequency Domain	175	(2)
Some Applications of Wiener Filters	177	(5)
Wiener Filters for Additive Noise Reduction	177	(1)
Wiener Filters and Separability of Signal and Noise	178	(1)
The Square-root Wiener Filter	179	(1)
Wiener Channel Equaliser	180	(1)
Time-alignment of Signals in Multichannel/Multisensor Systems	181	(1)
Implementation of Wiener Filters	182	(3)
The Choice of Wiener Filter Order	183	(1)
Improvements to Wiener Filters	184	(1)
Summary	185	(2)
Bibliography	185	(2)
Adaptive Filters	187	(22)
Introduction	188	(1)
State-space Kalman Filters	188	(7)
Derivation of the Kalman Filter Algorithm	190	(5)
Sample-adaptive Filters	195	(1)
Recursive Least Square Adaptive Filters	196	(5)
The Matrix Inversion Lemma	198	(1)
Recursive Time-update of Filter Coefficients	199	(2)
The Steepest-descent Method	201	(3)
Convergence Rate	203	(1)
Vector-valued Adaptation Step Size	204	(1)
The LMS Filter	204	(3)
Leaky LMS Algorithm	205	(1)
Normalised LMS Algorithm	206	(1)
Summary	207	(2)
Bibliography	208	(1)
Linear Prediction Models	209	(32)
Linear Prediction Coding	210	(9)
Frequency Response of LP Models	213	(1)
Calculation of Predictor Coefficients	214	(2)
Effect of Estimation of Correlation Function on LP Model Solution	216	(1)
The Inverse Filter: Spectral Whitening	216	(1)
The Prediction Error Signal	217	(2)
Forward, Backward and Lattice Predictors	219	(7)
Augmented Equations for Forward and Backward Predictors	220	(1)
Levinson-Durbin Recursive Solution	221	(2)
Lattice Predictors	223	(1)
Alternative Formulations of Least Square Error Prediction	224	(1)
Predictor Model Order Selection	225	(1)
Short- and Long-term Predictors	226	(2)
MAP Estimation of Predictor Coefficients	228	(2)
Probability Density Function of Predictor Output	229	(1)
Using the Prior pdf of the Predictor Coefficients	230	(1)
Formant-tracking LP Models	230	(2)
Sub-band Linear Prediction Model	232	(1)
Signal Restoration using Linear Prediction Models	233	(5)
Frequency-domain Signal Restoration using Prediction Models	235	(2)
Implementation of Sub-band Linear Prediction Wiener Filters	237	(1)
Summary	238	(3)
Bibliography	238	(3)
Power Spectrum and Correlation	241	(26)
Power Spectrum and Correlation	242	(1)
Fourier Series: Representation of Periodic Signals	243	(2)
Fourier Transform: Representation of Aperiodic Signals	245	(4)
Discrete Fourier Transform	246	(1)
Time/Frequency Resolutions, the Uncertainty Principle	247	(1)
Energy-spectral Density and Power-spectral Density	248	(1)
Nonparametric Power Spectrum Estimation	249	(5)
The Mean and Variance of Periodograms	250	(1)
Averaging Periodograms (Bartlett Method)	250	(1)
Welch Method: Averaging Periodograms from Overlapped and Windowed Segments	251	(1)
Blackman-Tukey Method	252	(1)
Power Spectrum Estimation from Autocorrelation of Overlapped Segments	253	(1)
Model-based Power Spectrum Estimation	254	(5)
Maximum-entropy Spectral Estimation	255	(2)
Autoregressive Power Spectrum Estimation	257	(1)
Moving-average Power Spectrum Estimation	257	(1)
Autoregressive Moving-average Power Spectrum Estimation	258	(1)
High-resolution Spectral Estimation Based on Subspace Eigenanalysis	259	(6)
Pisarenko Harmonic Decomposition	259	(2)
Multiple Signal Classification Spectral Estimation	261	(3)
Estimation of Signal Parameters via Rotational Invariance Techniques	264	(1)
Summary	265	(2)
Bibliography	266	(1)
Interpolation	267	(30)
Introduction	268	(6)
Interpolation of a Sampled Signal	268	(1)
Digital Interpolation by a Factor of I	269	(2)
Interpolation of a Sequence of Lost Samples	271	(2)
The Factors that affect Interpolation Accuracy	273	(1)
Polynomial Interpolation	274	(6)
Lagrange Polynomial Interpolation	275	(1)
Newton Polynomial Interpolation	276	(2)
Hermite Polynomial Interpolation	278	(1)
Cubic Spline Interpolation	278	(2)
Model-based Interpolation	280	(14)
Maximum a Posteriori Interpolation	281	(1)
Least Square Error Autoregressive Interpolation	282	(1)
Interpolation based on a Short-term Prediction Model	283	(3)
Interpolation based on Long- and Short-term Correlations	286	(3)
LSAR Interpolation Error	289	(1)
Interpolation in Frequency--Time Domain	290	(3)
Interpolation using Adaptive Codebooks	293	(1)
Interpolation through Signal Substitution	294	(1)
Summary	294	(3)
Bibliography	295	(2)
Spectral Amplitude Estimation	297	(22)
Introduction	298	(2)
Spectral Representation of Noisy Signals	299	(1)
Vector Representation of the Spectrum of Noisy Signals	299	(1)
Spectral Subtraction	300	(12)
Power Spectrum Subtraction	302	(1)
Magnitude Spectrum Subtraction	303	(1)
Spectral Subtraction Filter: Relation to Wiener Filters	303	(1)
Processing Distortions	304	(1)
Effect of Spectral Subtraction on Signal Distribution	305	(1)
Reducing the Noise Variance	306	(1)
Filtering Out the Processing Distortions	307	(1)
Nonlinear Spectral Subtraction	308	(2)
Implementation of Spectral Subtraction	310	(2)
Bayesian MMSE Spectral Amplitude Estimation	312	(3)
Application to Speech Restoration and Recognition	315	(1)
Summary	315	(4)
Bibliography	316	(3)
Impulsive Noise	319	(18)
Impulsive Noise	320	(3)
Autocorrelation and Power Spectrum of Impulsive Noise	322	(1)
Statistical Models for Impulsive Noise	323	(4)
Bernoulli--Gaussian Model of Impulsive Noise	324	(1)
Poisson--Gaussian Model of Impulsive Noise	324	(1)
A Binary-state Model of Impulsive Noise	325	(1)
Signal-to-impulsive-noise Ratio	326	(1)
Median Filters	327	(1)
Impulsive Noise Removal using Linear Prediction Models	328	(5)
Impulsive Noise Detection	328	(2)
Analysis of Improvement in Noise Detectability	330	(1)
Two-sided Predictor for Impulsive Noise Detection	331	(1)
Interpolation of Discarded Samples	332	(1)
Robust Parameter Estimation	333	(1)
Restoration of Archived Gramophone Records	334	(1)
Summary	335	(2)
Bibliography	336	(1)
Transient Noise Pulses	337	(14)
Transient Noise Waveforms	337	(2)
Transient Noise Pulse Models	339	(3)
Noise Pulse Templates	340	(1)
Autoregressive Model of Transient Noise Pulses	341	(1)
Hidden Markov Model of a Noise Pulse Process	342	(1)
Detection of Noise Pulses	342	(3)
Matched Filter for Noise Pulse Detection	343	(1)
Noise Detection based on Inverse Filtering	344	(1)
Noise Detection based on HMM	344	(1)
Removal of Noise Pulse Distortions	345	(4)
Adaptive Subtraction of Noise Pulses	345	(2)
AR-based Restoration of Signals Distorted by Noise Pulses	347	(2)
Summary	349	(2)
Bibliography	349	(2)
Echo Cancellation	351	(20)
Introduction: Acoustic and Hybrid Echoes	352	(1)
Telephone Line Hybrid Echo	353	(2)
Echo: the Sources of Delay in Telephone Networks	354	(1)
Echo Return Loss	355	(1)
Hybrid Echo Suppression	355	(1)
Adaptive Echo Cancellation	356	(4)
Echo Canceller Adaptation Methods	357	(1)
Convergence of Line Echo Canceller	358	(1)
Echo Cancellation for Digital Data Transmission	359	(1)
Acoustic Echo	360	(3)
Sub-band Acoustic Echo Cancellation	363	(2)
Multiple-input Multiple-output Echo Cancellation	365	(3)
Stereophonic Echo Cancellation Systems	365	(3)
Summary	368	(3)
Bibliography	368	(3)
Channel Equalisation and Blind Deconvolution	371	(38)
Introduction	372	(7)
The Ideal Inverse Channel Filter	373	(1)
Equalisation Error, Convolutional Noise	374	(1)
Blind Equalisation	374	(2)
Minimum- and Maximum-phase Channels	376	(1)
Wiener Equaliser	377	(2)
Blind Equalisation using the Channel Input Power Spectrum	379	(3)
Homomorphic Equalisation	380	(2)
Homomorphic Equalisation using a Bank of High-pass Filters	382	(1)
Equalisation based on Linear Prediction Models	382	(3)
Blind Equalisation through Model Factorisation	384	(1)
Bayesian Blind Deconvolution and Equalisation	385	(8)
Conditional Mean Channel Estimation	386	(1)
Maximum-likelihood Channel Estimation	386	(1)
Maximum a Posteriori Channel Estimation	386	(1)
Channel Equalisation based on Hidden Markov Models	387	(2)
MAP Channel Estimate based on HMMs	389	(1)
Implementations of HMM-based Deconvolution	390	(3)
Blind Equalisation for Digital Communications Channels	393	(5)
LMS Blind Equalisation	395	(2)
Equalisation of a Binary Digital Channel	397	(1)
Equalisation based on Higher-order Statistics	398	(8)
Higher-order Moments, Cumulants and Spectra	399	(2)
Higher-order Spectra of Linear Time-invariant Systems	401	(1)
Blind Equalisation based on Higher-order Cepstra	402	(4)
Summary	406	(3)
Bibliography	406	(3)
Speech Enhancement in Noise	409	(24)
Introduction	410	(1)
Single-input Speech-enhancement Methods	411	(14)
An Overview of a Speech-enhancement System	411	(3)
Wiener Filter for De-noising Speech	414	(3)
Spectral Subtraction of Noise	417	(1)
Bayesian MMSE Speech Enhancement	418	(1)
Kalman Filter	419	(3)
Speech Enhancement via LP Model Reconstruction	422	(3)
Multiple-input Speech-enhancement Methods	425	(5)
Beam-forming with Microphone Arrays	427	(3)
Speech Distortion Measurements	430	(3)
Bibliography	431	(2)
Noise in Wireless Communications	433	(16)
Introduction to Cellular Communications	434	(2)
Noise, Capacity and Spectral Efficiency	436	(2)
Communications Signal Processing in Mobile Systems	438	(1)
Noise and Distortion in Mobile Communications Systems	439	(5)
Multipath Propagation of Electromagnetic Signals	440	(1)
Rake Receivers for Multipath Signals	441	(1)
Signal Fading in Mobile Communications Systems	442	(1)
Large-scale Signal Fading	443	(1)
Small-scale Fast Signal Fading	444	(1)
Smart Antennas	444	(3)
Switched and Adaptive Smart Antennas	446	(1)
Space-Time Signal Processing -- Diversity Schemes	446	(1)
Summary	447	(2)
Bibliography	448	(1)
Index	449