Random Signals Detection, Estimation and Data Analysis

by Shanmugan, K. Sam; Breipohl, Arthur M.

• ISBN13: 9780471815556
• ISBN10: 0471815551
• Edition: 1st
• Format: Paperback
• Copyright: 1991-01-16
• Publisher: Wiley

Summary

Random Signals, Noise and Filtering develops the theory of random processes and its application to the study of systems and analysis of random data. The text covers three important areas: (1) fundamentals and examples of random process models, (2) applications of probabilistic models: signal detection, and filtering, and (3) statistical estimation--measurement and analysis of random data to determine the structure and parameter values of probabilistic models. This volume by Breipohl and Shanmugan offers the only one-volume treatment of the fundamentals of random process models, their applications, and data analysis.

Author Biography

K. Sam Shanmugan and Arthur M. Breipohl are the authors of Random Signals: Detection, Estimation and Data Analysis, published by Wiley.

Table of Contents

Introduction
Historical Perspective	3	(1)
Outline of the Book	4	(3)
References	7	(1)
Review of Probability and Random Variables
Introduction	8	(1)
Probability	9	(12)
Set Definitions	9	(3)
Sample Space	12	(1)
Probabilities of Random Events	12	(2)
Useful Laws of Probability	14	(1)
Joint, Marginal, and Conditional Probabilities	15	(6)
Random Variables	21	(12)
Distribution Functions	22	(2)
Discrete Random Variables and Probability Mass Function	24	(2)
Expected Values or Averages	26	(3)
Examples of Probability Mass Functions	29	(4)
Continuous Random Variables	33	(14)
Probability Density Functions	33	(10)
Examples of Probability Density Functions	43	(3)
Complex Random Variables	46	(1)
Random Vectors	47	(8)
Multivariate Gaussian Distribution	50	(1)
Properties of the Multivariate Gaussian Distribution	50	(3)
Moments of Multivariate Gaussian pdf	53	(2)
Transformations (Functions) of Random Variables	55	(21)
Scalar Valued Function of One Random Variable	57	(4)
Functions of Several Random Variables	61	(15)
Bounds and Approximations	76	(12)
Thebycheff Inequality	77	(1)
Chernoff Bound	78	(1)
Union Bound	79	(2)
Approximating the Distribution of Y = g(X1...,Xn)	81	(2)
Series Approximation of Probability Density Functions	83	(4)
Approximations of Gaussian Probabilities	87	(1)
Sequences of Random Variables and Convergence	88	(7)
Convergence Everywhere and Almost Everywhere	88	(1)
Convergence in Distribution and Central Limit Theorem	89	(4)
Convergence in Probability (in Measure) and the Law of Large Numbers	93	(1)
Convergence in Mean Square	94	(1)
Relationship Between Different Forms of Convergence	95	(1)
Summary	95	(1)
References	96	(1)
Problems	97	(14)
Random Processes and Sequences
Introduction	111	(2)
Definition of Random Processes	113	(6)
Concept of Random Processes	113	(1)
Notation	114	(2)
Probabilistic Structure	116	(1)
Classification of Random Processes	117	(2)
Formal Definition of Random Processes	119	(1)
Methods of Description	119	(6)
Joint Distribution	119	(2)
Analytical Description Using Random Variables	121	(1)
Average Values	121	(3)
Two or More Random Processes	124	(1)
Special Classes of Random Processes	125	(10)
More Definitions	126	(1)
Random Walk and Wiener Process	127	(4)
Poisson Process	131	(1)
Random Binary Waveform	132	(3)
Stationarity	135	(7)
Strict-sense Stationarity	135	(1)
Wide-sense Stationarity	136	(1)
Examples	137	(4)
Other Forms of Stationarity	141	(1)
Tests for Stationarity	142	(1)
Autocorrelation and Power Spectral Density Functions of Real WSS Random Processes	142	(18)
Autocorrelation Function of a Real WSS Random Process and Its Properties	143	(1)
Cross correlation Function and its Properties	144	(1)
Power Spectral Density Function of a WSS Random Process and Its Properties	145	(3)
Cross-power Spectral Density Function and Its Properties	148	(1)
Power Spectral Density Function of Random Sequences	149	(11)
Continuity, Differentiation, and Integration	160	(6)
Continuity	161	(1)
Differentiation	162	(3)
Integration	165	(1)
Time Averaging and Ergodicity	166	(19)
Time Averages	168	(8)
Ergodicity	176	(9)
Spectral Decomposition and Series Expansion of Random Processes	185	(4)
Ordinary Fourier Series Expansion	185	(2)
Modified Fourier Series for Aperiodic Random Signals	187	(1)
Karhunen-Loeve (K-L) Series Expansion	188	(1)
Sampling and Quantization of Random Signals	189	(13)
Sampling of Lowpass Random Signals	190	(6)
Quantization	196	(1)
Uniform Quantizing	197	(3)
Nonuniform Quantizing	200	(2)
Summary	202	(1)
References	203	(1)
Problems	204	(12)
Response of Linear Systems to Random Inputs
Classification of Systems	216	(2)
Lumped Linear Time-invariant Causal (LLTIVC) System	216	(1)
Memoryless Nonlinear Systems	217	(1)
Response of LTIVC Discrete Time Systems	218	(9)
Review of Deterministic System Analysis	218	(3)
Mean and Autocorrelation of the Output	221	(1)
Distribution Functions	222	(1)
Stationarity of the Output	222	(1)
Correlation and Power Spectral Density of the Output	223	(4)
Response of LTIVC Continuous Time Systems	227	(15)
Mean and Autocorrelation Function of the Output	228	(1)
Stationarity of the Output	229	(1)
Power Spectral Density of the Output	230	(4)
Mean-square Value of the Output	234	(4)
Multiple Input-Output Systems	238	(1)
Filters	239	(3)
Summary	242	(1)
References	243	(1)
Problems	244	(5)
Special Classes of Random Processes
Introduction	249	(1)
Discrete Linear Models	250	(26)
Autoregressive Processes	250	(12)
Partial Autocorrelation Coefficient	262	(3)
Moving Average Models	265	(6)
Autoregressive Moving Average Models	271	(4)
Summary of Discrete Linear Models	275	(1)
Markov Sequences and Processes	276	(19)
Analysis of Discrete-time Markov Chains	278	(11)
Continuous-time Markov Chains	289	(6)
Summary of Markov Models	295	(1)
Point Processes	295	(17)
Poisson Process	298	(5)
Application of Poisson Process---Analysis of Queues	303	(4)
Shot Noise	307	(5)
Summary of Point Processes	312	(1)
Gaussian Processes	312	(19)
Definition of Gaussian Process	313	(1)
Models of White and Band-limited Noise	314	(3)
Response of Linear Time-invariant Systems	317	(1)
Quadrature Representation of Narrowband (Gaussian) Processes	317	(5)
Effects of Noise in Analog Communication Systems	322	(8)
Noise in Digital Communication Systems	330	(1)
Summary of Noise Models	331	(1)
Summary	331	(1)
References	332	(1)
Problems	333	(8)
Signal Detection
Introduction	341	(2)
Binary Detection with a Single Observation	343	(9)
Decision Theory and Hypothesis Testing	344	(1)
MAP Decision Rule and Types of Errors	345	(3)
Bayes' Decision Rule---Costs of Errors	348	(3)
Other Decision Rules	351	(1)
Binary Detection with Multiple Observations	352	(12)
Independent Noise Samples	353	(2)
White Noise and Continuous Observations	355	(6)
Colored Noise	361	(3)
Detection of Signals with Unknown Parameters	364	(2)
M-ary Detection	366	(3)
Summary	369	(1)
References	370	(1)
Problems	370	(7)
Linear Minimum Mean-Square Error Filtering
Introduction	377	(2)
Linear Minimum Mean Squared Error Estimators	379	(18)
Estimating a Random Variable with a Constant	379	(1)
Estimating S with One Observation X	379	(4)
Vector Space Representation	383	(1)
Multivariable Linear Mean Squared Error Estimation	384	(7)
Limitations of Linear Estimators	391	(2)
Nonlinear Minimum Mean Squared Error Estimators	393	(2)
Jointly Gaussian Random Variables	395	(2)
Innovations	397	(9)
Multivariate Estimator Using Innovations	400	(1)
Matrix Definition of Innovations	401	(5)
Review	406	(1)
Digital Wiener Filters	407	(12)
Digital Wiener Filters with Stored Data	407	(4)
Real-time Digital Wiener Filters	411	(8)
Kalman Filters	419	(23)
Recursive Estimators	420	(1)
Scalar Kalman Filter	421	(11)
Vector Kalman Filter	432	(10)
Wiener Filters	442	(23)
Stored Data (Unrealizable Filters)	444	(4)
Real-time or Realizable Filters	448	(17)
Summary	465	(1)
References	466	(1)
Problems	467	(8)
Statistics
Introduction	475	(1)
Measurements	476	(3)
Definition of a Statistic	478	(1)
Parametric and Nonparametric Estimators	478	(1)
Nonparametric Estimators of Probability Distribution and Density Functions	479	(6)
Definition of the Empirical Distribution Function	479	(1)
Joint Empirical Distribution Functions	480	(1)
Histograms	481	(3)
Parzen's Estimator for a pdf	484	(1)
Point Estimators of Parameters	485	(8)
Estimators of the Mean	486	(1)
Estimators of the Variance	486	(1)
An Estimator of Probability	487	(1)
Estimators of the Covariance	487	(1)
Notation for Estimators	487	(1)
Maximum Likelihood Estimators	488	(3)
Bayesian Estimators	491	(2)
Measures of the Quality of Estimators	493	(10)
Bias	493	(3)
Minimum Variance, Mean Squared Error, RMS Error, and Normalized Errors	496	(1)
The Bias, Variance, and Normalized RMS Errors of Histograms	497	(4)
Bias and Variance of Parzen's Estimator	501	(1)
Consistent Estimators	502	(1)
Efficient Estimators	502	(1)
Brief Introduction to Interval Estimates	503	(1)
Distribution of Estimators	504	(9)
Distribution of X with Known Variance	504	(1)
Chi-square Distribution	505	(3)
(Student's) t Distribution	508	(2)
Distribution of S2 and X with Unknown Variance	510	(1)
F Distribution	511	(2)
Tests of Hypotheses	513	(16)
Binary Detection	514	(3)
Composite Alternative Hypothesis	517	(1)
Tests of the Mean of a Normal Random Variable	518	(2)
Tests of the Equality of Two Means	520	(2)
Tests of Variances	522	(1)
Chi-Square Tests	523	(5)
Summary of Hypothesis Testing	528	(1)
Simple Linear Regression	529	(11)
Analyzing the Estimated Regression	536	(2)
Goodness of Fit Test	538	(2)
Multiple Linear Regression	540	(7)
Two Controlled Variables	541	(1)
Simple Linear Regression in Matrix Form	542	(1)
General Linear Regression	543	(2)
Goodness of Fit Test	545	(1)
More General Linear Models	545	(2)
Summary	547	(1)
References	548	(1)
Appendix 8-A	549	(3)
Problems	552	(8)
Estimating the Parameters of Random Processes from Data
Introduction	560	(1)
Tests for Stationarity and Ergodicity	561	(4)
Stationarity Tests	562	(1)
Run Test for Stationarity	562	(3)
Model-free Estimation	565	(19)
Mean Value Estimation	565	(1)
Autocorrelation Function Estimation	566	(3)
Estimation of the Power Spectral Density (psd) Functions	569	(10)
Smoothing of Spectral Estimates	579	(5)
Bias and Variance of Smoothed Estimators	584	(1)
Model-based Estimation of Autocorrelation Functions and Power Spectral Density Functions	584	(29)
Preprocessing (Differencing)	587	(3)
Order Identification	590	(4)
Estimating the Parameters of Autoregressive Processes	594	(6)
Estimating the Parameters of Moving Average Processes	600	(5)
Estimating the Parameters of ARMA (p, q) Processes	605	(1)
ARIMA Preliminary Parameter Estimation	606	(2)
Diagnostic Checking	608	(5)
Summary	613	(1)
References	614	(1)
Problems	615	(11)
APPENDIXES
A. Fourier Transforms	626	(2)
B. Discrete Fourier Transforms	628	(2)
C. Z Transforms	630	(2)
D. Gaussian Probabilities	632	(1)
E. Table of Chi-Square Distributions	633	(4)
F. Table of Student's t Distribution	637	(2)
G. Table of F Distributions	639	(10)
H. Percentage Points of Run Distribution	649	(1)
I. Critical Values of the Durbin-Watson Statistic	650	(1)
Index	651