Signals, Systems and Inference

by Oppenheim, Alan V.; Verghese, George C.

• ISBN13: 9780133943283
• ISBN10: 0133943283
• Edition: 1st
• Format: Hardcover
• Copyright: 2015-04-01
• Publisher: Pearson

Summary

For upper-level undergraduate courses in deterministic and stochastic signals and system engineering

An Integrative Approach to Signals, Systems and Inference

Signals, Systems and Inference is a comprehensive text that builds on introductory courses in time- and frequency-domain analysis of signals and systems, and in probability. Directed primarily to upper-level undergraduates and beginning graduate students in engineering and applied science branches, this new textbook pioneers a novel course of study. Instead of the usual leap from broad introductory subjects to highly specialized advanced subjects, this engaging and inclusive text creates a study track for a transitional course. Properties and representations of deterministic signals and systems are reviewed and elaborated on, including group delay and the structure and behavior of state-space models.

The text also introduces and interprets correlation functions and power spectral densities for describing and processing random signals. Application contexts include pulse amplitude modulation, observer-based feedback control, optimum linear filters for minimum mean-square-error estimation, and matched filtering for signal detection. Model-based approaches to inference are emphasized, in particular for state estimation, signal estimation, and signal detection. The text explores ideas, methods and tools common to numerous fields involving signals, systems and inference: signal processing, control, communication, time-series analysis, financial engineering, biomedicine, and many others. Signals, Systems, and Inference is a long-awaited and flexible text that can be used for a rigorous course in a broad range of engineering and applied science curricula.

Table of Contents

1 Introduction 11

2 Signals and Systems 13

2.1 Signals, Systems, Models, Properties . . . . . . . . . . . . . . . . . . 13

2.1.1 System Properties . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2 Linear, Time-Invariant Systems . . . . . . . . . . . . . . . . . . . . . 17

2.2.1 Impulse-Response Representation of LTI Systems . . . . . . . 17

2.2.2 Eigenfunction and Transform Representation of LTI Systems 19

2.2.3 Fourier Transforms . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3 Deterministic Signals and their Fourier Transforms . . . . . . . . . . 23

2.3.1 Signal Classes and their Fourier Transforms . . . . . . . . . . 23

2.3.2 Parseval’s Identity, Energy Spectral Density, Deterministic Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.4 The Bilateral Laplace and Z-Transforms . . . . . . . . . . . . . . . . 29

2.4.1 The Bilateral Z-Transform . . . . . . . . . . . . . . . . . . . 29

2.4.2 The Inverse Z-Transform . . . . . . . . . . . . . . . . . . . . 32

2.4.3 The Bilateral Laplace Transform . . . . . . . . . . . . . . . . 33

2.5 Discrete-Time Processing of Continuous-Time Signals . . . . . . . . 34

2.5.1 Basic Structure for DT Processing of CT Signals . . . . . . . 34

2.5.2 DT Filtering, and Overall CT Response . . . . . . . . . . . . 37

2.5.3 Non-Ideal D/C converters . . . . . . . . . . . . . . . . . . . . 39

2.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.6.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.6.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 57

2.6.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 70

3 Frequency Domain Amplitude, Phase and Group Delay 81

3.1 Fourier Transform Magnitude and Phase . . . . . . . . . . . . . . . . 81

3.2 Group Delay and The Effect of Nonlinear Phase . . . . . . . . . . . 85

3.2.1 Narrowband Input Signals . . . . . . . . . . . . . . . . . . . . 85

3.2.2 Broadband Input Signals . . . . . . . . . . . . . . . . . . . . 87

3.3 All-Pass and Minimum-Phase Systems . . . . . . . . . . . . . . . . . 91

3.3.1 All-Pass Systems . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.3.2 Minimum-Phase Systems . . . . . . . . . . . . . . . . . . . . 94

3.3.3 The Group Delay of Minimum-Phase Systems . . . . . . . . . 94

3.4 Spectral Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.5.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.5.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 109

3.5.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 122

4 Pulse Amplitude Modulation 125

4.1 Baseband Pulse Amplitude Modulation . . . . . . . . . . . . . . . . 125

4.1.1 The Transmitted Signal . . . . . . . . . . . . . . . . . . . . . 126

4.1.2 The Received Signal . . . . . . . . . . . . . . . . . . . . . . . 127

4.1.3 Frequency-Domain Characterizations . . . . . . . . . . . . . . 128

4.1.4 Inter-Symbol Interference at the Receiver . . . . . . . . . . . 131

4.2 Nyquist Pulses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

4.3 Passband Pulse Amplitude Modulation . . . . . . . . . . . . . . . . . 136

4.3.1 Frequency-Shift Keying (FSK) . . . . . . . . . . . . . . . . . 137

4.3.2 Phase-Shift Keying (PSK) . . . . . . . . . . . . . . . . . . . . 137

4.3.3 Quadrature Amplitude Modulation . . . . . . . . . . . . . . . 139

4.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4.4.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4.4.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 149

4.4.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 153

5 State-Space Models 163

5.1 System Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

5.2 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

5.3 State-Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

5.3.1 DT State-Space Models . . . . . . . . . . . . . . . . . . . . . 177

5.3.2 CT State-Space Models . . . . . . . . . . . . . . . . . . . . . 180

5.3.3 Defining Properties of State-Space Models . . . . . . . . . . . 183

5.4 State-Space Models from LTI Input—Output Models . . . . . . . . . 185

5.5 Equilibria and Linearization of Nonlinear State-Space Models . . . . 191

5.5.1 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

5.5.2 Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

5.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

5.6.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 199

5.6.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 204

5.6.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 207

6 LTI State-Space Models 211

6.1 Continuous-Time and Discrete-Time LTI Models . . . . . . . . . . . 211

6.2 Zero-Input Response and Modal Representation . . . . . . . . . . . . 214

6.2.1 Undriven CT Systems . . . . . . . . . . . . . . . . . . . . . . 214

6.2.2 Undriven DT Systems . . . . . . . . . . . . . . . . . . . . . . 221

6.2.3 Asymptotic Stability of LTI Systems . . . . . . . . . . . . . . 224

6.3 General Response in Modal Coordinates . . . . . . . . . . . . . . . . 229

6.3.1 Driven CT Systems . . . . . . . . . . . . . . . . . . . . . . . 229

6.3.2 Driven DT Systems . . . . . . . . . . . . . . . . . . . . . . . 231

6.3.3 Similarity Transformations and Diagonalization . . . . . . . . 234

6.4 Transfer Functions, Hidden Modes, Reachability, Observability . . . 240

6.4.1 Input-State-Output Structure of CT Systems . . . . . . . . . 241

6.4.2 Input-State-Output Structure of DT Systems . . . . . . . . . 249

6.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

6.5.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 260

6.5.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 270

6.5.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 276

7 State Observers and State Feedback 281

7.1 Plant and Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

7.2 State Estimation and Observers . . . . . . . . . . . . . . . . . . . . . 283

7.2.1 Real-Time Simulation . . . . . . . . . . . . . . . . . . . . . . 284

7.2.2 The State Observer . . . . . . . . . . . . . . . . . . . . . . . 286

7.2.3 Observer Design . . . . . . . . . . . . . . . . . . . . . . . . . 287

7.3 State Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . 298

7.3.1 Open-Loop Control . . . . . . . . . . . . . . . . . . . . . . . . 299

7.3.2 Closed-Loop Control via LTI State Feedback . . . . . . . . . 299

7.3.3 LTI State Feedback Design . . . . . . . . . . . . . . . . . . . 301

7.4 Observer-Based Feedback Control . . . . . . . . . . . . . . . . . . . . 308

7.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

7.5.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 314

7.5.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 323

7.5.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 326

8 Probabilistic Models 329

8.1 The Basic Probability Model . . . . . . . . . . . . . . . . . . . . . . 329

8.2 Conditional Probability, Bayes’ Rule, and Independence . . . . . . . 330

8.3 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

8.4 Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 333

8.5 Jointly Distributed Random Variables . . . . . . . . . . . . . . . . . 335

8.6 Expectations, Moments and Variance . . . . . . . . . . . . . . . . . . 337

8.7 Correlation and Covariance for Bivariate Random Variables . . . . . 340

8.8 A Vector-Space Interpretation of Correlation Properties . . . . . . . 345

8.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347

8.9.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 347

8.9.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 349

8.9.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 354

9 Estimation 359

9.1 Estimation of a Continuous Random Variable . . . . . . . . . . . . . 359

9.2 From Estimates to the Estimator . . . . . . . . . . . . . . . . . . . . 365

9.2.1 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

9.3 Linear Minimum Mean Square Error Estimation . . . . . . . . . . . 371

9.3.1 Linear Estimation of One Random Variable From a Single

Measurement of Another . . . . . . . . . . . . . . . . . . . . 371

9.3.2 Multiple Measurements . . . . . . . . . . . . . . . . . . . . . 376

9.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

9.4.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 381

9.4.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 387

9.4.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 395

10 Hypothesis Testing 401

10.1 Binary Pulse Amplitude Modulation in Noise . . . . . . . . . . . . . 401

10.2 Hypothesis Testing with Minimum Error Probability . . . . . . . . . 403

10.2.1 Deciding with Minimum Conditional Probability of Error . . 404

10.2.2 MAP Decision Rule for Minimum Overall Probability of Error 405

10.2.3 Hypothesis Testing in Coded Digital Communication . . . . . 408

10.3 Binary Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . 411

10.3.1 False Alarm, Miss, and Detection . . . . . . . . . . . . . . . . 413

10.3.2 The Likelihood Ratio Test . . . . . . . . . . . . . . . . . . . . 415

10.3.3 Neyman-PearsonDecision Rule and Receiver Operating Characteristic

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416

10.4 Minimum Risk Decisions . . . . . . . . . . . . . . . . . . . . . . . . . 421

10.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423

10.5.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 423

10.5.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 429

10.5.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 436

10.5.4 Problems Commented Out in 2010 . . . . . . . . . . . . . . . 444

11 Random Processes 447

11.1 Definition and Examples of a Random Process . . . . . . . . . . . . 447

11.2 First and Second Moment Characterization of Random Processes . . 452

11.3 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

11.3.1 Strict-Sense Stationarity . . . . . . . . . . . . . . . . . . . . . 453

11.3.2 Wide-Sense Stationarity . . . . . . . . . . . . . . . . . . . . . 454

11.3.3 Some Properties of WSS Correlation and Covariance Functions

11.4 Further Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11.5 Ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

11.6 Linear Estimation of Random Processes . . . . . . . . . . . . . . . . 460

11.6.1 Linear Prediction . . . . . . . . . . . . . . . . . . . . . . . . . 460

11.6.2 Linear FIR Filtering . . . . . . . . . . . . . . . . . . . . . . . 462

11.7 LTI Filtering of WSS Processes . . . . . . . . . . . . . . . . . . . . . 463

11.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

11.8.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 470

11.8.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 477

11.8.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 485

11.8.4 Unused Problems . . . . . . . . . . . . . . . . . . . . . . . . . 495

11.8.5 Possible Examples . . . . . . . . . . . . . . . . . . . . . . . . 496

12 Power Spectral Density 497

12.1 Spectral Distribution of Expected Instantaneous Power . . . . . . . . 498

12.1.1 Power Spectral Density (PSD) . . . . . . . . . . . . . . . . . 498

12.1.2 Fluctuation Spectral Density . . . . . . . . . . . . . . . . . . 502

12.1.3 Cross-Spectral Density . . . . . . . . . . . . . . . . . . . . . . 507

12.2 Expected Time-Averaged Power Spectrum and the Einstein-Wiener-Khinchin Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509

12.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513

12.3.1 Revealing Cyclic Components . . . . . . . . . . . . . . . . . . 513

12.3.2 Modeling Filters . . . . . . . . . . . . . . . . . . . . . . . . . 514

12.3.3 Whitening Filters . . . . . . . . . . . . . . . . . . . . . . . . . 517

12.3.4 Sampling Bandlimited Random Processes . . . . . . . . . . . 518

12.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

12.4.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 523

12.4.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 530

12.4.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 535

13 Signal Estimation 547

13.1 LMMSE Estimation for Random Variables . . . . . . . . . . . . . . . 547

13.2 FIR Wiener Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550

13.3 The Unconstrained DT Wiener Filter . . . . . . . . . . . . . . . . . 555

13.4 Causal DT Wiener Filtering . . . . . . . . . . . . . . . . . . . . . . . 564

13.5 Optimal Observers and Kalman Filtering . . . . . . . . . . . . . . . 571

13.5.1 Causal Estimation of a Signal Corrupted by Additive Noise . 571

13.5.2 Observer Implementation of the Wiener Filter . . . . . . . . 573

13.5.3 Optimal State Estimates and Kalman Filtering . . . . . . . . 575

13.6 Estimation of CT Signals . . . . . . . . . . . . . . . . . . . . . . . . 576

13.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578

13.7.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 578

13.7.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 598

13.7.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 601

14 Signal Detection 603

14.1 Hypothesis Testing with Multiple Measurements . . . . . . . . . . . 604

14.2 Detecting a Known Signal in IID Gaussian Noise . . . . . . . . . . . 606

14.2.1 The Optimal Solution . . . . . . . . . . . . . . . . . . . . . . 607

14.2.2 Characterizing Performance . . . . . . . . . . . . . . . . . . . 609

14.2.3 Matched Filtering . . . . . . . . . . . . . . . . . . . . . . . . 612

14.3 Extensions of Matched-Filter Detection . . . . . . . . . . . . . . . . 614

14.3.1 Infinite-Duration, Finite-Energy Signals . . . . . . . . . . . . 614

14.3.2 Maximizing SNR for Signal Detection in White Noise . . . . 614

14.3.3 Detection in Colored Noise . . . . . . . . . . . . . . . . . . . 617

14.3.4 Continuous-Time Matched Filters . . . . . . . . . . . . . . . 620

14.3.5 Matched FIltering and Nyquist Pulse Design . . . . . . . . . 621

14.3.6 Unknown Arrival Time, and Pulse Compression . . . . . . . . 622

14.4 Signal Discrimination in IID Gaussian Noise . . . . . . . . . . . . . . 624

14.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631

14.5.1 Basic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 631

14.5.2 Advanced Problems . . . . . . . . . . . . . . . . . . . . . . . 640

14.5.3 Extension Problems . . . . . . . . . . . . . . . . . . . . . . . 650