CONTENTS
Preface xi
Contributors xiii
1 Kalman Filters 1
Simon Haykin
1.1 Introduction = 1
1.2 Optimum Estimates = 3
1.3 Kalman Filter = 5
1.4 Divergence Phenomenon: Square-Root Filtering = 10
1.5 Rauch–Tung–Striebel Smoother = 11
1.6 Extended Kalman Filter = 16
1.7 Summary = 20
References = 20
2 Parameter-Based Kalman Filter Training:
Theory and Implementation 23
Gintaras V. Puskorius and Lee A. Feldkamp
2.1 Introduction = 23
2.2 Network Architectures = 26
2.3 The EKF Procedure = 28
2.3.1 Global EKF Training = 29
2.3.2 Learning Rate and Scaled Cost Function = 31
2.3.3 Parameter Settings = 32
2.4 Decoupled EKF (DEKF) = 33
2.5 Multistream Training = 35
v
2.5.1 Some Insight into the Multistream Technique = 40
2.5.2 Advantages and Extensions of Multistream
Training = 42
2.6 Computational Considerations = 43
2.6.1 Derivative Calculations = 43
2.6.2 Computationally Efficient Formulations for
Multiple-Output Problems = 45
2.6.3 Avoiding Matrix Inversions = 46
2.6.4 Square-Root Filtering = 48
2.7 Other Extensions and Enhancements = 51
2.7.1 EKF Training with Constrained Weights = 51
2.7.2 EKF Training with an Entropic Cost Function = 54
2.7.3 EKF Training with Scalar Errors = 55
2.8 Automotive Applications of EKF Training = 57
2.8.1 Air=Fuel Ratio Control = 58
2.8.2 Idle Speed Control = 59
2.8.3 Sensor-Catalyst Modeling = 60
2.8.4 Engine Misfire Detection = 61
2.8.5 Vehicle Emissions Estimation = 62
2.9 Discussion = 63
2.9.1 Virtues of EKF Training = 63
2.9.2 Limitations of EKF Training = 64
2.9.3 Guidelines for Implementation and Use = 64
References = 65
3 Learning Shape and Motion from Image Sequences 69
Gaurav S. Patel, Sue Becker, and Ron Racine
3.1 Introduction = 69
3.2 Neurobiological and Perceptual Foundations of our Model = 70
3.3 Network Description = 71
3.4 Experiment 1 = 73
3.5 Experiment 2 = 74
3.6 Experiment 3 = 76
3.7 Discussion = 77
References = 81
vi CONTENTS
4 Chaotic Dynamics 83
Gaurav S. Patel and Simon Haykin
4.1 Introduction = 83
4.2 Chaotic (Dynamic) Invariants = 84
4.3 Dynamic Reconstruction = 85
4.4 Modeling Numerically Generated Chaotic Time Series = 87
4.4.1 Logistic Map = 87
4.4.2 Ikeda Map = 91
4.4.3 Lorenz Attractor = 99
4.5 Nonlinear Dynamic Modeling of Real-World
Time Series = 106
4.5.1 Laser Intensity Pulsations = 106
4.5.2 Sea Clutter Data = 113
4.6 Discussion = 119
References = 121
5 Dual Extended Kalman Filter Methods 123
Eric A. Wan and Alex T. Nelson
5.1 Introduction = 123
5.2 Dual EKF – Prediction Error = 126
5.2.1 EKF – State Estimation = 127
5.2.2 EKF –Weight Estimation = 128
5.2.3 Dual Estimation = 130
5.3 A Probabilistic Perspective = 135
5.3.1 Joint Estimation Methods = 137
5.3.2 Marginal Estimation Methods = 140
5.3.3 Dual EKF Algorithms = 144
5.3.4 Joint EKF = 149
5.4 Dual EKF Variance Estimation = 149
5.5 Applications = 153
5.5.1 Noisy Time-Series Estimation and Prediction = 153
5.5.2 Economic Forecasting – Index of Industrial
Production = 155
5.5.3 Speech Enhancement = 157
5.6 Conclusions = 163
Acknowledgments = 164
CONTENTS vii
Appendix A: Recurrent Derivative of the Kalman Gain = 164
Appendix B: Dual EKF with Colored Measurement Noise = 166
References = 170
6 Learning Nonlinear Dynamical System Using the
Expectation-Maximization Algorithm 175
Sam T. Roweis and Zoubin Ghahramani
6.1 Learning Stochastic Nonlinear Dynamics = 175
6.1.1 State Inference and Model Learning = 177
6.1.2 The Kalman Filter = 180
6.1.3 The EM Algorithm = 182
6.2 Combining EKS and EM = 186
6.2.1 Extended Kalman Smoothing (E-step) = 186
6.2.2 Learning Model Parameters (M-step) = 188
6.2.3 Fitting Radial Basis Functions to Gaussian
Clouds = 189
6.2.4 Initialization of Models and Choosing Locations
for RBF Kernels = 192
6.3 Results = 194
6.3.1 One- and Two-Dimensional Nonlinear State-Space
Models = 194
6.3.2 Weather Data = 197
6.4 Extensions = 200
6.4.1 Learning the Means and Widths of the RBFs = 200
6.4.2 On-Line Learning = 201
6.4.3 Nonstationarity = 202
6.4.4 Using Bayesian Methods for Model Selection and
Complexity Control = 203
6.5 Discussion = 206
6.5.1 Identifiability and Expressive Power = 206
6.5.2 Embedded Flows = 207
6.5.3 Stability = 210
6.5.4 Takens’ Theorem and Hidden States = 211
6.5.5 Should Parameters and Hidden States be Treated
Differently? = 213
6.6 Conclusions = 214
Acknowledgments = 215
viii CONTENTS
Appendix: Expectations Required to Fit the RBFs = 215
References = 216
7 The Unscented Kalman Filter 221
Eric A. Wan and Rudolph van der Merwe
7.1 Introduction = 221
7.2 Optimal Recursive Estimation and the EKF = 224
7.3 The Unscented Kalman Filter = 234
7.3.1 State-Estimation Examples = 237
7.3.2 The Unscented Kalman Smoother = 240
7.4 UKF Parameter Estimation = 243
7.4.1 Parameter-Estimation Examples = 2
7.5 UKF Dual Estimation = 249
7.5.1 Dual Estimation Experiments = 249
7.6 The Unscented Particle Filter = 254
7.6.1 The Particle Filter Algorithm = 259
7.6.2 UPF Experiments = 263
7.7 Conclusions = 269
Appendix A: Accuracy of the Unscented Transformation = 269
Appendix B: Efficient Square-Root UKF Implementations = 273
References = 277
Index 283
CONTENTS ix