Contents iii
1 Concepts, models and de.nitions 1
1.1 De.ning nonlinearity . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Where does nonlinearity come from? . . . . . . . . . . . . . . 2
1.3 Stationarity and nonstationarity . . . . . . . . . . . . . . . . 3
1.4 Invertibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6 Seasonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.7 Conditional distributions . . . . . . . . . . . . . . . . . . . . . 11
1.8 Wold.s representation and Volterra expansion . . . . . . . . . 12
1.9 Additive models . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.10 Spectral analysis . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.11 Chaos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Nonlinear models in economic theory 17
2.1 Disequilibrium models . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Labour market models . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.2 Practice . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 Exchange rates in a target zone . . . . . . . . . . . . . . . . . 23
2.3.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.2 Practice . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4 Production theory . . . . . . . . . . . . . . . . . . . . . . . . . 27
3 Parametric nonlinear models 31
3.1 General considerations . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Switching regression models . . . . . . . . . . . . . . . . . . . 35
3.2.1 Standard switching regression model . . . . . . . . . . 35
3.2.2 Vector threshold autoregressive model . . . . . . . . . . 37
3.3 Markov-switching regression models . . . . . . . . . . . . . . 39
3.4 Smooth transition regression models . . . . . . . . . . . . . . . 41
3.4.1 Standard smooth transition model . . . . . . . . . . . . 41
3.4.2 Additive, multiple and time-varying STR models . . . 43
3.4.3 Vector smooth transition autoregressive model . . . . . 44
3.5 Polynomial models . . . . . . . . . . . . . . . . . . . . . . . . 45
3.6 Arti…cial neural network models . . . . . . . . . . . . . . . . . 46
3.7 Min-max models . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.8 Nonlinear moving average models . . . . . . . . . . . . . . . . 50
3.9 Bilinear models . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.10 Time-varying parameters and state space models . . . . . . . 52
3.11 Random coe¢ cient and volatility models . . . . . . . . . . . . 54
4 The nonparametric approach 57
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2 Autocovariance and spectrum . . . . . . . . . . . . . . . . . . 58
4.3 Density, conditional mean and variance . . . . . . . . . . . . . 60
4.3.1 Non-Gaussian marginals . . . . . . . . . . . . . . . . . 60
4.3.2 Conditional quantities . . . . . . . . . . . . . . . . . . 61
4.4 Dependence measures for nonlinear processes . . . . . . . . . 62
4.4.1 Local measures of dependence . . . . . . . . . . . . . . 63
4.4.2 Global measures of dependence . . . . . . . . . . . . . 66
4.4.3 Measures based on density and distribution functions . 67
4.4.4 The copula . . . . . . . . . . . . . . . . . . . . . . . . 68
5 Parametric linearity tests 71
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.2 Consistent misspeci…cation tests . . . . . . . . . . . . . . . . . 72
5.3 Lagrange multiplier or score test . . . . . . . . . . . . . . . . 74
5.3.1 Standard case . . . . . . . . . . . . . . . . . . . . . . . 74
5.3.2 Test in stages and a robust version . . . . . . . . . . . 76
5.4 Locally equivalent alternatives . . . . . . . . . . . . . . . . . . 77
5.5 Model only identi…ed under the alternative . . . . . . . . . . . 78
5.5.1 Identi…cation problem . . . . . . . . . . . . . . . . . . 78
5.5.2 General solution . . . . . . . . . . . . . . . . . . . . . . 79
5.5.3 Lagrange multiplier type tests . . . . . . . . . . . . . . 83
5.5.4 Monte Carlo tests . . . . . . . . . . . . . . . . . . . . . 85
5.5.5 Values to nuisance parameters . . . . . . . . . . . . . . 87
5.6 Testing against unspeci…ed alternatives . . . . . . . . . . . . . 88
5.6.1 Regression Error Speci…cation Test . . . . . . . . . . . 88
5.6.2 Tests based on expansions . . . . . . . . . . . . . . . . 90
5.7 Asymptotic relative e¢ ciency . . . . . . . . . . . . . . . . . . 91

5.7.1 De…nition . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.7.2 An example . . . . . . . . . . . . . . . . . . . . . . . . 94
5.8 Which test to use? . . . . . . . . . . . . . . . . . . . . . . . . 96
6 Testing parameter constancy 99
6.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.2 Generalizing the Chow-test . . . . . . . . . . . . . . . . . . . . 100
6.2.1 Testing against a single break . . . . . . . . . . . . . . 100
6.2.2 Testing against multiple breaks . . . . . . . . . . . . . 102
6.3 Lagrange multiplier type tests . . . . . . . . . . . . . . . . . . 104
6.3.1 Testing a stationary single-equation model . . . . . . . 104
6.3.2 Testing a stationary vector autoregressive model . . . . 108
6.3.3 Testing a nonstationary vector autoregressive model . 110
6.4 Tests based on recursive estimation . . . . . . . . . . . . . . . 113
6.4.1 Cumulative sum tests . . . . . . . . . . . . . . . . . . . 113
6.4.2 Moving sum tests . . . . . . . . . . . . . . . . . . . . . 115
6.4.3 Fluctuation tests . . . . . . . . . . . . . . . . . . . . . 116
6.4.4 Tests against stochastic parameters . . . . . . . . . . . 117
6.4.5 Testing the constancy of cointegrating relationships . . 119
7 Nonparametric speci…cation tests 121
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.2 Nonparametric linearity tests . . . . . . . . . . . . . . . . . . 123
7.2.1 Nonparametric tests. The spectral domain . . . . . . . 123
7.2.2 Testing linearity in the conditional mean and condi-
tional variance . . . . . . . . . . . . . . . . . . . . . . 124
7.2.3 Estimation . . . . . . . . . . . . . . . . . . . . . . . . 127
7.2.4 Asymptotic theory . . . . . . . . . . . . . . . . . . . . 128
7.2.5 Finite-sample properties and use of the asymptotics . 130
7.2.6 A bootstrap approach to testing . . . . . . . . . . . . 130
7.3 Testing for speci…c functional forms . . . . . . . . . . . . . . 132
7.3.1 Tests based on residuals . . . . . . . . . . . . . . . . . 133
7.3.2 Conditional mean and conditional variance testing . . . 136
7.3.3 Continuous time . . . . . . . . . . . . . . . . . . . . . 138
7.4 Selecting lags . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7.5 Testing for additivity and interaction . . . . . . . . . . . . . . 142
7.5.1 Testing in additive models . . . . . . . . . . . . . . . . 142
7.5.2 A simulated example . . . . . . . . . . . . . . . . . . . 146
7.6 Tests for partial linearity . . . . . . . . . . . . . . . . . . . . . 148
7.7 Tests of independence . . . . . . . . . . . . . . . . . . . . . . 149
7.7.1 Traditional tests . . . . . . . . . . . . . . . . . . . . . 150

7.7.2 Rank correlation . . . . . . . . . . . . . . . . . . . . . 151
7.7.3 Frequency based tests . . . . . . . . . . . . . . . . . . 153
7.7.4 BDS test . . . . . . . . . . . . . . . . . . . . . . . . . 153
7.7.5 Distribution based tests of independence . . . . . . . . 155
7.7.6 Generalized spectrum and tests of independence . . . 160
7.7.7 Density based tests of independence . . . . . . . . . . 163
7.7.8 Some examples of independence testing . . . . . . . . 169
8 Conditional heteroskedasticity 173
8.1 Autoregressive conditional heteroskedasticity . . . . . . . . . . 174
8.1.1 The ARCH model . . . . . . . . . . . . . . . . . . . . . 174
8.2 The Generalized ARCH model . . . . . . . . . . . . . . . . . . 175
8.2.1 Why Generalized ARCH? . . . . . . . . . . . . . . . . 175
8.2.2 Families of univariate GARCH models . . . . . . . . . 175
8.2.3 Nonlinear GARCH . . . . . . . . . . . . . . . . . . . . 178
8.2.4 Time-varying GARCH . . . . . . . . . . . . . . . . . . 180
8.2.5 Moment structure of …rst-order GARCH models . . . . 182
8.2.6 Moment structure of higher-order GARCH models . . . 183
8.2.7 Integrated and fractionally integrated GARCH . . . . . 184
8.2.8 Stylized facts and the GARCH model . . . . . . . . . . 186
8.2.9 Building univariate GARCH models . . . . . . . . . . . 189
8.3 Family of Exponential GARCH models . . . . . . . . . . . . . 200
8.3.1 Moment structure of EGARCH models . . . . . . . . . 200
8.3.2 Stylized facts and the EGARCH model . . . . . . . . . 201
8.3.3 Building EGARCH models . . . . . . . . . . . . . . . . 202
8.4 Stochastic Volatility model . . . . . . . . . . . . . . . . . . . . 208
8.4.1 De…nition . . . . . . . . . . . . . . . . . . . . . . . . . 208
8.4.2 Moment structure of ARSV models . . . . . . . . . . . 209
8.4.3 Stylized facts and the stochastic volatility model . . . . 209
8.4.4 Estimation of ARSV models . . . . . . . . . . . . . . . 210
8.4.5 Comparing the ARSV model with GARCH . . . . . . . 211
8.5 GARCH-in-mean model . . . . . . . . . . . . . . . . . . . . . 211
8.6 Realized volatility . . . . . . . . . . . . . . . . . . . . . . . . . 212
8.7 Multivariate GARCH models . . . . . . . . . . . . . . . . . . 214
8.7.1 General multivariate GARCH model . . . . . . . . . . 214
8.7.2 Link to random coe¢ cient models . . . . . . . . . . . . 215
8.7.3 Constant conditional correlation GARCH . . . . . . . . 216
8.7.4 Testing the constant correlation assumption and the
Dynamic Conditional Correlation model . . . . . . . . 219
8.7.5 Other extensions to the CCC-GARCH model . . . . . 221
8.7.6 The BEKK-GARCH model . . . . . . . . . . . . . . . 223

8.7.7 Factor GARCH models . . . . . . . . . . . . . . . . . . 226
9 State space models 233
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
9.2 Linear state space models . . . . . . . . . . . . . . . . . . . . 235
9.3 Time-varying parameter models . . . . . . . . . . . . . . . . . 237
9.4 Nonlinear state space models . . . . . . . . . . . . . . . . . . 238
9.4.1 Extended Kalman …lter . . . . . . . . . . . . . . . . . . 239
9.4.2 Kitagawa’s grid approximation . . . . . . . . . . . . . 240
9.4.3 Monte Carlo methods . . . . . . . . . . . . . . . . . . . 243
9.4.4 Particle …lters . . . . . . . . . . . . . . . . . . . . . . . 243
9.4.5 Approximating with a Gaussian density . . . . . . . . 246
9.5 Hidden Markov chains and regimes . . . . . . . . . . . . . . . 249
9.5.1 Hidden Markov chains . . . . . . . . . . . . . . . . . . 249
9.5.2 Mixture models . . . . . . . . . . . . . . . . . . . . . 253
9.6 Estimating parameters . . . . . . . . . . . . . . . . . . . . . . 256
9.6.1 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . 257
9.6.2 Identi…cation . . . . . . . . . . . . . . . . . . . . . . . 259
9.6.3 Estimation in linear models . . . . . . . . . . . . . . . 260
9.6.4 The nonlinear case . . . . . . . . . . . . . . . . . . . . 262
9.6.5 Estimation in hidden Markov and mixture models . . . 264
10 Nonparametric models 267
10.1 Additive models . . . . . . . . . . . . . . . . . . . . . . . . . 268
10.1.1 Estimation in purely additive models . . . . . . . . . . 270
10.1.2 Marginal integration . . . . . . . . . . . . . . . . . . . 270
10.1.3 Back…tting and smoothed back…tting . . . . . . . . . . 272
10.1.4 Additive models with interactions . . . . . . . . . . . 275
10.1.5 A simulated example . . . . . . . . . . . . . . . . . . . 277
10.1.6 Nonparametric and additive estimation of the condi-
tional variance function . . . . . . . . . . . . . . . . . 279
10.2 Some related models . . . . . . . . . . . . . . . . . . . . . . . 284
10.2.1 Functional coe¢ cient autoregressive models . . . . . . 284
10.2.2 Transformation of dependent variables and the ACE
algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 285
10.2.3 Regression trees, splines and MARS . . . . . . . . . . 285
10.2.4 Quantile regression . . . . . . . . . . . . . . . . . . . . 286
10.3 Semiparametric models . . . . . . . . . . . . . . . . . . . . . 287
10.3.1 Index models . . . . . . . . . . . . . . . . . . . . . . . 289
10.3.2 Projection pursuit regression . . . . . . . . . . . . . . 290
10.3.3 Partially linear models . . . . . . . . . . . . . . . . . . 291

10.4 Robust and adaptive estimation . . . . . . . . . . . . . . . . . 292
11 Nonlinear and nonstationary models 295
11.1 Long memory models . . . . . . . . . . . . . . . . . . . . . . 295
11.2 Linear unit root models . . . . . . . . . . . . . . . . . . . . . 301
11.3 Vector autoregressions and cointegration . . . . . . . . . . . . 304
11.4 Nonlinear I(1) processes . . . . . . . . . . . . . . . . . . . . . 307
11.5 Nonlinear error-correction models . . . . . . . . . . . . . . . . 310
11.6 Parametric nonlinear regression . . . . . . . . . . . . . . . . . 314
11.7 Nonlinear cointegration framework . . . . . . . . . . . . . . . 320
11.8 Stochastic unit root models . . . . . . . . . . . . . . . . . . . 322
12 Estimating parametric models 325
12.1 Optimization without derivatives . . . . . . . . . . . . . . . . 326
12.1.1 Grid and line searches . . . . . . . . . . . . . . . . . . 326
12.1.2 Conjugate directions . . . . . . . . . . . . . . . . . . . 327
12.1.3 Simulated annealing . . . . . . . . . . . . . . . . . . . 329
12.1.4 Evolutionary algorithms . . . . . . . . . . . . . . . . . 332
12.2 Methods requiring derivatives . . . . . . . . . . . . . . . . . . 336
12.2.1 Gradient methods . . . . . . . . . . . . . . . . . . . . . 336
12.2.2 Variable metric methods . . . . . . . . . . . . . . . . . 341
12.3 Other methods . . . . . . . . . . . . . . . . . . . . . . . . . . 342
12.3.1 EM algorithm . . . . . . . . . . . . . . . . . . . . . . . 342
12.3.2 Sequential estimation for neural networks . . . . . . . . 345
13 Basic nonparametric estimates 347
13.1 Density estimation . . . . . . . . . . . . . . . . . . . . . . . . 347
13.1.1 Kernel estimation . . . . . . . . . . . . . . . . . . . . . 347
13.1.2 Bias and variance reduction . . . . . . . . . . . . . . . 349
13.1.3 Choice of bandwidth . . . . . . . . . . . . . . . . . . . 351
13.1.4 Variable bandwidth and nearest neighbour estimation 352
13.1.5 Multivariate density estimation . . . . . . . . . . . . . 352
13.2 Nonparametric regression estimation . . . . . . . . . . . . . . 353
13.2.1 Kernel regression estimation . . . . . . . . . . . . . . . 354
13.2.2 Local polynomial estimation . . . . . . . . . . . . . . 355
13.2.3 Bias, convolution and higher-order kernels . . . . . . . 356
13.2.4 Nearest neighbour estimation . . . . . . . . . . . . . . 358
13.2.5 Splines and MARS . . . . . . . . . . . . . . . . . . . . 359
13.2.6 Series expansion . . . . . . . . . . . . . . . . . . . . . . 360
13.2.7 Choice of bandwidth for nonparametric regression . . . 360

14 Forecasting from nonlinear models 363
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
14.2 Conditional mean forecasts . . . . . . . . . . . . . . . . . . . . 365
14.2.1 Analytical point forecasts . . . . . . . . . . . . . . . . 365
14.2.2 Numerical techniques in forecasting . . . . . . . . . . . 366
14.3 Forecasting with nonparametric models . . . . . . . . . . . . . 371
14.4 Forecast accuracy . . . . . . . . . . . . . . . . . . . . . . . . . 374
14.5 Usefulness of forecasts . . . . . . . . . . . . . . . . . . . . . . 376
14.6 Forecasting volatility . . . . . . . . . . . . . . . . . . . . . . . 381
14.7 Overview of forecasting . . . . . . . . . . . . . . . . . . . . . . 382
15 Nonlinear impulse responses 385
15.1 Generalized impulse response function . . . . . . . . . . . . . 385
15.2 Graphical representation . . . . . . . . . . . . . . . . . . . . . 388
16 Building nonlinear models 391
16.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
16.2 Nonparametric and semiparametric models . . . . . . . . . . . 392
16.3 Building smooth transition models . . . . . . . . . . . . . . . 397
16.3.1 The three stages of the modelling procedure . . . . . . 397
16.3.2 Speci…cation . . . . . . . . . . . . . . . . . . . . . . . . 398
16.3.3 Estimation of parameters . . . . . . . . . . . . . . . . . 402
16.3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 404
16.3.5 Graphical evaluation tools . . . . . . . . . . . . . . . . 411
16.3.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 413
16.4 Building switching regression models . . . . . . . . . . . . . . 443
16.4.1 Speci…cation . . . . . . . . . . . . . . . . . . . . . . . . 443
16.4.2 Estimation and evaluation . . . . . . . . . . . . . . . . 446
16.4.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 448
16.5 Building arti…cial neural network models . . . . . . . . . . . . 460
16.5.1 Speci…cation . . . . . . . . . . . . . . . . . . . . . . . . 460
16.5.2 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 463
16.5.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 464
16.5.4 Alternative modelling approaches . . . . . . . . . . . . 465
16.5.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 465
16.6 Two forecast comparisons . . . . . . . . . . . . . . . . . . . . 472
16.6.1 Forecasting Wolf’s annual sunspot numbers . . . . . . 472
16.6.2 Forecasting the monthly US unemployment rate . . . . 476

17 Other topics 479
17.1 Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
17.2 Seasonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
17.2.1 Time-varying seasonality . . . . . . . . . . . . . . . . . 485
17.2.2 Temporal aggregation and time-varying seasonality . . 491
17.2.3 Nonlinear …lters in seasonal adjustment . . . . . . . . . 492
17.3 Outliers and nonlinearity . . . . . . . . . . . . . . . . . . . . 493
17.3.1 What is an outlier? . . . . . . . . . . . . . . . . . . . . 493
17.3.2 Model-based de…nitions . . . . . . . . . . . . . . . . . . 494
Acronyms and abbreviations 499
Bibliography 503