SEMIPARAMETRIC
REGRESSION FOR THE
APPLIEDECONOMETRICIAN

ADONIS YATCHEW
University of Toronto

Contents
List of Figures and Tables page xv
Preface xvii
1 Introduction to Differencing 1
1.1 A Simple Idea 1
1.2 Estimation of the Residual Variance 2
1.3 The Partial Linear Model 2
1.4 Specification Test 4
1.5 Test of Equality of Regression Functions 4
1.6 Empirical Application: Scale Economies in Electricity
Distribution 7
1.7 Why Differencing? 8
1.8 Empirical Applications 11
1.9 Notational Conventions 12
1.10 Exercises 12
2 Background and Overview 15
2.1 Categorization of Models 15
2.2 The Curse of Dimensionality and the Need for Large
Data Sets 17
2.2.1 Dimension Matters 17
2.2.2 Restrictions That Mitigate the Curse 17
2.3 Local Averaging Versus Optimization 19
2.3.1 Local Averaging 19
2.3.2 Bias-Variance Trade-Off 19
2.3.3 Naive Optimization 22
2.4 A Bird’s-Eye View of Important Theoretical Results 23
2.4.1 Computability of Estimators 23
2.4.2 Consistency 23
2.4.3 Rate of Convergence 23
ix
x Contents
2.4.4 Bias-Variance Trade-Off 25
2.4.5 Asymptotic Distributions of Estimators 26
2.4.6 How Much to Smooth 26
2.4.7 Testing Procedures 26
3 Introduction to Smoothing 27
3.1 A Simple Smoother 27
3.1.1 The Moving Average Smoother 27
3.1.2 A Basic Approximation 28
3.1.3 Consistency and Rate of Convergence 29
3.1.4 Asymptotic Normality and Confidence Intervals 29
3.1.5 Smoothing Matrix 30
3.1.6 Empirical Application: Engel Curve Estimation 30
3.2 Kernel Smoothers 32
3.2.1 Estimator 32
3.2.2 Asymptotic Normality 34
3.2.3 Comparison to Moving Average Smoother 35
3.2.4 Confidence Intervals 35
3.2.5 Uniform Confidence Bands 36
3.2.6 Empirical Application: Engel Curve Estimation 37
3.3 Nonparametric Least-Squares and Spline Smoothers 37
3.3.1 Estimation 37
3.3.2 Properties 39
3.3.3 Spline Smoothers 40
3.4 Local Polynomial Smoothers 40
3.4.1 Local Linear Regression 40
3.4.2 Properties 41
3.4.3 Empirical Application: Engel Curve Estimation 42
3.5 Selection of Smoothing Parameter 43
3.5.1 Kernel Estimation 43
3.5.2 Nonparametric Least Squares 44
3.5.3 Implementation 46
3.6 Partial Linear Model 47
3.6.1 Kernel Estimation 47
3.6.2 Nonparametric Least Squares 48
3.6.3 The General Case 48
3.6.4 Heteroskedasticity 50
3.6.5 Heteroskedasticity and Autocorrelation 51
3.7 Derivative Estimation 52
3.7.1 Point Estimates 52
3.7.2 Average Derivative Estimation 53
3.8 Exercises 54
Contents xi
4 Higher-Order Differencing Procedures 57
4.1 Differencing Matrices 57
4.1.1 Definitions 57
4.1.2 Basic Properties of Differencing and Related
Matrices 58
4.2 Variance Estimation 58
4.2.1 The mth-Order Differencing Estimator 58
4.2.2 Properties 59
4.2.3 Optimal Differencing Coefficients 60
4.2.4 Moving Average Differencing Coefficients 61
4.2.5 Asymptotic Normality 62
4.3 Specification Test 63
4.3.1 A Simple Statistic 63
4.3.2 Heteroskedasticity 64
4.3.3 Empirical Application: Log-Linearity of
Engel Curves 65
4.4 Test of Equality of Regression Functions 66
4.4.1 A Simplified Test Procedure 66
4.4.2 The Differencing Estimator Applied to the
Pooled Data 67
4.4.3 Properties 68
4.4.4 Empirical Application: Testing Equality of Engel
Curves 69
4.5 Partial Linear Model 71
4.5.1 Estimator 71
4.5.2 Heteroskedasticity 72
4.6 Empirical Applications 73
4.6.1 Household Gasoline Demand in Canada 73
4.6.2 Scale Economies in Electricity Distribution 76
4.6.3 Weather and Electricity Demand 81
4.7 Partial Parametric Model 83
4.7.1 Estimator 83
4.7.2 Empirical Application: CES Cost Function 84
4.8 Endogenous Parametric Variables in the Partial Linear Model 85
4.8.1 Instrumental Variables 85
4.8.2 Hausman Test 86
4.9 Endogenous Nonparametric Variable 87
4.9.1 Estimation 87
4.9.2 Empirical Application: Household Gasoline
Demand and Price Endogeneity 88
4.10 Alternative Differencing Coefficients 89
4.11 The Relationship of Differencing to Smoothing 90
xii Contents
4.12 Combining Differencing and Smoothing 92
4.12.1 Modular Approach to Analysis of the Partial
Linear Model 92
4.12.2 Combining Differencing Procedures in Sequence 92
4.12.3 Combining Differencing and Smoothing 93
4.12.4 Reprise 94
4.13 Exercises 94
5 Nonparametric Functions of Several Variables 99
5.1 Smoothing 99
5.1.1 Introduction 99
5.1.2 Kernel Estimation of Functions of Several Variables 99
5.1.3 Loess 101
5.1.4 Nonparametric Least Squares 101
5.2 Additive Separability 102
5.2.1 Backfitting 102
5.2.2 Additively Separable Nonparametric Least Squares 103
5.3 Differencing 104
5.3.1 Two Dimensions 104
5.3.2 Higher Dimensions and the Curse of Dimensionality 105
5.4 Empirical Applications 107
5.4.1 Hedonic Pricing of Housing Attributes 107
5.4.2 Household Gasoline Demand in Canada 107
5.5 Exercises 110
6 Constrained Estimation and Hypothesis Testing 111
6.1 The Framework 111
6.2 Goodness-of-Fit Tests 113
6.2.1 Parametric Goodness-of-Fit Tests 113
6.2.2 Rapid Convergence under the Null 114
6.3 Residual Regression Tests 115
6.3.1 Overview 115
6.3.2 U-statistic Test – Scalar x’s, Moving
Average Smoother 116
6.3.3 U-statistic Test – Vector x’s, Kernel Smoother 117
6.4 Specification Tests 119
6.4.1 Bierens (1990) 119
6.4.2 H¨ardle and Mammen (1993) 120
6.4.3 Hong and White (1995) 121
6.4.4 Li (1994) and Zheng (1996) 122
6.5 Significance Tests 124
Contents xiii
6.6 Monotonicity, Concavity, and Other Restrictions 125
6.6.1 Isotonic Regression 125
6.6.2 Why Monotonicity Does Not Enhance the Rate
of Convergence 126
6.6.3 Kernel-Based Algorithms for Estimating Monotone
Regression Functions 127
6.6.4 Nonparametric Least Squares Subject to
Monotonicity Constraints 127
6.6.5 Residual Regression and Goodness-of-Fit Tests
of Restrictions 128
6.6.6 Empirical Application: Estimation of Option Prices 129
6.7 Conclusions 134
6.8 Exercises 136
7 Index Models and Other Semiparametric Specifications 138
7.1 Index Models 138
7.1.1 Introduction 138
7.1.2 Estimation 138
7.1.3 Properties 139
7.1.4 Identification 140
7.1.5 Empirical Application: Engel’s Method for
Estimation of Equivalence Scales 140
7.1.6 Empirical Application: Engel’s Method for Multiple
Family Types 142
7.2 Partial Linear Index Models 144
7.2.1 Introduction 144
7.2.2 Estimation 146
7.2.3 Covariance Matrix 147
7.2.4 Base-Independent Equivalence Scales 148
7.2.5 Testing Base-Independence and Other Hypotheses 149
7.3 Exercises 151
8 Bootstrap Procedures 154
8.1 Background 154
8.1.1 Introduction 154
8.1.2 Location Scale Models 155
8.1.3 Regression Models 156
8.1.4 Validity of the Bootstrap 157
8.1.5 Benefits of the Bootstrap 157
8.1.6 Limitations of the Bootstrap 159
8.1.7 Summary of Bootstrap Choices 159
8.1.8 Further Reading 160
xiv Contents
8.2 Bootstrap Confidence Intervals for Kernel Smoothers 160
8.3 Bootstrap Goodness-of-Fit and Residual Regression Tests 163
8.3.1 Goodness-of-Fit Tests 163
8.3.2 Residual Regression Tests 164
8.4 Bootstrap Inference in Partial Linear and Index Models 166
8.4.1 Partial Linear Models 166
8.4.2 Index Models 166
8.5 Exercises 171
Appendixes
Appendix A – Mathematical Preliminaries 173
Appendix B – Proofs 175
Appendix C – Optimal Differencing Weights 183
Appendix D – Nonparametric Least Squares 187
Appendix E – Variable Definitions 194
References 197
Index 209