A Predictive Approach to Model Selection and Multicollinearity

SUMMARY
We argue for the adoption of a predictive approach to model specification. Specifically, we derive the
difference between means and the ratio of determinants of covariance matrices when a subset of explanatory
variables is included or excluded from a regression. Results for an economic application are presented as an
example. 01997 by John Wiley & Sons, Ltd. J. appl. econom. 12: 67-75, 1997.
(No. of Figures: 2. No. of Tables: 3. Number of Refs: 12.)
1. INTRODUCTION
This paper addresses the question of when it might be reasonable to simplify a linear regression
model by eliminating a subset of variables in a regression equation. We are thus concerned with
the problem of model selection, which we approach from a predictive Bayesian viewpoint. This
problem, of course, has been intensively studied from many other viewpoints. Since nonexperimental
data in general, and economics data in particular, are often highly correlated,
model specification is closely related to the problem of multicollinearity.
Our approach is to compare the predictive densities for an equation with and without the set of
variables in question and to argue that the set may be safely omitted if the omission has little or
no effect on the predictive density. This approach can be utilized in more general settings; for
example, Marriott et al. (1996) take a very similar approach in the context of a model with
serially correlated errors. The predictive density seems to us the best instrument for making this
kind of choice because it is defined in terms of observable values of the dependent variable, about
which an investigator is likely to have a considerable amount of information. He or she is likely
to know how much of an effect is important in a particular context and will know something
about the costs of making errors of various sizes and directions. In contrast, little or nothing is
known about values of regression coefficients in almost all econometric studies. The pervasive
use of the null hypothesis value of zero for a coefficient is a sign of this ignorance. Zellner's (1971)
Bayesian analysis of model selection requires that the prior probability of the truth of each model
be specified. Although our method does not require the specification of such probabilities and
may therefore be somewhat lacking in Bayesian rigour, our analysis and examples show that the
method has practical advantages and provides useful information.