Preface ix
PART I FOUNDATIONS 1
1. Sources of Error 3
Prescription 4
Fundamental Concepts 4
Ad Hoc, Post Hoc Hypotheses 7
2. Hypotheses: The Why of Your Research 13
Prescription 13
What Is a Hypothesis? 13
How precise must a hypothesis be? 14
Found Data 16
Null hypothesis 16
Neyman–Pearson Theory 17
Deduction and Induction 21
Losses 22
Decisions 24
To Learn More 25
3. Collecting Data 27
Preparation 27
Measuring Devices 28
Determining Sample Size 31
Fundamental Assumptions 36
Experimental Design 38
Four Guidelines 39
Are Experiments Really Necessary? 42
To Learn More 42
PART II HYPOTHESIS TESTING AND ESTIMATION 45
4. Estimation 47
Prevention 47
Desirable and Not-So-Desirable Estimators 47
Interval Estimates 51
Improved Results 55
Summary 56
To Learn More 56
5. Testing Hypotheses: Choosing a Test Statistic 57
Comparing Means of Two Populations 59
Comparing Variances 67
Comparing the Means of K Samples 71
Higher-Order Experimental Designs 73
Contingency Tables 79
Inferior Tests 80
Multiple Tests 81
Before You Draw Conclusions 81
Summary 84
To Learn More 84
6. Strengths and Limitations of Some Miscellaneous Statistical
Procedures 87
Bootstrap 88
Bayesian Methodology 89
Meta-Analysis 96
Permutation Tests 99
To Learn More 99
7. Reporting Your Results 101
Fundamentals 101
Tables 104
Standard Error 105
p-Values 110
Confidence Intervals 111
Recognizing and Reporting Biases 113
Reporting Power 115
Drawing Conclusions 115
vi CONTENTS
Summary 116
To Learn More 116
8. Interpreting Reports 119
With A Grain of Salt 119
Rates and Percentages 122
Interpreting Computer Printouts 123
9. Graphics 125
The Soccer Data 125
Five Rules for Avoiding Bad Graphics 126
One Rule for Correct Usage of Three-Dimensional Graphics 133
The Misunderstood Pie Chart 135
Two Rules for Effective Display of Subgroup Information 136
Two Rules for Text Elements in Graphics 140
Multidimensional Displays 141
Choosing Graphical Displays 143
Summary 143
To Learn More 144
PART III BUILDING A MODEL 145
10. Univariate Regression 147
Model Selection 147
Estimating Coefficients 155
Further Considerations 157
Summary 160
To Learn More 162
11. Alternate Methods of Regression 163
Linear vs. Nonlinear Regression 164
Least Absolute Deviation Regression 164
Errors-in-Variables Regression 165
Quantile Regression 169
The Ecological Fallacy 170
Nonsense Regression 172
Summary 172
To Learn More 172
12. Multivariable Regression 175
Caveats 175
Factor Analysis 178
General Linearized Models 178
Reporting Your Results 181
CONTENTS vii
A Conjecture 182
Decision Trees 183
Building a Successful Model 185
To Learn More 186
13. Validation 187
Methods of Validation 188
Measures of Predictive Success 191
Long-Term Stability 193
To Learn More 194
Appendix A 195
Appendix B 205
Glossary, Grouped by Related but Distinct Terms 219
Bibliography 223
Author Index 243
Subject Index 249
viii CONTENTS