Regression Analysis by Example - 4th edition
CONTENTS
Preface
1 Introduction
1.1 What Is Regression Analysis?
1.2 Publicly Available Data Sets
1.3 Selected Applications of Regression Analysis
1.3.1 Agricultural Sciences
1.3.2 Industrial and Labor Relations
1.3.3 History
1.3.4 Government
1.3.5 Environmental Sciences
1.4 Steps in Regression Analysis
1.4.1 Statement of the Problem
1 .4.2
1.4.3 Data Collection
1.4.4 Model Specification
1.4.5 Method of Fitting
1.4.6 Model Fitting
1.4.7 Model Criticism and Selection
1.4.8 Objectives of Regression Analysis
1.5 Scope and Organization of the Book
Exercises
2 Simple Linear Regression
2.1 Introduction
2.2 Covariance and Correlation Coefficient
2.3 Example: Computer Repair Data
2.4 The Simple Linear Regression Model
2.5 Parameter Estimation
2.6 Tests of Hypotheses
2.7 Confidence Intervals
2.8 Predictions
2.9 Measuring the Quality of Fit
2.10 Regression Line Through the Origin
2.1 1 Trivial Regression Models
2.12 Bibliographic Notes
Exercises
3 Multiple Linear Regression
3.1 Introduction
3.2 Description of the Data and Model
3.3 Example: Supervisor Performance Data
3.4 Parameter Estimation
3.5 Interpretations of Regression Coefficients
3.6 Properties of the Least Squares Estimators
3.7 Multiple Correlation Coefficient
3.8 Inference for Individual Regression Coefficients
3.9 Tests of Hypotheses in a Linear Model
3.10 Predictions
3.1 1 Summary
Exercises
Appendix: Multiple Regression in Matrix Notation
4 Regression Diagnostics: Detection of Model Violations
4.1 Introduction
4.2 The Standard Regression Assumptions
4.3 Various Types of Residuals
4.4 Graphical Methods
4.5 Graphs Before Fitting a Model
4.5.1 One-Dimensional Graphs
4.5.2 Two-Dimensional Graphs
4.5.3 Rotating Plots
4.5.4 Dynamic Graphs
Graphs After Fitting a Model
Checking Linearity and Normality Assumptions
Leverage, Influence, and Outliers
4.8.1
4.8.2 Outliers in the Predictors
4.8.3 Masking and Swamping Problems
Measures of Influence
4.9.1 Cook’s Distance
4.9.2 Welsch and Kuh Measure
4.9.3 Hadi’s Influence Measure
The Potential-Residual Plot
What to Do with the Outliers?
Role of Variables in a Regression Equation
4.12.1 Added-Variable Plot
4.12.2 Residual Plus Component Plot
Effects of an Additional Predictor
Robust Regression
Exercises
Outliers in the Response Variable
Qualitative Variables as Predictors
5.1 Introduction
5.2 Salary Survey Data
5.3 Interaction Variables
5.4 Systems of Regression Equations
5.4.1 Models with Different Slopes and Different Intercepts 130
5.4.2 Models with Same Slope and Different Intercepts 137
5.4.3 Models with Same Intercept and Different Slopes 138
5.5 Other Applications of Indicator Variables 139
5.6 Seasonality 140
5.7 Stability of Regression Parameters Over Time 141
Exercises 143
Transformation of Variables 151
6.1 Introduction 151
6.2 Transformations to Achieve Linearity 153
6.3 Bacteria Deaths Due to X-Ray Radiation 155
6.3.1 Inadequacy of a Linear Model 156
6.3.2 Logarithmic Transformation for Achieving Linearity 158
6.4 Transformations to Stabilize Variance
6.5 Detection of Heteroscedastic Errors
6.6 Removal of Heteroscedasticity
6.7 Weighted Least Squares
6.8 Logarithmic Transformation of Data
6.9 Power Transformation
6.10 Summary
Exercises
Weighted Least Squares
7.1 Introduction
7.2 Heteroscedastic Models
7.2.1 Supervisors Data
7.2.2 College Expense Data
7.3 Two-Stage Estimation
7.4 Education Expenditure Data
7.5 Fitting a Dose-Response Relationship Curve
Exercises
The Problem of Correlated Errors
Introduction: Autocorrelation
Consumer Expenditure and Money Stock
Durbin-Watson Statistic
Removal of Autocorrelation by Transformation
Iterative Estimation With Autocorrelated Errors
Autocorrelation and Missing Variables
Analysis of Housing Starts
Limitations of Durbin-Watson Statistic
Indicator Variables to Remove Seasonality
Regressing Two Time Series
Exercises
Analysis of Collinear Data
9.1 Introduction
9.2 Effects on Inference
9.3 Effects on Forecasting
9.4 Detection of Multicollinearity
9.5 Centering and Scaling
9.5.1
9.5.2 Scaling in No-Intercept Models
Centering and Scaling in Intercept Models
9.6 Principal Components Approach
9.7 Imposing Constraints
9.8
9.9 Computations Using Principal Components
9.10 Bibliographic Notes
Searching for Linear Functions of the P’s
Exercises
Appendix: Principal Components
10 Biased Estimation of Regression Coefficients
10.1
10.2
10.3
10.4
10.5
10.6
10.7
10.8
10.9
Introduction
Principal Components Regression
Removing Dependence Among the Predictors
Constraints on the Regression Coefficients
Principal Components Regression: A Caution
Ridge Regression
Estimation by the Ridge Method
Ridge Regression: Some Remarks
Summary
Exercises
Appendix: Ridge Regression
11 Variable Selection Procedures
Formulation of the Problem
Consequences of Variables Deletion
Uses of Regression Equations
1 1.4.1
1 1.4.2 Estimation and Prediction
1 1.4.3 Control
Criteria for Evaluating Equations
1 1.5.1 Residual Mean Square
11.5.2 Mallows C,
1 I S.3 Information Criteria: Akaike and Other Modified
Multicollinearity and Variable Selection
Evaluating All Possible Equations
11 3.1 Forward Selection Procedure
1 1.8.2 Backward Elimination Procedure
11 3.3 Stepwise Method
1 1.9 General Remarks on Variable Selection Methods
1 1.10 A Study of Supervisor Performance
1 1.1 1 Variable Selection With Collinear Data
1 1.12 The Homicide Data
1 1.1 Introduction
1 1.2
1 1.3
1 1.4
Description and Model Building
1 1.5
Forms
1 1.6
1 1.7
11.8 Variable Selection Procedures
1 1.13 Variable Selection Using Ridge Regression
1 1.14 Selection of Variables in an Air Pollution Study
1 1.15 A Possible Strategy for Fitting Regression Models
1 1.16 Bibliographic Notes
Exercises
Appendix: Effects of Incorrect Model Specifications
12 Logistic Regression
12.1
12.2
12.3
12.4
12.5
12.6
12.7
12.8
12.9
Introduction
Modeling Qualitative Data
The Logit Model
Example: Estimating Probability of Bankruptcies
Logistic Regression Diagnostics
Determination of Variables to Retain
Judging the Fit of a Logistic Regression
The Multinomial Logit Model
12.8.1 Multinomial Logistic Regression
12.8.2 Example: Determining Chemical Diabetes
12.8.3
12.8.4
Classification Problem: Another Approach
Exercises
Ordered Response Category: Ordinal Logistic
Regression
Example: Determining Chemical Diabetes Revisited
13 Further Topics
13.1 Introduction
13.2 Generalized Linear Model
13.3 Poisson Regression Model
13.4 Introduction of New Drugs
13.5 Robust Regression
13.6 Fitting a Quadratic Model
13.7 Distribution of PCB in U.S. Bays
Exercises
Appendix A: Statistical Tables
References