What Is Regression Analysis? Types, Assumptions, and Procedure
Daniel Ha · Seoul National University PhD StudentWhat Is Regression Analysis?
Regression analysis estimates the size and direction of the effect that one or more independent variables (causes) have on a dependent variable (outcome), expressed as a regression equation. Where correlation only measures how strongly two variables move together, regression answers how much Y changes when X increases by one unit, and it supports prediction.
In papers it is used mainly for hypothesis testing. A hypothesis such as "job satisfaction has a positive effect on organizational commitment" is tested through the sign of the regression coefficient and its p-value.
When Should You Use Regression Analysis?
Use it when the dependent variable is continuous and you want to test or predict an effect relationship between variables. If your goal is simply comparing group means, a t-test or ANOVA fits better.
| Situation | Appropriate method |
|---|---|
| Several factors affecting a continuous outcome | Multiple regression |
| Binary outcome (pass/fail, churn/retain) | Logistic regression |
| Incremental explanatory power by entry order | Hierarchical regression |
| Mean difference between two groups only | t-test (not regression) |
What Types of Regression Analysis Are There?
There are four main types, distinguished by the number of predictors and the form of the outcome variable.
| Type | Independent variables | Dependent variable | Typical use |
|---|---|---|---|
| Simple regression | 1 | Continuous | Effect of a single variable |
| Multiple regression | 2 or more | Continuous | Simultaneous effects of several factors |
| Logistic regression | 1 or more | Binary | Probability of an event (odds ratio) |
| Hierarchical regression | Entered in blocks | Continuous | Added explanatory power (ΔR²) over controls |
In social science papers especially, hierarchical regression — entering demographic controls first and then the focal variables to see the increase in explanatory power — is common, as is testing mediation and moderation with tools such as the PROCESS macro.
What Are the Assumptions of Regression Analysis?
Five assumptions must hold before you can trust the results: linearity, independence of errors, homoscedasticity, normality, and absence of multicollinearity. Violations distort both coefficients and significance, so check them before interpreting anything.
| Assumption | How to check |
|---|---|
| Linearity | Scatterplots, residual plots |
| Independence of errors | Durbin-Watson statistic (near 2) |
| Homoscedasticity | Residuals vs. predicted values plot |
| Normality | Q-Q plot of residuals, skewness/kurtosis |
| No multicollinearity | VIF (below 10, or below 5 to be safe) |
How Do You Run a Regression Analysis?
Check assumptions, fit the model, then interpret model fit and coefficients in that order.
Step 1: Specify Variables and Check Assumptions
Define the dependent and independent variables, then verify the five assumptions above with scatterplots and VIF. If the outcome is binary, switch to logistic regression.
Step 2: Fit the Model and Check Model Fit
Estimate the regression equation and examine fit. R² is the proportion of variance in the dependent variable explained, adjusted R² corrects for the number of predictors, and the F-test judges the significance of the model as a whole.
Step 3: Interpret the Coefficients
Read the unstandardized coefficient (B) for the size of each effect and the standardized coefficient (β) to compare the relative influence of predictors. A p-value below the threshold (usually .05) indicates a statistically significant effect.
Step 4: Report the Results
Present B, standard error, β, t, p, R², and F together in a table, and state the results of your assumption checks in the text or a footnote.
How High Does R² Need to Be?
There is no absolute cutoff — it depends on the field. Models of human behavior in the social sciences can be meaningful with an R² of 0.1 to 0.3, while engineering and the natural sciences expect higher values. Do not judge a model by R² alone, but weigh it together with satisfied assumptions and theoretical soundness.
What Is the Difference Between Multiple and Hierarchical Regression?
Multiple regression enters all predictors at once, while hierarchical regression enters them in theoretically ordered blocks so you can see the increment in explanatory power (ΔR²) at each step. Use hierarchical regression when you want the unique contribution of your focal variables after removing the influence of control variables.
What Are Common Mistakes in Regression Analysis?
Skipping assumption checks, ignoring multicollinearity, and overstating causality are the most common. In particular, regression results from observational or survey data should never be declared causal — describe them as an effect or an association with appropriate care.
Summary
Regression analysis follows the sequence check assumptions, fit the model, interpret coefficients, report, and the type you choose depends on the form of the outcome (continuous or binary) and your analytic goal (incremental explanatory power, mediation, moderation). Judge the result not by R² alone but by satisfied assumptions and theoretical coherence.
Related methods include correlation analysis for the relationship between two variables and structural equation modeling for testing causal structures among several latent variables at once.