数据范儿_范老师-AI时代科研统计新范式-计量科研AI导航

1. Introduction: Two Concepts Separated by a Single Character, Worlds Apart

If you flip through the chapters on linear regression in any econometrics textbook, you will repeatedly encounter two terms that look almost identical but have completely different meanings—

Error and Residual.

The model is written as:

$Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$

That $\varepsilon_i$ is the error.

After you run your regression, Stata can generate a new variable for you called residual, which equals:

$\hat{\varepsilon}_i = Y_i - \hat{\beta}_0 - \hat{\beta}_1 X_i$

This is the residual.

Beginners easily mistake them for the same thing—aren't they both "the part the model doesn't explain"? Aren't the symbols both ε (one with a hat, one without)? When discussing assumptions about the error term ( $\varepsilon \sim N(0, \sigma^2)$ ), isn't it the residual plot that is used to test these assumptions?

But there is a fundamental difference between error and residual: one you can never observe, and the other you can compute every time you run a regression.

Confusing the two leads to a cascade of conceptual errors—including but not limited to: thinking that small residuals mean the model is correct, thinking that independent residuals imply independent errors, and mechanically equating diagnostic results on residuals with diagnostics on errors.

Core message: The error is an unobservable random disturbance in the theoretical model—a population concept; the residual is the difference between the observed value and the fitted value, computable from sample data—a sample concept. The residual is the "shadow" of the error—you can see the shadow, but you cannot conflate the shadow with the object that casts it.

2. Error—The Theoretical Quantity Underlying the Model

2.1 What Is the Error?

In the population regression model:

$Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$

$\varepsilon_i$ is defined as $Y_i - (\beta_0 + \beta_1 X_i)$ —that is, the vertical distance between the true Y value and the true population regression line.

Because $\beta_0$ and $\beta_1$ are unknown true parameters, $\varepsilon_i$ is also forever unobservable. It is not "something you haven't computed yet"—it is something that fundamentally cannot be computed.

2.2 What Does the Error Contain?—Three Components

The error term $\varepsilon_i$ is not a "trash can that holds anything." Theoretically, it contains three things:

Component One: Omitted Variables.

Wages depend not only on years of education but also on ability, family background, industry, region, luck… You will inevitably omit some of these to varying degrees. Variables that are omitted, systematically affect Y, and enter $\varepsilon$ .

This is why the error term is directly related to omitted variable bias (OVB): if an omitted variable affects both Y and is correlated with X, it will appear in both ε and X—leading to Cov(X, ε) ≠ 0 and biased OLS estimates.

Component Two: Measurement Error.

The X and Y you measure themselves contain error. Self-reported income is typically lower than true income, GDP accounting involves substantial estimation, and subjective ratings in surveys contain random fluctuations. These measurement errors also enter ε.

Component Three: Intrinsic Randomness.

Even if you measure all relevant variables accurately, human behavior itself contains irreducible random components. Two people who are identical in age, education, family background, and ability will still not have identical wages—because there exists genuine, irreducible individual randomness. This part is the "purest" form of error.

2.3 Econometric Methods Are Largely Built on Assumptions About the Error

Why do textbooks devote so much space to "assumptions about the error term"? Because all the desirable properties of OLS—unbiasedness, efficiency, validity of hypothesis tests—are built on assumptions about ε, not on assumptions about residuals.

The Five Classical Assumptions (Gauss-Markov Assumptions):

Assumption	Mathematical Statement	If Violated
① Zero Conditional Mean	$\mathbb{E}[\varepsilon \mid X] = 0$	OLS is biased and inconsistent—this is the most fatal
② Homoskedasticity	$\text{Var}(\varepsilon_i \mid X) = \sigma^2$ (constant)	OLS remains unbiased but is no longer efficient; standard errors are wrong
③ No Autocorrelation	$\text{Cov}(\varepsilon_i, \varepsilon_j) = 0$ (i≠j)	OLS is unbiased but standard errors are wrong
④ X Is Fixed (or Exogenous)	$\text{Cov}(X, \varepsilon) = 0$	OLS is biased—this is a direct corollary of Assumption ①
⑤ Normality (needed for small-sample inference)	$\varepsilon \sim N(0, \sigma^2)$	t-tests and F-tests are inaccurate in small samples

Note a key fact: all five assumptions are about ε (the error), not about residuals. But the only thing you can observe is the residual—this is the root of all tension.

3. Residual—The Only Clue You Have in Hand

3.1 What Is a Residual?

When you run an OLS regression on sample data, you obtain a fitted value $\hat{Y}_i = \hat{\beta}_0 + \hat{\beta}_1 X_i$ . The residual is the gap between the actual value and the fitted value:

$\hat{\varepsilon}_i = Y_i - \hat{Y}_i = Y_i - \hat{\beta}_0 - \hat{\beta}_1 X_i$

Note the notation: the error is $\varepsilon_i = Y_i - \beta_0 - \beta_1 X_i$ (using true parameters), while the residual is $\hat{\varepsilon}_i = Y_i - \hat{\beta}_0 - \hat{\beta}_1 X_i$ (using estimated parameters). The only difference is whether β or β̂ is used. That little hat makes all the difference.

3.2 What Are Residuals Used For?—Residual Diagnostics

This is the most important practical use of residuals. Because you cannot see ε, you can only indirectly infer whether the assumptions about the error hold by observing the behavior of residuals.

Common residual diagnostic tools:

Residual vs. fitted value plot: Checks homoskedasticity. If the spread of residuals expands as fitted values increase (fan shape), this is a typical signal of heteroskedasticity.
Q-Q plot of residuals: Checks normality. If the quantiles of residuals systematically deviate from the theoretical quantiles of a normal distribution, the error is likely not normally distributed.
Residual vs. independent variable plot: Checks for nonlinear relationships. If residuals exhibit a curved pattern (e.g., U-shaped) across the range of X, the linear functional form may be misspecified.
Autocorrelation function (ACF) plot of residuals: For time series data, checks whether residuals exhibit autocorrelation. If they do, the error likely exhibits autocorrelation as well.
Cook's Distance / leverage plot: Checks whether individual observations exert a disproportionate influence on the regression results (influence diagnostics).

The core logic of these diagnostics is: if residuals exhibit some systematic pattern, then the errors that generated those residuals most likely have the same problem—because if the model is correctly specified, the residual should be an "imperfect but reasonable approximation" of the error.

4. Residual ≠ Error: What Goes Wrong When You Confuse Them?

4.1 Core Differences at a Glance

Dimension	Error (ε)	Residual (ε̂)
Definition	$Y_i - (\beta_0 + \beta_1 X_i)$	$Y_i - (\hat{\beta}_0 + \hat{\beta}_1 X_i)$
World it belongs to	Population (theoretical model)	Sample (empirical computation)
Observability	Unobservable—because β is the unknown truth	Observable—computed as soon as you run the regression
Degrees of freedom	n freely varying quantities	n − k (k = number of parameters)—residuals are subject to k linear constraints
Mutual independence	Under classical assumptions, ε across observations are independent	Even if ε are independent, residuals are not independent of each other—they sum to zero and are constrained by β̂
Zero mean	Assumed $\mathbb{E}[\varepsilon] = 0$ (population mean)	Holds automatically: $\sum \hat{\varepsilon}_i = 0$ (as long as the model includes an intercept)
Relationship with X	Assumed Cov(X, ε) = 0	Holds automatically: Cov(X, ε̂) = 0 (OLS orthogonality condition)
Variance estimation	Variance $\sigma^2$ is unknown	$\frac{\sum \hat{\varepsilon}_i^2}{n-k}$ provides an unbiased estimator of $\sigma^2$

4.2 Danger One of Confusion: Treating the OLS Orthogonality Condition as "Evidence That Assumptions Are Satisfied"

OLS has a mathematical property: the sample covariance between residuals and X is automatically zero. $\sum X_i \hat{\varepsilon}_i = 0$ —this is not an assumption; it is a direct consequence of the OLS first-order conditions. You do not need to "test" it; it always holds.

But if someone confuses residuals with errors, they might mistakenly think: "Residuals are uncorrelated with X → the assumption Cov(X, ε) = 0 is satisfied → my OLS is unbiased."

This reasoning is completely wrong. The fact that residuals and X are uncorrelated in the sample is a mathematical necessity and has no logical connection to whether the error is truly unrelated to X. Cov(X, ε) = 0 is an assumption about the true error, and it cannot be "verified" using the sample properties of residuals—because residuals are forced to be orthogonal to X.

This confusion is one of the most dangerous logical errors in econometrics.

4.3 Danger Two of Confusion: Believing Residuals Are Independent of Each Other

Under the classical assumptions, the errors ε₁, ε₂, ..., εn across different observations are assumed to be mutually independent. But even if this assumption holds, residuals are not independent of each other.

This is because residuals satisfy two constraints (in simple regression):

$\sum_{i=1}^{n} \hat{\varepsilon}_i = 0, \quad \sum_{i=1}^{n} X_i \hat{\varepsilon}_i = 0$

This means—if you know the first n−2 residuals and the values of X, you can determine the last two residuals exactly. Therefore, residuals must exhibit correlation among themselves—even if the errors have no relationship whatsoever.

So you cannot use the sample correlation of residuals to "prove" that errors are free of autocorrelation—residuals have inherent structural correlation that has nothing to do with the errors.

4.4 Danger Three of Confusion: Directly Treating the Distribution of Residuals as the Distribution of Errors

The distribution of residuals is more concentrated than the distribution of errors—because OLS has already "extracted" some of the variation in the sample to estimate parameters. In estimating β, the residuals "sacrifice" k degrees of freedom (k = number of parameters).

In other words, $\text{Var}(\hat{\varepsilon}_i) < \text{Var}(\varepsilon_i) = \sigma^2$ . Residuals fluctuate less than errors—because the OLS fit has already "tracked" some of the variation. This is why σ² is estimated using $\frac{\sum \hat{\varepsilon}_i^2}{n-k}$ rather than $\frac{\sum \hat{\varepsilon}_i^2}{n}$ —the denominator in the latter is too large and would systematically underestimate the variance of the error.

If you confuse the variances of the two, you will underestimate σ², which in turn underestimates standard errors, producing spuriously over-optimistic significance.

5. The Connection Between Residuals and Errors—The Residual Is the Error's "Shadow"

Having discussed the differences at length, residuals are of course not entirely unrelated to errors. The bridge between them lies in the following:

5.1 Residuals Can Be Used to Estimate the Variance of the Error

The variance of the error, $\sigma^2 = \text{Var}(\varepsilon_i)$ , is an unknown population parameter. Its unbiased estimator is:

$\hat{\sigma}^2 = \frac{\sum_{i=1}^{n} \hat{\varepsilon}_i^2}{n - k}$

where k is the number of regression parameters (including the intercept). This is the residual sum of squares divided by the degrees of freedom—the "RMSE" or "Root MSE" in OLS output is precisely $\sqrt{\hat{\sigma}^2}$ .

The importance of this formula cannot be overstated. It is the source of OLS standard errors:

$\text{se}(\hat{\beta}_1) = \sqrt{\frac{\hat{\sigma}^2}{\sum (X_i - \bar{X})^2}}$

Without residuals, you cannot compute standard errors. Without standard errors, you cannot conduct hypothesis tests. This is the lifeline connecting residuals to statistical inference.

5.2 The Validity of Residual Diagnostics: Why Looking at Errors Through Residuals Is Reasonable

Although residual ≠ error, under the premise that the model is correctly specified, the behavior of residuals should "resemble" the errors. If the errors are truly independently and identically distributed with constant variance, then the residuals (apart from the few known structural differences) should also look roughly that way.

Therefore, the validity of residual diagnostics rests on a "meta-assumption"—your model is broadly correct. Under this premise, severely abnormal residuals (such as pronounced fan shapes, curvature, or strong autocorrelation) constitute legitimate grounds for questioning the model specification or the assumptions about the error.

Conversely, if you already know the model specification may be problematic (e.g., a key variable is omitted), then the results of residual diagnostics may themselves be misleading—the residuals may contain systematic patterns arising from the relationship between the omitted variable and X.

6. Summary: One Table to Settle It All

What you want to know…	What you look at…	Because…
Is the error homoskedastic?	Residual vs. fitted value plot	If the error is homoskedastic, residuals should also be evenly spread
Is the error normal?	Q-Q plot of residuals	If the error is normal, residuals should also be approximately normal (biased in small samples, but still informative)
Is the error autocorrelated?	ACF plot of residuals	If the error is not autocorrelated, residuals should not show obvious autocorrelation patterns either
What is the variance of the error, $\sigma^2$ ?	$\frac{\sum \hat{\varepsilon}_i^2}{n-k}$	This is the unbiased estimator of $\sigma^2$

But remember—you cannot use residuals to "verify" Cov(X, ε) = 0. Because Cov(X, ε̂) = 0 is a mathematical identity automatically guaranteed by OLS. That is not diagnosis; that is circular reasoning.

One sentence to close:

"The error is the truth you cannot see; the residual is the clue in your hand. You can never equate the clue with the truth—but a good detective must learn to infer the truth from the clues. The core competence of an econometrician is to maintain this vigilance of 'both believing and doubting' when facing residuals."

7. Presentation Suggestions for Bilibili / WeChat Official Account

Bilibili video: For the core visual, use the "shadow metaphor"—the center of the frame shows the true population regression line and the error (displayed as a dashed/semi-transparent line, implying invisibility), while visible residual segments (solid lines) appear at the sample data points. Voiceover: The error is what you cannot see; the residual is the only thing you can see. The "degrees of freedom difference" between the two can be shown with animation—from n residuals, k degrees of freedom are "locked" into parameter estimation, leaving n−k independent pieces of information.
WeChat article: The four-dimension comparison table (world it belongs to / observability / degrees of freedom / relationship with X) is recommended as an infographic. For the three dangers of confusion, pair each with a "wrong reasoning → correct reasoning" comparison card.
Recommended titles:
- Main title: 《Residuals and Errors Differ by Only One Character, Yet They Are Worlds Apart》
- Alternative title: 《What You See After Running a Regression Is the Residual—The True Error You Can Never See》
Key quote:

"The error is the truth you cannot see; the residual is the clue in your hand. You can never equate the clue with the truth—but a good detective must learn to infer the truth from the clues."