数据范儿_范老师-AI时代科研统计新范式-计量科研AI导航

1. Introduction: In Multiple Regression, What Does "Controlling for Other Variables" Actually Mean?

Everyone who has studied econometrics will say one thing: "In multiple regression, the coefficient is the partial effect of X on Y, holding other variables constant."

But what does this statement actually mean at the implementation level? When the computer runs reg y x1 x2 x3, how does it "control" for x2 and x3? Does it fix x2 and x3 at some value? Does it find individuals in the sample who have the same x2 and x3 to make comparisons?

None of these.

The FWL Theorem (Frisch-Waugh-Lovell Theorem) provides the precise mathematical answer to this question. Moreover, this answer not only explains how multiple regression works but also directly connects to several of the most important methods in econometrics—two-stage least squares (2SLS), Double Machine Learning, panel fixed effects estimation, and even the construction of partial regression plots.

Core message: The FWL Theorem states that in a multiple regression, the coefficient β̂₁ of X₁ is equivalent to a "three-step" procedure: (1) regress X₁ on all other X's, take the residuals (the part of X₁ not explained by other variables); (2) regress Y on all other X's, take the residuals (the part of Y not explained by other variables); (3) regress the Y residuals on the X₁ residuals in a simple regression. This is the mathematical essence of "controlling"—not "fixing" but "partialling out": strip away from both X₁ and Y everything that can be linearly explained by other variables, and the remaining "net" versus "net" relationship is the partial effect.

2. Formal Statement of the FWL Theorem

2.1 Theorem Statement

Consider the multiple regression model:

$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_k X_k + \varepsilon$

The coefficient of interest is β₁. The FWL Theorem asserts that the β̂₁ obtained from the following two approaches are numerically identical:

Approach A (direct multiple regression): Run OLS of Y on all X₁, X₂, ..., Xₖ, and take β̂₁.

Approach B (three-step procedure):

Step 1: Regress X₁ (as the dependent variable) on all other X's (i.e., X₂, ..., Xₖ), and extract the residuals $\tilde{X}_1$ :

$\tilde{X}_1 = X_1 - \hat{X}_1 \quad \text{(where } \hat{X}_1 \text{ is X₁ predicted by the other X's)}$

This residual $\tilde{X}_1$ is the "unique part" of X₁ that cannot be linearly explained by the other X's.

Step 2: Regress Y (as the dependent variable) on the same set of other X's, and extract the residuals $\tilde{Y}$ :

$\tilde{Y} = Y - \hat{Y} \quad \text{(where } \hat{Y} \text{ is Y predicted by the other X's)}$

This residual $\tilde{Y}$ is the "unique part" of Y that cannot be linearly explained by the other X's.

Step 3: Regress $\tilde{Y}$ on $\tilde{X}_1$ in a simple regression (without an intercept):

$\tilde{Y} = \beta_1 \tilde{X}_1 + u$

The coefficient obtained is exactly equal to β̂₁ from Approach A.

$\hat{\beta}_1^{\text{FWL}} = \frac{\sum \tilde{X}_{1i} \tilde{Y}_i}{\sum \tilde{X}_{1i}^2} = \hat{\beta}_1^{\text{OLS}}$

2.2 Conditions for the Theorem to Hold

FWL holds exactly for all OLS regressions—it is an algebraic property of OLS, not an asymptotic property, and does not require large-sample assumptions. Any coefficient in any linear regression model can be obtained through the above "three-step residual method."

However, there is one key qualification: the "other X's" used in Step 2 and Step 1 must be the same set of variables. That is, when X₁ and Y are "purified," they must use exactly the same set of control variables.

3. Intuitive Understanding—"Purification" Is More Accurate than "Control"

3.1 Understanding Through Geometry

Imagine a three-dimensional space. Y is the vertical axis, and X₁ and X₂ are two horizontal axes. Data points are scattered throughout this three-dimensional space.

When you run reg y x1 x2, what is the geometric meaning of β̂₁?

FWL tells you:

First, project all points in the direction of the X₂ axis—this is equivalent to "removing the linear influence of X₂ from both X₁ and Y."
The residuals $\tilde{X}_1$ and $\tilde{Y}$ after projection lie in the plane orthogonal to X₂.
In this plane, regress $\tilde{Y}$ on $\tilde{X}_1$ in a simple regression—this gives you β̂₁.

"Controlling" is essentially an "orthogonal projection"—simultaneously stripping the linear influence of other variables from both your X of interest and Y, then searching for the relationship in the remaining residuals.

3.2 The "Dividing the Cake" Analogy

Think of the variation in Y as a cake. You have three variables X₁, X₂, X₃ that want to divide this cake.

X₂ reaches in first and takes a large piece (the part of Y linearly explained by X₂). X₃ also takes a piece (the part of Y linearly explained by X₃, specifically the portion that does not overlap with what X₂ took). The remaining crumbs are $\tilde{Y}$ —the part of Y that cannot be explained by X₂ and X₃.

But X₁ doesn't come to the cake intact either. Part of X₁ has also been "taken away" by X₂ and X₃—the part of X₁ linearly explained by X₂ and X₃ is $\hat{X}_1$ , and what remains, $\tilde{X}_1$ , is what is truly "unique" to X₁.

β̂₁ measures: how much of Y's unique part ( $\tilde{Y}$ ) can be explained by X₁'s unique part ( $\tilde{X}_1$ ).

3.3 A Concrete Numerical Example

Suppose you study "wage (Y) ~ education (X₁) + work experience (X₂)."

Person	Wage (Y)	Education (X₁)	Experience (X₂)
A	5000	12	5
B	8000	16	10
C	6000	14	8

Approach A: Direct multiple regression. reg Y X₁ X₂ → β̂₁ = some value.

Approach B: The FWL three steps.

Step 1: Regress X₁ on X₂ → $\tilde{X}_1$ = education residuals = the part of education "unrelated to experience." If education and experience are correlated (people with more years of education typically also have more experience), then $\tilde{X}_1$ is actually "abnormal education"—for example, you have 16 years of education, but given your experience level, you "should" have only 13 years; the extra 3 years constitute $\tilde{X}_1$ .

Step 2: Regress Y on X₂ → $\tilde{Y}$ = wage residuals = the part of wages "unrelated to experience."

Step 3: Regress $\tilde{Y}$ on $\tilde{X}_1$ → the coefficient obtained is the partial effect of education on wages "after purging the influence of work experience." This value is exactly equal to the education coefficient from the multiple regression in Approach A.

4. Four Important Implications of the FWL Theorem—Each Is Crucial for Understanding Econometric Methods

Implication 1: The Coefficient in a Multiple Regression Is a Simple Regression Coefficient—on "Purified" Data

This implication sounds simple, but its significance is profound. Any multiple regression is essentially a simple regression—except that the data and variables have been "cleaned" in advance.

In econometric software, reg y x1 x2 x3 is mathematically equivalent to three FWL decompositions: the coefficient on X₁ comes from a simple regression of "purified Y" on "purified X₁"; the coefficient on X₂ comes from a simple regression of "purified Y" on "purified X₂" (where purification removes the influence of X₁ and X₃); and so on.

This explains a common experience: when you add a new variable to your regression that is highly correlated with X₁, the standard error of β̂₁ increases. Because the new variable takes away more of X₁'s variation during the "purification" of X₁—the variance of $\tilde{X}_1$ decreases → in the simple regression of Step 3, the denominator shrinks → the standard error increases.

Implication 2: "Removing" the Linear Effect of Control Variables ≠ "Fixing" Control Variables

Many people understand "controlling" as "keeping the control variable constant." This intuition is not fundamentally wrong conceptually, but it is mathematically imprecise.

FWL tells you: multiple regression is not about finding people with "the same X₂" to compare—it is about removing the linear projection of X₂. If the relationship between X₂ and Y is not linear (e.g., U-shaped), the residual $\tilde{Y}$ from a linear projection still contains the nonlinear components of X₂. This means—when you "control" for X₂, you only control its linear component. The nonlinear component remains in the residual.

This directly leads to an important practical reminder: if your control variable has a nonlinear relationship with Y, a simple linear multiple regression cannot fully "control" for it. You need to include X₂² or use a more flexible functional form.

Implication 3: The Coefficient β̂₁ Is Entirely Driven by the "Unique Variation" of X₁

$\text{Var}(\hat{\beta}_1) = \frac{\sigma^2}{\sum \tilde{X}_{1i}^2} = \frac{\sigma^2}{\text{SST}_1 \cdot (1 - R_1^2)}$

where $R_1^2$ is the R² from regressing X₁ on the other X's.

The denominator $\sum \tilde{X}_{1i}^2$ is the "unique variation" of X₁—it equals the total variation of X₁ multiplied by $(1 - R_1^2)$ . If other variables can explain 95% of X₁'s variation, then the "effective sample information" available for estimating β₁ is only 5% of the original.

This is also the meaning of VIF = $1/(1 - R_1^2)$ from the FWL perspective: VIF measures "how much the denominator in Step 3 of FWL has shrunk."

Implication 4: The Asymmetry Between the Core Variable and Control Variables

FWL reveals an important asymmetry in multiple regression:

The "purification" of X₁ uses all linear combinations of X₂ through Xₖ. Thus, X₁ is stripped of everything that can be linearly explained by other variables.
The "purification" of Y also uses all linear combinations of X₂ through Xₖ. But Y may contain other things—such as nonlinear functions of X₂—that are related to X₂ but cannot be fully captured by linear terms of X₂. These things remain in $\tilde{Y}$ .

This means: you can only "control" other variables to the extent that they can be captured linearly. Anything beyond the linear range is not controlled. This is why linear regression is always an approximation, and why modern causal inference methods (such as Double ML) use machine learning to "purify" more flexibly—this will be elaborated in Section 7.

5. The FWL Theorem and Two-Stage Least Squares (2SLS)—2SLS Is "FWL with Instrumental Variables"

5.1 The Two Stages of 2SLS Are the Two Residual Steps of FWL

The standard statement of two-stage least squares is:

First stage: Regress the endogenous variable X on the instrumental variables Z (and exogenous control variables W), obtaining the predicted values $\hat{X}$ .
Second stage: Regress Y on $\hat{X}$ (and W), obtaining β̂₁.

But from the FWL perspective, what 2SLS does is clearer:

In fact, 2SLS is equivalent to a "FWL + IV" combined operation:

If we rewrite 2SLS in the FWL manner:

Step 1: Regress X on all exogenous control variables W, take residuals $\tilde{X}$ (the part of X not explainable by W).
Step 2: Regress Z on all exogenous control variables W, take residuals $\tilde{Z}$ (the part of Z not explainable by W—i.e., the "unique variation" of the instrument).
Step 3: Use $\tilde{Z}$ as an instrumental variable for $\tilde{X}$ in an IV estimation.

This FWL form of 2SLS reveals:

$\hat{\beta}_1^{\text{2SLS}} = \frac{\sum \tilde{Z}_i \tilde{Y}_i}{\sum \tilde{Z}_i \tilde{X}_i}$

where $\tilde{Y}$ is the residual from regressing Y on W, $\tilde{X}$ is the residual from regressing X on W, and $\tilde{Z}$ is the residual from regressing Z on W.

Intuitive meaning: 2SLS is essentially, "after removing the linear influence of all exogenous control variables W, using the instrument's residuals to identify the causal effect of the endogenous variable's residuals on the outcome's residuals."

5.2 Why Is Understanding This Important?

First, it helps you understand why 2SLS requires "exogenous control variables W" to appear in both stages. FWL requires that X₁ and Y be purified by the same set of variables—if you control for W in the first stage but not in the second, the β̂₁ you obtain has a different meaning from that in the full model.

Second, it explains why the first-stage F-statistic tests the explanatory power of $\tilde{Z}$ for $\tilde{X}$ , not the raw explanatory power of Z for X. The F-test after FWL purification truly measures "how much of the unique variation in the endogenous variable can be predicted by the unique variation of the instrument"—correlations before adding control variables don't count, as those may be driven by the control variables.

Third, it paves the way for understanding the Control Function approach. The control function method is essentially, within the FWL framework, putting the first-stage residuals (the part of $\tilde{X}$ not explained by $\tilde{Z}$ ) back into the Y equation as additional control variables—using residuals to "absorb" endogeneity rather than using predicted values to "replace" the endogenous variable.

6. The FWL Theorem and Double Machine Learning—Using ML to Replace Linear Projection

6.1 The Core Idea of Double ML—It's FWL, Just with an Upgraded "Purification" Method

The Double/Debiased Machine Learning (DML) method proposed by Chernozhukov et al. (2018) is one of the most important methodological advances in causal inference in recent years. Its core logic is a nonlinear upgrade of the FWL Theorem:

	Classical FWL (OLS)	Double ML
Purify Y	Regress Y on W with a linear model, take residuals	Regress Y on W with any ML model (random forest, LASSO, neural networks, etc.), take residuals
Purify X	Regress X on W with a linear model, take residuals	Regress X on W with any ML model, take residuals
Estimate β₁	Simple linear regression of $\tilde{Y}$ on $\tilde{X}_1$	Simple linear regression of $\tilde{Y}$ on $\tilde{X}_1$
Flexibility of "control"	Only controls linear effects of W	Controls arbitrary nonlinear functional forms and interaction effects of W

Double ML is essentially "using machine learning to run the first two steps of FWL."

6.2 Why Is "Double" Needed? (Why Must Both X and Y Be Purified?)

This is the answer directly given by the FWL Theorem. If you only purify Y but not X, the coefficient you obtain is not the partial effect—because part of X can still be explained by W, and if this part is correlated with the Y residuals, the meaning of the coefficient is not clean.

If you only purify X but not Y, the same logic applies—the effect of W in Y has not been stripped away, and the X residuals may simultaneously capture the part of the Y residuals that belongs to W.

Both sides must be purified—this is not a "ritual" of Double ML, but a mathematical requirement of the FWL Theorem. Only when X and Y are purified using the same set of W (and with the same functional form) will the coefficient from the residual-on-residual regression be correct.

6.3 Sample Splitting—An Explanation from the FWL Perspective

A hallmark operation of Double ML is cross-fitting: split the sample into K folds, use K-1 folds to train the ML model for purifying X and Y, and use the remaining fold's residuals to estimate β₁. Rotate K times and take the average.

Why is this necessary? The FWL perspective provides an intuitive explanation:

In classical FWL, "purification" and "estimation" use the same sample—this is not a problem because linear regression has limited in-sample overfitting. But if you use flexible ML models (such as deep tree models) for purification, purifying and estimating on the same sample leads to overfitting bias—the ML model will "purify" away noise as well, causing the variances of $\tilde{X}_1$ and $\tilde{Y}$ to be underestimated and biasing the distribution of β̂₁.

Sample splitting ensures that the "sample used for purification" and the "sample used for estimation" are independent—this severs the pathway through which overfitting propagates from the purification step to the estimation step.

In one sentence: Double ML = FWL Theorem + flexible ML purification + sample splitting to prevent overfitting. FWL is the skeleton, ML is the muscle, and sample splitting is the protective shield.

7. What Else Can the FWL Theorem Help You Understand?

7.1 Within Estimation in Panel Fixed Effects

A classic implementation of the fixed effects model is the within transformation—subtract each variable's individual mean, then run OLS on the transformed data.

But there is an equivalent implementation—LSDV (Least Squares Dummy Variables): run OLS directly on the raw data, but include N-1 individual dummy variables.

Why are these two approaches equivalent? The FWL Theorem.

Because in LSDV, the dummy variables are the "other X's," and the coefficient on X is equivalent to: first regress X on the individual dummy variables (→ predicted by individual means), take residuals (→ deviations from individual means, i.e., within deviations); do the same for Y. Then regress Y's within deviations on X's within deviations. This is the within estimator.

FWL lets you see the equivalence at a glance—and why fixed effects only use "within variation" to identify coefficients.

7.2 Added Variable Plots / Partial Regression Plots

In Stata, the avplot command produces a scatter plot: the X-axis is $\tilde{X}_1$ (residuals from regressing X₁ on the other X's), the Y-axis is $\tilde{Y}$ (residuals from regressing Y on the other X's), with a regression line overlaid—the slope of this line is β̂₁.

The diagnostic value of this plot lies in:

Outliers: outliers appearing in the FWL residual space have unusually high leverage on β̂₁.
Nonlinear relationships: if the fitted line is clearly not the best description of the data, it may suggest that you should include nonlinear terms of X₁ in the model.
Influential points: individual points that look normal in the raw data but are outliers in the residual space may have a disproportionate influence on β̂₁—something you cannot see in a scatter plot of the raw variables.

FWL helps you understand—avplot is not plotting two different variables; it is literally plotting your regression coefficient.

7.3 High-Dimensional Fixed Effects and Demeaning

Suppose your model includes "firm fixed effects" and "year fixed effects"—two sets of high-dimensional dummy variables. Running OLS directly would require inverting a matrix with thousands of dummy variables—which may be computationally infeasible.

Solution: Iterative FWL. First take residuals of X on year fixed effects, then on firm fixed effects (the order can be swapped, iterating until convergence). This is the core algorithm of the reghdfe command (the most popular high-dimensional fixed effects estimation package in Stata)—it does not directly invert the matrix but instead "absorbs" the fixed effects through repeated residual extraction.

7.4 Multicollinearity Diagnostics (VIF)

As discussed in Implication 3 of Section 4, VIF is the scaling factor for the denominator in Step 3 of FWL. A variable's VIF tells you—how much variation information this variable loses during the purification step of FWL.

7.5 The Relationship Between Matching Methods and Regression

Propensity score matching (PSM) and multiple regression are often compared. FWL reveals their shared intellectual core:

Regression: uses linear projections of other X's to "purify" X₁ and Y.
Matching: uses other X's to find the "most similar" individuals (similar in propensity score), then compares Y within the matched subsample.

The common logic of both is FWL—first remove the influence of confounders from the comparison, then examine the relationship between X and Y. The only difference is the method of removal: regression uses linear projection, matching uses stratification/pairing.

8. Summary

Five core takeaways from the FWL Theorem:

β̂₁ = a simple regression of "purified Y" on "purified X₁." Every coefficient in a multiple regression is mathematically equivalent to the coefficient from a simple regression after two rounds of residual extraction.
The mathematical essence of "controlling" is "orthogonal projection," not "holding fixed." FWL says: you simultaneously strip the linear influence of other X's from both X₁ and Y, and what remains is the partial effect. This is not "fixing"; this is "partialling out."
2SLS is FWL plus an instrumental variable. Both stages are FWL residual extractions, except that in the final step you use $\tilde{Z}$ (rather than $\tilde{X}$ itself) to estimate the coefficient. Once you understand FWL, the structure of 2SLS is no longer mysterious.
Double ML is FWL + ML purification + sample splitting. Flexible machine learning models replace linear regression for residual extraction (the first two steps), and sample splitting prevents overfitting—but the skeleton remains FWL.
FWL is a "unifying field"—fixed effects, partial regression plots, VIF, and matching methods can all be understood within the FWL framework. It is not a "corner piece of knowledge" in econometrics but the backbone running through the entire linear model system.

One sentence to close:

"The FWL Theorem tells you: behind every coefficient in a multiple regression stands a humble simple regression—except that this simple regression does not use the raw data but residuals that have been 'cleaned' by other variables. Once you understand FWL, you understand what 'controlling' means mathematically. And from 2SLS to Double ML, all more complex methods are merely different 'detergents' used in this cleaning process."

9. Presentation Suggestions for Bilibili/WeChat Official Account

Bilibili video: It is recommended to use "washing/purification" as the visual metaphor running through the entire video. Opening: a three-dimensional coordinate system with data points scattered along the Y-axis, X₁-axis, and X₂-axis. Then a beam of light shines in the X₂ direction—data points project onto the X₂ axis, and the "shadow" parts of X₁ and Y (the parts explainable by X₂) are stripped away. The remaining residuals lie in the plane orthogonal to X₂—in this plane, the simple regression line of residuals on residuals is β̂₁. Then unfold three "upgrade packages": Upgrade Package 1 (2SLS): the instrument's residuals serve as a "stand-in" for the X residuals to estimate the coefficient. Upgrade Package 2 (Double ML): replace the linear projection with an ML model (using a flexible surface instead of a rigid plane to strip away effects). Upgrade Package 3 (Fixed Effects): the projection of individual dummy variables → is within-demeaning. At the end, return to the original three-dimensional plot, with narration: "All the methods you have learned are doing the same thing—simultaneously washing the influence of confounders off X and Y, then searching for causality in what remains. FWL is the mathematical manual for this 'cleaning.'"
WeChat Official Account: For the FWL three steps, it is recommended to create a vertical flowchart (raw data → purify X → purify Y → residual-on-residual regression). The correspondence between 2SLS and FWL should be presented as a two-column comparison table (left column: FWL three steps; right column: 2SLS two stages, aligned row by row). Double ML vs. classical FWL should be presented as a three-column comparison chart (classical FWL / Double ML / why an upgrade is needed). For FWL's "unifying field" (the FWL perspective on seven methods), a center-radial diagram is recommended—FWL in the center, with seven rays connecting to fixed effects, 2SLS, Double ML, VIF, partial regression plots, matching, and high-dimensional fixed effects. For the partial regression plot (avplot), it is recommended to include a real Stata output screenshot.
Recommended titles:
- Main title: 《The FWL Theorem: The Mathematical Truth Behind "Controlling for Other Variables" in Multiple Regression》
- Alternative title: 《From FWL to Double ML—Why "Purification" Is the Key to Understanding All Causal Methods?》
- New media title: 《How Does "Controlling for Variables" in Regression Actually Work? The FWL Theorem Explains It in Three Steps》
Key quotes:

"Every coefficient in a multiple regression is mathematically equivalent to a humble simple regression—except that this simple regression does not use the raw data but residuals that have been 'cleaned' by other variables."

"2SLS is FWL plus an instrumental variable. Double ML is FWL with linear purification upgraded to machine learning purification. Once you understand FWL, these methods are no longer isolated 'techniques' but different branches growing from the same tree."

"The action of 'controlling' is called controlling for in English, but FWL tells you that its true mathematical operation is partialling out—not holding something still, but projecting away the part that does not belong to you."

"FWL helps you understand why both X and Y need to be purified—you use the residuals of residuals to identify the effect. The order of subtraction can be swapped, but both sides must be subtracted. The word 'double' in Double ML originates precisely here."