数据范儿_范老师-AI时代科研统计新范式-计量科研AI导航

1. Introduction: One Term, Two Meanings, Two Entirely Different Questions

"Linear regression" — at first glance, the name seems self-evident — isn't it just drawing a straight line?

But if you have carefully looked at different types of regression models, you have certainly encountered this confusion:

$Y = \beta_0 + \beta_1 X + \varepsilon$ → This is a straight line, OK.
$Y = \beta_0 + \beta_1 \ln(X) + \varepsilon$ → This is not a straight line between X and Y (Y increases with X, but at a decreasing rate — a curve). Yet it is still called a "linear regression model."
$Y = \beta_0 + \beta_1 X + \beta_2 X^2 + \varepsilon$ → This is clearly not a straight line between X and Y (it is a parabola). But it is also a "linear regression model."

What? Things that are clearly curves can also be called "linear"?

This brings us to the core proposition of this article: In econometrics, "linear" refers to "linear in parameters," not "linear in X." These two notions of "linearity" answer questions at two different levels — one concerns "what the graph looks like," and the other concerns "whether OLS can be used for estimation."

Key takeaway: Linear in X means "a one-unit change in X produces a constant change in Y" — this is constancy of the marginal effect. Linear in parameters means "the parameters enter the model as additive first-power terms, with no β², no e^β" — this is about the applicability of the estimation method. The vast majority of regression models with curved shapes you encounter are linear in parameters, and therefore can be estimated by OLS.

2. Linear in X — The Straight Line You Draw on a Scatter Plot

2.1 Meaning: Constant Marginal Effect

The model $Y = \beta_0 + \beta_1 X + \varepsilon$ is called "linear in X" because:

$\frac{\partial Y}{\partial X} = \beta_1 \quad \text{(a constant)}$

When X goes from 1 to 2, Y increases by β₁. When X goes from 100 to 101, Y still increases by β₁.

The marginal effect does not vary with the starting value of X — no matter where you observe X, the effect of one additional unit of X on Y is exactly the same.

2.2 Graphically — It Is a Straight Line

What you see on the scatter plot is a straight line with slope β₁. This is the most intuitive notion of "linearity" — the linearity your eyes tell you about.

2.3 But Many Real-World Relationships Are Not Linear in X

The effect of education on wages: one more year of primary school and one more year of PhD have effects on wages of very different magnitudes.
The effect of income on consumption: when income rises from 10,000 to 20,000, consumption changes a lot; when income rises from 1 million to 1.01 million, consumption barely changes.
The effect of study time on exam scores: going from 0 hours to 1 hour has a significant effect; going from 10 hours to 11 hours has a negligible effect.

In these cases, insisting on "linear in X" would incorrectly assume a constant marginal effect. You need a way to represent "marginal effects that vary with X" — without giving up the convenient and powerful tool that is OLS.

This is where "linear in parameters" comes into play.

3. Linear in Parameters — Can OLS Be Used for Estimation?

3.1 Meaning: Parameters Enter as First-Power Additive Terms

A model is linear in parameters if and only if each of its parameters enters the model as an independent term, with each parameter raised to the first power and added to the other parameters (rather than multiplied, divided, or nested inside a function).

Consider a few examples:

Model	Linear in parameters?	Reason
$Y = \beta_0 + \beta_1 X$	✅ Yes	β₀ and β₁ are both first-power and additive
$Y = \beta_0 + \beta_1 \ln(X)$	✅ Yes	ln(X) is nonlinear, but β₁ is still first-power
$Y = \beta_0 + \beta_1 X + \beta_2 X^2$	✅ Yes	X² is nonlinear, but β₁ and β₂ are both first-power
$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 (X_1 \times X_2)$	✅ Yes	The interaction term (X₁×X₂) is nonlinear in the X's, but β₃ is still first-power
$Y = \beta_0 + e^{\beta_1 X}$	❌ No	β₁ is nested inside an exponential function
$Y = \frac{\beta_0}{1 + \beta_1 X}$	❌ No	β₀ and β₁ do not enter the model additively

The key question is: do the parameters themselves enter the model as "first-power + additive" terms? Whether the parameter is multiplied by X, ln(X), X², or X₁X₂ — none of this affects "linearity in parameters." These choices determine whether the model is "linear in X," but they do not affect "whether OLS can be used."

3.2 Why Does "Linear in Parameters" Determine Whether OLS Can Be Used?

The core of OLS estimation is solving a system of equations — the normal equations — which set the partial derivatives of the sum of squared residuals $\sum (Y_i - \hat{Y}_i)^2$ with respect to each parameter equal to zero.

When the model is linear in parameters, these partial derivative equations form a system of linear equations in the parameters — which has a closed-form solution (namely, $\hat{\beta} = (X'X)^{-1}X'Y$ ). You do not need numerical optimization, iteration, or to worry about converging to a local minimum — you simply plug into the formula and obtain a unique, determinate answer.

When the model is nonlinear in parameters, the partial derivative equations become a system of nonlinear equations in the parameters — with no closed-form solution, requiring iterative numerical methods (such as Gauss-Newton or gradient descent). At this point, you enter the domain of Nonlinear Least Squares (NLS), which requires specialized software and methods.

3.3 Nonlinear in X + Linear in Parameters = OLS Still Works — This Is the True Magic of "Linear Regression"

Returning to the two "curved" models from the introduction:

Model A: $Y = \beta_0 + \beta_1 \ln(X) + \varepsilon$

This is not linear in X — the partial effect of X on Y is $\beta_1 / X$ , which decays as X grows. But it is linear in the parameter β₁ — β₁ is still first-power and additive. So you can use OLS; you just need to create a new variable ln_X = ln(X) before running the regression, and then use reg Y ln_X. Stata does not even require you to compute it manually — it treats this as an "ordinary linear regression."

Model B: $Y = \beta_0 + \beta_1 X + \beta_2 X^2 + \varepsilon$

This is not linear in X — the marginal effect is $\beta_1 + 2\beta_2 X$ , which varies with X. But it is linear in the parameters β₁ and β₂ — both parameters are first-power and additive. You only need to compute X_sq = X^2 first, then run reg Y X X_sq. This is still a standard task that OLS can handle.

The essential rule: Whether a model is linear in parameters is an algebraic property of how the parameters enter the model. As long as each parameter satisfies "additively separable and first-power," OLS can be used. This allows OLS to handle a wide range of nonlinear transformations at the variable level.

4. When Does OLS Really Become Insufficient?

When parameters themselves enter the model in a nonlinear fashion, the closed-form solution framework of OLS breaks down. For example:

Exponential model:

$Y = \beta_0 + e^{\beta_1 X} + \varepsilon$

You cannot transform this into a model linear in β₁ by "first creating a new variable and then plugging it in" — because β₁ is inside the exponential function, and the exponential function cannot be decomposed into the form "β₁ to the first power times something."

Nonlinear Least Squares (NLS) uses numerical methods and iterative search to find the $\hat{\beta}_1$ that minimizes the sum of squared residuals. It has no closed-form solution and may converge to a local minimum rather than the global optimum.

Logistic growth model:

$Y = \frac{\beta_0}{1 + e^{-\beta_1 (X - \beta_2)}} + \varepsilon$

β₀, β₁, and β₂ are nested in a complex way inside the exponential function in the denominator; the model is not linear in any of the parameters.

In this case, NLS or its extensions (such as maximum likelihood estimation (MLE) in Logit models) are required.

But the vast majority of nonlinear problems faced in the social sciences do not actually require NLS — because through logarithms, squared terms, cubic terms, interaction terms, piecewise linear (spline) specifications, and similar approaches, these relationships can be adequately approximated and expressed while preserving "linearity in parameters."

5. In Practice: The Scatter Plot Shows a Nonlinear Relationship — What Should I Do?

This is the most practically useful part of the article. When you face a nonlinear scatter plot, there is a systematic "upgrade path" to follow.

The Upgrade Path: From Simple to Flexible, Preserving OLS Usability

Level 0: $Y = \beta_0 + \beta_1 X$ (straight line)

If the scatter plot is essentially a straight line and the residuals show no curved pattern — stop here.

Level 1: $Y = \beta_0 + \beta_1 \ln(X)$ (logarithmic transformation)

→ Appropriate when: The scatter plot shows Y growing rapidly at low levels of X and leveling off at higher levels (diminishing marginal effects). Returns to education, the effect of income on consumption, the effect of firm size on innovation — these common patterns all lend themselves to logarithmic transformations.

→ Still linear in parameters; OLS is applicable.

Level 2: $Y = \beta_0 + \beta_1 X + \beta_2 X^2$ (quadratic term)

→ Appropriate when: The scatter plot shows curvature — the marginal effect accelerates or decelerates as X increases. The Environmental Kuznets Curve (income vs. pollution, rising then falling), the effect of age on income (an inverted U-shape, rising then falling) — these are classic applications of quadratic terms.

→ Still linear in parameters; OLS is applicable.

Level 3: Adding an interaction term $X_1 \times X_2$

→ Appropriate when: The effect of X₁ on Y varies with the level of X₂. For example, does the return to education differ by gender? Does the effect of the minimum wage on employment differ by the degree of industry competition?

→ Still linear in parameters; OLS is applicable.

Level 4: Piecewise linear (Spline / Piecewise Linear)

→ Appropriate when: The scatter plot shows a structural break at some critical value of X (a sudden change in slope). For example, a change in trend before and after a policy implementation, or a different pattern of effects above a certain threshold.

→ Still linear in parameters (each segment has a slope parameter, and they enter the model linearly); OLS is applicable.

Level 5: Nonparametric or semiparametric methods (e.g., locally weighted regression LOWESS, kernel regression, GAM)

→ Appropriate when: The parametric approaches above cannot adequately capture the complexity of the pattern. At this point, you abandon the "linear in parameters" framework and use nonparametric regression (such as lowess) or generalized additive models (GAM) for fitting. These methods no longer rely on parameters; instead, they let the data "speak for themselves" — fitting separately in each local region of X.

→ Cost: reduced interpretability, limited out-of-sample extrapolation, and fewer available tests.

A Decision Flowchart

The scatter plot shows that the relationship between X and Y does not look like a straight line
        │
        ├→ Is it monotonic (only increasing, or only decreasing)?
        │     └→ Yes → Diminishing marginal effects? → Try ln(X)
        │          → Increasing marginal effects? → Try e^X or keep the original scale
        │
        ├→ Is there obvious curvature (e.g., rising then falling)?
        │     └→ Yes → Try X + X²
        │
        ├→ Does the slope change markedly at some value of X?
        │     └→ Yes → Try piecewise linear
        │
        └→ Can none of the above adequately describe the pattern?
              └→ Consider nonparametric methods

The key point is — do not immediately conclude that you need nonparametric regression just because the scatter plot is not a straight line. Most nonlinear patterns can be addressed through logarithmic transformations, powers, interactions, and piecewise specifications — all of which remain within the OLS framework.

6. Summary: Two Notions of "Linearity," One Reassuring Conclusion

Dimension	Linear in X	Linear in Parameters
What question it asks	"Is the marginal effect of X constant?"	"Can this model's parameters be estimated by OLS?"
Graphical meaning	The scatter plot is a straight line	The scatter plot can be any shape (a log transformation produces a curve)
Mathematical meaning	∂Y/∂X = constant	Each β enters the model as a first-power, additive term
If violated	Remedy with transformations such as ln(X), X²	Use nonlinear least squares (NLS) or maximum likelihood estimation (MLE)

Three principles to take away:

The "linear" in "linear regression" refers to linearity in parameters, not linearity in X. This is standard econometric terminology. Your model can be highly nonlinear in X (with ln(X), X², interaction terms), but as long as it is additive and first-power in β, OLS can be used.
When facing a nonlinear scatter plot, your first reaction should be "transform variables within the OLS framework," not "switch to nonparametric methods." ln(X), squared terms, interaction terms, piecewise linear specifications — most nonlinear patterns can be adequately captured with this toolkit.
"Why not just use nonparametric or machine learning methods directly?" — If you only care about prediction, that is fine. But if you need to interpret coefficients, conduct hypothesis tests, compute confidence intervals, or discuss policy implications, parametric methods within the OLS framework provide a complete set of mature inference tools that nonparametric methods do not yet offer.

One-sentence conclusion:

"The 'linear' in 'linear regression' is not the 'straight line' you see with your eyes on a scatter plot — it is the fact that when you take derivatives for least squares, you find that the parameters can be cleanly solved by an explicit closed-form solution. That closed-form solution is your OLS."

7. Presentation Suggestions for Bilibili / WeChat Official Account

Bilibili video: Consider using a "transformation animation" to show how the same model is judged under the two different notions of "linearity" — the left side of the screen shows a scatter plot (clearly curved) with large text reading "Nonlinear in X"; the right side shows the mathematical expression of the same model $Y = \beta_0 + \beta_1 \ln(X)$ , with β₁ highlighted and flashing, and large text reading "Linear in parameters." Narration: "OLS sees the right side — it does not care whether X has been logged; it only looks at whether β is first-power."
WeChat Official Account: Present the model examples as card-style layouts with "Linear in parameters = ✅ / Nonlinear in parameters = ❌." Present the upgrade path (five levels) as a vertical staircase diagram, with each level labeled "Still linear in parameters."
Recommended titles:
- Main title: "What Does 'Linear' in Linear Regression Actually Mean? — It Is Not the Straight Line in the Graph"
- Alternative title: "Linear in X vs. Linear in Parameters — Whether OLS Can Run Depends on the Latter"
Memorable quote:

"The 'linear' in linear regression is not the straight line your eyes see on a scatter plot — it is whether your parameters can be solved in one shot by OLS."