From Linear Regression to Logistic Models—Core Principles, Transitional Reasoning, and Economically Exploitable Implications

1. Introduction: Y Takes Only 0 or 1—Can You Still Use OLS Directly?

You study "whether an individual starts a business"—Y = 1 indicates entrepreneurship, Y = 0 indicates not. You want to predict an individual's probability of entrepreneurship using age, education, gender, and family wealth.

Your first reaction is—run a linear probability model (LPM):

You run this regression using OLS. Stata produces a clean output—coefficients are significant, signs are reasonable. Then you check the fitted values:

reg entrepreneur age education wealth
predict yhat
sum yhat

You find that—some fitted values are −0.12, and some are 1.35. An individual's "predicted probability of entrepreneurship" is −12% or 135%—which is an illegal output mathematically.

You realize—for a 0/1 dependent variable, linear regression has a fundamental logical mismatch. You need a different model—one whose output naturally lies between 0 and 1.

This is the logical starting point of Logistic regression.

**Core message: The transition from OLS to Logistic is not about "switching commands"—it is a fundamental reconstruction of probability modeling. OLS applied to a 0/1 variable has three structural problems: predicted values may fall outside [0,1], the error term is inherently heteroskedastic, and assuming that probability changes at a constant rate with X is logically untenable (changes in probability must slow down near the boundaries). The Logistic model elegantly solves all three problems through a nonlinear transformation—mapping the probability p to $\ln(p/(1-p))$, i.e., the log-odds. This transformation changes the dependent variable from the bounded [0,1] to the unbounded $(-\infty, +\infty)$, making linear modeling feasible. But the cost is—the interpretation of coefficients changes: the Logistic coefficient is not the marginal effect of X on probability, but the effect of X on the log-odds. Therefore, there are four levels of core economic quantities that can be deeply exploited in Logistic regression: (1) Odds Ratio—$\exp(\hat{\beta})$ tells you, holding other things constant, by what factor the "odds" of Y occurring are multiplied when X increases by one unit; (2) Average Marginal Effect (AME)—the average of the marginal effects of X on probability across all individuals; it is the quantity in Logit closest to the OLS coefficient and the most directly interpretable economically; (3) Predicted Probability—given specific values of X, what is the probability that Y = 1; this is the most practically meaningful quantity for individual decision-making; (4) Odds Ratio vs. Relative Risk—when the event rate is low, the odds ratio ≈ relative risk, but when the event is common, the two differ substantially; researchers need to judge which one is more appropriate to report for economic intuition."

2. Three Dilemmas of the Linear Probability Model (LPM)—Why Is OLS Insufficient for 0/1 Variables?

2.1 Dilemma One: Predicted Probabilities Exceed [0, 1]

This is the most intuitive problem. The LPM's predicted value is $\hat{p} = X\hat{\beta}$. Since Xβ can take any value in $(-\infty, +\infty)$—while probabilities are defined on [0,1]—the LPM will inevitably produce illegal predictions at certain values of X.

When X takes extreme values—such as a very wealthy, older individual—the LPM may yield an "entrepreneurship probability" above 1, which is conceptually absurd.

2.2 Dilemma Two: Inherent Heteroskedasticity—The LPM's Error Variance Is a Function of p

For a Bernoulli variable $Y \sim \text{Bernoulli}(p)$, $\text{Var}(Y) = p(1-p)$. This variance is a function of p—it is largest when p = 0.5 (0.25) and approaches 0 as p approaches 0 or 1.

In the LPM, different observations have different $p_i = X_i\beta$—and therefore naturally different error variances. Heteroskedasticity is not a "defect" of the data—it is a constructive consequence of a 0/1 dependent variable and is inevitable. You can use robust standard errors to fix inference—but the existence of heteroskedasticity is itself a structural violation of OLS's spherical error assumption.

2.3 Dilemma Three: Constant Marginal Effects Are Logically Untenable

The LPM assumes that a one-unit increase in X changes the probability of Y by a constant $\hat{\beta}$ units—regardless of the current baseline probability.

But think about it—if your entrepreneurship probability is already 95% (you have strong resources and entrepreneurial experience), how much can an additional ¥10,000 in wealth push this probability up? It cannot rise by 5 percentage points—because the ceiling for probability is 100%. Similarly, if your entrepreneurship probability is 2% (you are very young and lack resources), a ¥10,000 decrease in wealth can barely push the probability down—because the floor is 0%.

Changes in probability must slow down near the boundaries—this is determined by the logic of probability itself, not by data characteristics. The LPM assumes constant marginal effects—which cannot hold near the ceiling and floor of probability.

3. From Linear Regression to Logistic—Step by Step

3.1 Step One: Objective—We Need a "Model of Probability"

What we want to model is $\Pr(Y = 1 \mid X) = p(X)$—the probability that Y = 1 given X. This p(X) must be a function whose values lie in (0, 1).

Any function taking values in (0, 1) can be a candidate—and the Logistic function is just one particularly natural choice.

3.2 Step Two: An Ingenious Transformation—Stretching p onto the Entire Real Line

The core problem: p ∈ (0, 1) is bounded, while Xβ is unbounded. We need a bridge—mapping (0, 1) to $(-\infty, +\infty)$.

This bridge is the odds and the log-odds (Logit):

• When p = 0.5 → Odds = 1 ("1 to 1"—the chance of occurring equals the chance of not occurring).
• When p → 0 → Odds → 0.
• When p → 1 → Odds → +∞.

The odds take values in $(0, +\infty)$—now it is semi-unbounded. Taking a natural logarithm:

• When p = 0.5 → Logit = 0 (centrally symmetric).
• When p → 0 → Logit → −∞.
• When p → 1 → Logit → +∞.

The Logit function perfectly maps (0, 1) to $(-\infty, +\infty)$. This is exactly the form we need—now we can safely set Xβ equal to this unbounded quantity.

3.3 Step Three: Set Logit = Xβ—This Is the Logistic Regression Model

This is the core equation of Logistic regression. The left-hand side is Logit(p)—an unbounded quantity. The right-hand side is Xβ—also unbounded. Now the linear model no longer imposes any illogical constraints.

3.4 Step Four: Solve for p—Obtaining the Logistic Function

Solving the Logit equation for p:

This is the Logistic function—an S-shaped curve:

• When Xβ → −∞ → p → 0 ("near-zero response").
• When Xβ → +∞ → p → 1 ("near-certain response").
• Near p = 0.5 (Xβ = 0), the curve is steepest—the marginal effect of X on probability is largest.
• On both sides, the curve becomes flatter—the marginal effect of X on probability approaches zero.

This S-shaped curve emerges from the natural logic of the data—it is not "chosen" (although the analogous probit model uses the normal distribution rather than the logistic distribution, producing a similar S-shaped curve; in practice, logit and probit almost always yield identical marginal effects—the difference lies in tail behavior and the distributional assumption on the latent variable. The logistic distribution has heavier tails than the normal distribution, but in the vast majority of empirical applications, the choice between logit and probit has virtually no impact on conclusions. Logit is more commonly used mainly because the economic interpretation of the odds ratio is more intuitive than probit coefficients).

3.5 An Alternative Introduction—The Latent Variable Perspective

Logistic regression can also be understood through a latent variable framework. Suppose there exists an unobservable continuous latent variable $Y_i^*$—the "entrepreneurial propensity":

What you observe is only $Y_i = 1$ when $Y_i^* > 0$, and $Y_i = 0$ otherwise. That is, when the entrepreneurial propensity exceeds a threshold (normalized to 0), the individual chooses to start a business.

Under this framework:

Economic significance of the latent variable perspective: The Logistic model can be understood as—individuals have a continuous, unobservable "propensity" ($Y^*$), which is linearly determined by X. When the propensity exceeds a certain threshold, the individual makes the "yes" choice (Y = 1). This aligns perfectly with the random utility model in economics—$\varepsilon$ is the individual-specific, unobservable component of preferences or ability, while Xβ represents systematic utility differences.

4. Economically Exploitable Implications in Logistic Regression—Four Levels

Now we arrive at the most important part of this article. After running a Logistic regression, Stata produces a coefficient table—but the direct interpretation of these coefficients is not economically intuitive, nor is it the core content you should present to readers. Here are four levels of economic interpretation, from shallow to deep:

4.1 Level One: Odds Ratio—"Multiplied by What Factor"

Definition:

Interpretation: Holding other conditions constant, for each one-unit increase in X, the odds of Y occurring are multiplied by $\exp(\hat{\beta}_1)$.

Example: You study "the effect of family wealth on the probability of entrepreneurship." The Logit regression yields $\hat{\beta}_{\text{wealth}} = 0.38$.

Interpretation: For each one-unit increase in family wealth (e.g., ¥100,000), the odds of entrepreneurship become 1.46 times the original—or equivalently, the odds increase by 46%.

Why does the odds ratio have economic intuition? In everyday life, people understand "the likelihood doubled" or "the likelihood increased by 50%"—the odds ratio directly corresponds to this language. It is the most economically intuitive coefficient transformation in Logistic regression.

But note: The odds ratio is not the risk ratio. The relationship between probability and odds is $p = \frac{\text{odds}}{1 + \text{odds}}$. When the baseline probability p is small (p < 0.1), the odds ratio approximates the risk ratio (RR ≈ OR). But when the baseline probability is large (e.g., p = 0.5, i.e., a 50% entrepreneurship rate), the odds ratio and risk ratio differ substantially—OR = 1.46 does not mean the probability is 1.46 times the original (that would be a 73% probability, which typically overestimates the true change in probability). Therefore, the OR is suitable for qualitative statements but not for quantitative predictions.

logit entrepreneur wealth age education
* Report odds ratios rather than coefficients
logit, or
* The OR = exp(β) in the output

4.2 Level Two: Average Marginal Effect (AME)—"By How Many Percentage Points Does the Probability Change"

This is the most directly interpretable quantity in Logistic regression.

Definition:

That is—for each individual i, compute the change in probability from a small change in X at i's specific values of X (the marginal effect)—then average these marginal effects across all individuals.

Why is AME needed? Because in the Logistic model, the marginal effect varies with X—there is no single "effect of X." The AME provides "the average effect of X on probability across all individuals"—which is closest to the economic interpretation of an OLS coefficient and is most easily understood by economics readers.

logit entrepreneur wealth age education
margins, dydx(*) post
* AME output: AME of wealth = 0.042
* Interpretation: For each one-unit increase in wealth, the probability of entrepreneurship increases by 4.2 percentage points on average

AME and the odds ratio are complementary: The odds ratio answers "by what factor are the odds multiplied"—this is a relative effect. The AME answers "by how many percentage points does the probability change"—this is an absolute effect. In economic policy analysis, the AME is usually more practically meaningful than the odds ratio—"this policy increases the entrepreneurship rate by 4.2 percentage points" is more direct and actionable than "the odds of entrepreneurship become 1.46 times the original."

4.3 Level Three: Predicted Probability—"What Is the Entrepreneurship Probability of a Specific Individual?"

The AME gives an average—but sometimes you need to know the predicted probability for a specific type of individual.

* Predict the entrepreneurship probability of a "typical" individual
margins, at(wealth=50 age=35 education=16)
* Output: Pr(entrepreneur) = 0.23
* Interpretation: For an individual with wealth of 500,000, aged 35, with 16 years of education—the predicted probability of entrepreneurship is 23%

Economic meaning of predicted probability: In credit approval, insurance pricing, and policy evaluation, you need specific probability values—not odds ratios or marginal effects. Predicted probability directly provides decision-makers with a "point estimate"—"this person's probability of entrepreneurship is 23%." This is the most central output of the Logistic model in industry applications (credit scoring, customer churn prediction, medical diagnosis).

4.4 Level Four: Marginal Effects at Specific Values (MEM/MER)—"At a Certain Key Point, How Large Is the Change in X?"

Sometimes you are not concerned with the average marginal effect across all individuals (AME)—but rather the marginal effect for a specific type of individual.

• MEM (Marginal Effect at the Mean): The marginal effect evaluated at the sample means of all X—i.e., the marginal effect for a "typical individual."
• MER (Marginal Effect at a Representative value): The marginal effect evaluated at a value of X that you care about—such as a policy threshold or clinical cutoff.

* MEM: At the means of X
margins, dydx(*) atmeans
 
* MER: At wealth = 100,000 (policy focus on low-income individuals)
margins, dydx(wealth) at(wealth=10)

Applications of MEM/MER: Policy evaluation typically focuses on specific populations—such as "is the entrepreneurship probability of the lowest income group particularly sensitive to changes in wealth?" In the low-income group, the marginal effect of wealth may be much larger than in the high-income group (because the entrepreneurship probability of the high-income group is already near the ceiling, and additional wealth has little effect on pushing the probability further). The AME cannot capture this heterogeneity—MEM and MER can.

4.5 Correspondence Table for the Four Levels

5. Common Pitfalls and Practical Considerations in Logistic Regression

5.1 Coefficients Cannot Be Compared Across Models

In the Logit model, the magnitude of $\hat{\beta}$ depends not only on the true relationship between X and Y—but also on the variance of unobserved heterogeneity ($\text{Var}(\varepsilon)$ is normalized to $\pi^2/3$). If you add new variables (improving model fit), the magnitude of $\hat{\beta}$ may change—even if the true effect of X on Y has not changed at all.

Intuition: Logit coefficients are estimated under a fixed "error variance." If you add more explanatory variables, the model's explanatory power increases, and the error variance (on the latent variable scale) is compressed—coefficients naturally become larger. This change does not reflect a true change in the effect of X—it only reflects a change in model explanatory power.

Therefore:

• ❌ You cannot compare Logit coefficients from the same model across different samples.
• ❌ You cannot compare coefficients of the same variable across different models.
• ✅ You should compare AMEs—the AME is on the probability scale and is not affected by this normalization issue.
• ✅ If you must compare across models, use OLS (LPM) coefficients—OLS does not have this normalization problem.

5.2 The "Divide by 4" Rule—Quickly Converting Logit Coefficients to an Upper Bound on Probability Changes

A useful rule of thumb: In Logit regression, the marginal effect of X on probability reaches its maximum at p = 0.5, and this maximum is approximately $\hat{\beta}/4$.

• If $\hat{\beta} = 0.4$ → at any value of X, the maximum change in probability is at most approximately 0.4 / 4 = 0.10 (10 percentage points).
• This provides a quick way to assess the "magnitude" of a Logit coefficient—$\hat{\beta} = 0.08$ → the probability changes by at most 2 pp, and the effect may be economically unimportant.

5.3 Odds Ratio ≠ Risk Ratio

When the baseline probability is not low, this distinction is critical.

• If the baseline entrepreneurship probability is p = 0.03 → OR = 1.46 → new probability ≈ 0.044 (using the RR ≈ OR approximation gives 0.03 × 1.46 = 0.044, which is quite accurate when p is low).
• If the baseline entrepreneurship probability is p = 0.50 → OR = 1.46 → new probability ≠ 0.73 (the OR cannot be directly multiplied by the probability). The true new probability is $\frac{0.5 \times 1.46}{1 + 0.5 \times (1.46 - 1)} = 0.594$, not 0.73.
• When p > 0.1, you should report marginal effects (AME) in your paper rather than relying solely on the OR to tell the story.

5.4 Complete Separation and Quasi-Complete Separation

In some data, X can perfectly predict Y (e.g., everyone with X above a certain threshold is an entrepreneur)—in this case, the Logit $\hat{\beta}$ tends toward ±∞, and Stata will report an error of "complete separation."

Solutions:

• Check whether there are perfectly predicting variables (such as deterministic grouping in tabular form).
• If they exist → use penalized likelihood (Firth Logit) or remove the variable.
• Or use the linear probability model LPM (the LPM is not affected by this problem).

5.5 Logit vs. Probit—Which Should You Choose in Practice?

The difference between the two lies in the distributional assumption on $\varepsilon$: Logit assumes the logistic distribution (heavy tails), while Probit assumes the normal distribution (thin tails). In actual data:

• The AMEs of the two are almost identical—the difference is far smaller than sampling error.
• Advantage of Logit: the odds ratio (OR) has a direct economic interpretation. Probit coefficients have no similar intuitive interpretation.
• Empirical practice: Use Logit and present the AME as the core economic quantity—the AME almost always coincides numerically with the AME from Probit.

6. Common Misconceptions

6.1 Misconception One: "The Logit Coefficient = The Effect of X on Probability"

This is the most widespread misunderstanding. The Logit coefficient $\hat{\beta}$ is the effect of X on the log-odds—not on probability. You need to compute the AME to obtain an interpretation on the probability scale. If a paper's empirical section reports only a Logit coefficient table without providing the AME—reviewers will almost certainly require it to be added. In economics, the Logit coefficient itself is not regarded as the final quantity to report—the AME is.

6.2 Misconception Two: "OR > 1 → X Increases → The Probability of Y Must Increase"

OR > 1 does mean that an increase in X → the odds increase → the direction of the probability change is consistent (positive). But the magnitude of the OR does not equal the magnitude of the probability change—and the effect of the OR on probability varies across individuals (heterogeneity). "OR > 1 → probability increases" is correct in direction, but the magnitude of the probability change cannot be read directly from the OR—you need the AME.

6.3 Misconception Three: "The Logit Model Does Not Need to Check for Heteroskedasticity—It Has Already Handled It"

Logit automatically handles the constructive heteroskedasticity in the LPM (because the Logit likelihood function directly models the Bernoulli distribution). But this means you do not have "heteroskedasticity that needs to be tested and corrected"—not that Logit estimates are robust under all misspecifications. If the error variance of the latent variable itself varies with X (heteroskedasticity at the latent variable level), Logit estimates remain inconsistent. This is the problem addressed by heteroskedastic choice models.

6.4 Misconception Four: "The AME Is in Stata's margins Output → I Report It → Done"

When computing the AME, there are two key options to note:

• The post option: Stores the marginal effects as estimation results for subsequent operations.
• The vce(unconditional) option: If you are using survey data or clustered data, ensure that standard errors are clustered at the appropriate level.

logit y x1 x2, cluster(id)
margins, dydx(*) post
* The correct standard errors for the AME are automatically carried forward after margins

7. Summary

Core knowledge of Logistic regression:

1. Three inadequacies of OLS for 0/1 variables: predicted probabilities may fall outside the bounds, inherent heteroskedasticity, and constant marginal effects being logically untenable for probabilities. These problems are not "data problems"—they are structural gaps between linear models and probability modeling.
2. The ingenuity of the Logit transformation: $\ln(p/(1-p))$ maps [0,1] to $(-\infty, +\infty)$—allowing linear models to operate safely in an unbounded space. The Logistic function obtained by solving backward naturally produces an S-shaped curve—slowing down at probability boundaries and being most sensitive at the center.
3. Four levels of economic interpretation—from coarse to fine: The odds ratio (OR) is suitable for qualitative statements about "by what factor the likelihood is multiplied." The AME is the core economic quantity—"by how many percentage points the probability changes on average." Predicted probability suits specific individual decision contexts. MEM/MER suits policy evaluation targeting specific populations.
4. The AME is the core output of Logit in economics—not the coefficient table. Reviewers and readers expect you to present the AME—not $\hat{\beta}$, not the OR—but the average marginal effect of X on probability.

One-sentence conclusion:

"Logistic regression shifts you from 'linear thinking' to 'odds thinking.' The linear model says: 'When X increases by one unit, Y changes by a constant β units.' The Logistic model says: 'When X increases by one unit, the odds of Y occurring are multiplied by e^β—and the change in probability depends on how high your current probability is.' This shift from 'constant increments' to 'multiplicative changes'—is not a mathematical trick—it is the logical necessity of probability modeling. Because probability has boundaries, and boundaries imply deceleration. A probability change from 5% to 10%—and a change from 90% to 95%—are symmetric on the odds scale, but inherently different on the probability scale. Logit accepts this asymmetry from the outset—and redirects your economic intuition from probability changes to odds changes. Then the AME brings you back to probability—allowing you to tell your economic story on a more honest scale."

8. Presentation Suggestions for Bilibili/WeChat Official Account

• Bilibili video: Use "from a straight line to an S-curve" as the core visual transition. Opening: a straight line passes through a scatter plot (0/1 variables)—the line extends beyond the top 1 and bottom 0 (red warning). Voiceover: "Linear regression says—probability is a linear function of X. But probability can only be between 0 and 1—predicting that someone has a 135% probability of starting a business is meaningless." The straight line morphs into an S-shaped curve—steepest at p=0.5, flattening at both ends. Annotation: "The Logistic function—a natural S-shape. It decelerates at the boundaries and is most sensitive at the center. This is not mathematical decoration—it is the logic of probability itself." Act One "Odds—stretching to infinity": the p-axis (0→1) is stretched into the odds axis (0→+∞), then taking the logarithm (−∞→+∞). Three numerical examples run in parallel—p=0.1 → odds=0.111 → logit=−2.20. p=0.5 → odds=1 → logit=0. p=0.9 → odds=9 → logit=2.20. Animation shows the stretching—"from bounded to unbounded, the linear model is safe." Act Two "Four interpretations": a four-quadrant screen. Top-left—Odds Ratio (exp(β)=1.46, "odds increased by 46%"). Top-right—AME (+4.2 pp, "entrepreneurship rate increased by 4.2 percentage points"). Bottom-left—Predicted probability ("this person's entrepreneurship probability is 23%"). Bottom-right—MEM ("in the low-income group, the marginal effect of wealth is 6.1 pp"). The four quadrants highlight sequentially, with different economic stories activated in each quadrant. Act Three "The divide-by-4 rule": a Logit coefficient β=0.4 is fed into the "divide-by-4 machine"→ output 0.10 ("probability changes by at most 10 percentage points"). Voiceover: "This is the quick check you do yourself before reporting Logit results—β/4 is the upper bound of the marginal effect. If it is too small—your X has almost no economic importance on the probability scale."

• WeChat Official Account: Present the three dilemmas of the LPM as three warning cards. Present the mapping process from p → odds → logit as a three-layer stretching diagram—this is the core visual of the entire article. Present the four economic quantities (OR, AME, predicted probability, MEM) as a four-quadrant infographic—each quadrant with definition, mathematical expression, economic intuition, and Stata command. Present the "divide-by-4" rule as a quick-tip card. Present the four common misconceptions as correction cards. Place the comparison table with the LPM (prediction range, marginal effects, heteroskedasticity handling, coefficient interpretation) at the end of the article.

• Recommended titles:

  • Main title: 《From OLS to Logistic—When Your Y Takes Only 0 or 1, What Happens to the Model?》
  • Alternative title: 《Logistic Regression Coefficients, Odds Ratios, and AME—Which One Should You Report?》
  • New media title: 《Your Y Is 0/1, but You Are Still Using OLS?—The Logic, Transformation, and Economic Interpretation of the Logistic Model》

• Key quotes:

  "Linear regression tells you—when X changes by one unit, Y changes by a constant β units. Logistic tells you—when X changes by one unit, the odds of Y are multiplied by e^β. The change in probability depends on your baseline probability: going from 5% to 10% is easy, going from 90% to 95% is also easy—but going from 50% to 51% versus from 50% to 55%, in the world of Logit, the former requires far less change in X than the latter."

  "The AME is Logistic's translation for economists—it translates coefficients on the log-odds scale back into marginal effects on the probability scale. What you should report in your paper is the AME—not β̂, not the OR. The former speaks of 'differences in log-odds,' the latter speaks of 'percentage-point changes in probability.' Economics audiences want the latter."

  "The divide-by-4 rule—β̂/4 is the upper bound of the marginal effect. A Logit coefficient of β̂ = 0.08—under no circumstances can the maximum effect of X on probability exceed 2 percentage points. Before you get excited about writing your conclusions, do this check—is your X economically relevant? Or is it only statistically significant?"

  "Logistic takes you from the world of straight lines to the world of S-curves. In the world of straight lines, every step is the same size. In the world of S-curves, the same step becomes smaller and smaller near the boundaries—because probability has walls, and walls make the effect of each step diminish. This is the logic of probability itself—not a data characteristic, not a model choice—it is what the concept of 'possibility' itself tells you."