Must Dummy Variables Be One Fewer Than the Number of Categories? When Are Exceptions Allowed? Is This the Same as One-Hot Encoding in Machine Learning?

1. Opening: Stata Automatically Drops One Dummy Variable for You—Do You Know Which One It Drops and Why?

You run a regression in Stata that includes "region" (East/Central/West, three categories):

reg y x i.region

The Stata output shows only two regional dummy variables—for instance, _Iregion_2 (Central) and _Iregion_3 (West). East disappears.

You flip through your econometrics textbook—"For a qualitative variable with k categories, you only need to include k−1 dummy variables; otherwise, you fall into the dummy variable trap." You know Stata dropped one for you, but have you ever wondered:

• Why k−1? Why not k−2? Would dropping two also work?
• Which one does Stata drop by default? Is it the one with the smallest alphabetical order as the base, or can you specify?
• Are there situations where you can include all k dummy variables? If so, how should the coefficients be interpreted?
• Is this the same as One-Hot Encoding in machine learning? Why must OLS drop one, while in LASSO and neural networks, it seems all categories have corresponding parameters?

Key message: The reason dummy variables must be one fewer than the number of categories is that when the model includes an intercept term, including all k dummy variables creates a perfect linear dependency with the intercept (the sum of k dummy variables is identically equal to 1, which exactly equals the constant 1 multiplied by the intercept). This is the "dummy variable trap"—OLS cannot be solved mathematically because the X'X matrix is not invertible. There are two ways to resolve this: drop one dummy variable (keep the intercept), or drop the intercept (keep all dummy variables). One-Hot Encoding in machine learning is mathematically the exact same operation as dummy variables—but it is handled differently across algorithms: OLS must address the collinearity issue (drop one or remove the intercept), while regularized models (LASSO, Ridge) and tree-based models have different sensitivities to collinearity and can sometimes retain all category encodings.

2. The Mathematics of the Dummy Variable Trap—Why k−1?

2.1 The Mechanism Generating Perfect Collinearity

Suppose you have a qualitative variable "region" with 3 categories: East, Central, and West.

Approach A (include all):

$Y = \beta_0 + \beta_1 \cdot \text{East} + \beta_2 \cdot \text{Central} + \beta_3 \cdot \text{West} + \varepsilon$

where East = 1 if in the East and 0 otherwise, and Central and West are defined analogously. But note—for every observation:

$\text{East} + \text{Central} + \text{West} = 1 \quad \text{(holds identically for all individuals)}$

And the intercept term β₀ is multiplied by an "implicit variable" that is also always 1. Therefore:

$\beta_0 \cdot \mathbf{1} = \beta_0 \cdot (\text{East} + \text{Central} + \text{West})$

This means the intercept column equals the sum of the three dummy variable columns. The columns of the X matrix exhibit perfect linear dependency—X'X is not invertible, and OLS cannot be solved. Stata will detect this dependency and automatically drop one of the dummy variables (or report an error).

Approach B (drop one—the standard practice):

Drop East, and the model becomes:

$Y = \beta_0 + \beta_2 \cdot \text{Central} + \beta_3 \cdot \text{West} + \varepsilon$

Now:

• East (base group): Central = 0, West = 0 → E[Y | East] = β₀
• Central: Central = 1, West = 0 → E[Y | Central] = β₀ + β₂
• West: Central = 0, West = 1 → E[Y | West] = β₀ + β₃

Interpretation of β₂: The mean difference in Y between Central and East (the base group). Interpretation of β₃: The mean difference in Y between West and East (the base group).

Approach C (drop the intercept—another valid approach):

$Y = \beta_1 \cdot \text{East} + \beta_2 \cdot \text{Central} + \beta_3 \cdot \text{West} + \varepsilon$

Now each β directly equals the conditional mean of Y for that group:

$\beta_1 = \mathbb{E}[Y \mid \text{East}], \quad \beta_2 = \mathbb{E}[Y \mid \text{Central}], \quad \beta_3 = \mathbb{E}[Y \mid \text{West}]$

Approaches B and C are mathematically equivalent (producing identical fitted values), but the interpretation of the coefficients differs. In Approach B, β₂ and β₃ represent "differences relative to the base group"; in Approach C, β₁, β₂, and β₃ represent "absolute means of each group."

2.2 Why k−1, Not Fewer?

Dropping one dummy variable is sufficient to break the perfect collinearity. Dropping more than one does not make the model more "correct"—it only causes you to lose information. If you have 3 categories but only include 1 dummy variable (for example, only distinguishing "East or not"), you are effectively merging "Central" and "West" into a single group—this might be reasonable if there is truly no difference between them, but you would not achieve this by "dropping variables"; rather, you would do it through recoding (merging Central and West into a single "non-East" category).

Conclusion: k−1 is the number that is just sufficient to break perfect collinearity while preserving all category information. Drop one fewer → collinearity is broken. Drop two fewer → information is lost, not because "it must be done this way," but because you have actively chosen to merge categories.

3. When Are Exceptions Allowed?—Four Scenarios Where You Can Include k Dummy Variables

3.1 Exception 1: Dropping the Intercept Term (Regression Through the Origin)

As discussed above, once the intercept is removed from the model, all k dummy variables can be included:

$Y = \beta_1 D_1 + \beta_2 D_2 + \cdots + \beta_k D_k + \varepsilon$

In Stata:

reg y i.region, noconstant   // Drop the intercept; all k dummy variables are retained

When to use this?

• When theory suggests Y should be zero when X = 0 (e.g., "no input, no output").
• When you want to directly obtain the mean of each category rather than differences relative to a base group.
• In certain fixed effects models, when you have already absorbed all unit fixed effects using i.id, dropping the intercept and including all unit dummy variables is computationally more convenient (although xtreg, fe and reghdfe handle this automatically).

Caveat: After dropping the intercept, the way R² is computed differs from the usual case (it is no longer "the proportion of variation around the mean explained by the model"), and the R² reported by Stata may appear unusually high—this is normal in noconstant regressions and does not indicate a better model.

3.2 Exception 2: Multiple Qualitative Variables—Each Variable Follows Its Own k−1 Rule

When you include multiple qualitative variables simultaneously, each qualitative variable drops one dummy variable individually, rather than reducing the total number of dummy variables by one.

For example: a model includes "region" (3 categories) and "industry" (5 categories).

• Region: include 2 dummy variables (drop East)
• Industry: include 4 dummy variables (drop Agriculture)
• Total: 2 + 4 = 6 dummy variables, plus the intercept → no problem

Why doesn't 2 + 4 = 6 cause collinearity? Because the regional dummy variables sum to 1, and the industry dummy variables sum to 1, but the sum of the regional dummies does not equal the sum of the industry dummies—there is no perfect linear dependency between the two sets of dummy variables. Collinearity only exists within the set of dummy variables generated by a single qualitative variable, not across different sets.

3.3 Exception 3: Regularized Models—LASSO and Ridge

In generalized linear models in machine learning, if you add L1 (LASSO) or L2 (Ridge) regularization, you can include all k dummy variables—even in the presence of perfect collinearity, the regularization term makes the X'X matrix invertible (because a positive definite matrix λI is added to X'X).

But this does not mean you should do so—the cost of including all of them is that the interpretation of the coefficients becomes dependent on the choice of the regularization parameter λ. In LASSO, the coefficient for a particular category may be shrunk to zero—but "which category gets shrunk to zero" depends on the choice of λ and small perturbations in the data, rather than being a base group you actively chose.

In practice: In Python's sklearn.linear_model.Lasso or Ridge, even if you perform One-Hot Encoding retaining all k columns, the model will run—not because the mathematical principles have changed, but because regularization adds a diagonal matrix to X'X, breaking the singularity. However, this logic differs from dropping the intercept in OLS—it relies on an external penalty parameter.

3.4 Exception 4: Tree-Based Models and Distance-Based Models—Collinearity Has Little Effect

In decision trees, random forests, and gradient boosting machines (GBDT, XGBoost, LightGBM), perfect collinearity does not cause mathematical unsolvability—because these models use one variable at a time for splitting, rather than simultaneously solving a system of linear equations for all coefficients. If you include all k dummy variables, the tree model may select some of them and ignore others—the extra dummy variables will not cause the algorithm to crash.

Similarly, in K-Nearest Neighbors or Support Vector Machines (SVM), including all k dummy variables causes information from certain dimensions to be double-counted in distance calculations—but this is usually not fatal, just inelegant.

However, note: If you use One-Hot Encoding in tree-based models, a large number of categories can lead to a dimensional explosion (e.g., "city" with 300 categories → 299 dummy variables). This inflates the number of splits the tree model needs to search at each node, reducing efficiency. In such cases, Label Encoding or Target Encoding may be better—but that is another topic within machine learning.

4. Dummy Variables and One-Hot Encoding—Are They the Same?

4.1 Mathematically Identical—But the Choice of "Dropping a Column" Differs

Dummy variable coding and One-Hot Encoding are mathematically identical: both transform a qualitative variable with k categories into k binary 0/1 columns.

However, in practice:

Core conclusion: Dummy variables and One-Hot Encoding are the same encoding method, differing only in the default parameters of the two fields. In econometrics, the default is drop='first', because the model has an intercept by default. In machine learning, the default is drop=None, because the model does not necessarily have an explicit intercept, and preprocessing pipelines typically separate encoding from modeling—the encoder does not know whether the downstream model includes an intercept.

4.2 Dummy Encoding vs. One-Hot Encoding—A Translation Artifact

In Chinese, "哑变量编码" (dummy variable encoding) and "独热编码" (one-hot encoding) are often treated by many as two different methods—this is an artificial division created by translation. In reality:

• Dummy Variable Encoding typically refers to the k−1 column version (the default in statistics/econometrics).
• One-Hot Encoding typically refers to the k column version (the default in machine learning).

However, in English statistical and econometric literature, dummy variable, indicator variable, and binary variable are often used interchangeably, and k−1 is not the default assumption—it is the specific scenario of "a linear model with an intercept" that requires k−1. The encoding itself does not care whether it is k or k−1—it is simply the operation of "converting categories into 0/1 columns." The choice of k or k−1 depends on how you model the intercept.

4.3 A Comparative Example—Stata vs. Python

Stata (standard practice in econometrics):

reg y x i.region              // Automatically drops one → 2 dummy variables
reg y x ibn.region, noconstant // Includes all → 3 dummy variables

Python (standard practice in machine learning):

# pandas
pd.get_dummies(df['region'], drop_first=False)  # Default: 3 columns
pd.get_dummies(df['region'], drop_first=True)   # Equivalent to Stata default: 2 columns
 
# sklearn
from sklearn.preprocessing import OneHotEncoder
OneHotEncoder(drop=None)      # Default: 3 columns
OneHotEncoder(drop='first')   # Equivalent to Stata default: 2 columns

R (can go either way):

lm(y ~ x + region, data)         # Automatically drops one → k−1
lm(y ~ x + region - 1, data)     # Includes all, drops the intercept → k

5. Choosing the Base Group—The Dropped Category Is Not "Unimportant" but Your "Comparison Anchor"

5.1 How to Choose the Base Group—Three Principles

Because the coefficients on the k−1 dummy variables are all interpreted relative to the dropped base group, your choice of which category serves as the base group directly determines the meaning of all dummy variable coefficients.

Principle 1: Choose the most theoretically natural "control state" or "default state."

• Studying "the impact of a policy on GDP" → base group = "regions where the policy was not implemented."
• Studying "the effect of education on wages" → base group = "primary school or below" or "junior high school" (lowest education level).
• Studying "the effect of treatment on health" → base group = "the control group that did not receive treatment."

Principle 2: Choose a group with a sufficiently large sample size as the base.

If a category has only 10 observations and you use it as the base group—all other groups' coefficients are compared against the mean of these 10 observations—the standard errors will be very large (the "anchor" itself is unstable).

Principle 3: Clearly report in your paper what the base group is and why you chose it.

Do not just write "regional dummy variables are controlled for." You should write "regional dummy variables use the East region as the base group (this group has the largest sample size and serves as the reference for the policy)."

5.2 Can You Specify the Base Group in Stata?

Yes. Stata uses the category with the smallest coding as the base group by default. You can specify it using ib:

* Use "West" as the base group (assuming region = 3 is West)
reg y x ib3.region
 
* Use the group with the smallest coding as the base (Stata default)
reg y x i.region
 
* Use the group with the largest coding as the base
reg y x ib(last).region
 
* Use the group with the highest frequency as the base
reg y x ib(freq).region

6. Common Misconceptions and Caveats

6.1 Misconception 1: Standardizing Dummy Variables

Dummy variables (0/1) should not be standardized. A standardized dummy variable is no longer 0/1, and its coefficient cannot be interpreted as "the mean difference between groups"—you have effectively turned a clean group identifier into an unintelligible value.

6.2 Misconception 2: Forgetting the Base Group of Dummy Variables in Interaction Terms

If M is a dummy variable (M = 1 for female, M = 0 for male as the base), the coefficient on X × M is interpreted as "how much more the effect of X on Y is for women than for men." If you include all k dummy variables, the coefficient on X × each category dummy is interpreted as "the effect of X within that category"—the interpretation differs, but the information is equivalent.

6.3 Misconception 3: Believing That "Dropping One Dummy Variable" Loses Information

It does not. k−1 dummy variables plus an intercept contain exactly the same information as k dummy variables without an intercept. The two sets of coefficients can be transformed into each other—"differences relative to the base group" and "absolute means of each group" contain exactly the same amount of information, just expressed differently.

7. Summary

Four core takeaways about dummy variables:

1. A qualitative variable with k categories = k−1 dummy variables (in a model with an intercept). It is not that "one fewer is more parsimonious," but rather "one more and the model is dead"—the sum of k dummy variables is identically equal to the intercept column, making X'X non-invertible.

2. Three legitimate exceptions: dropping the intercept (noconstant) → you can include k; multiple qualitative variables → each variable follows its own k−1 rule; regularized models (LASSO/Ridge)/tree-based models → collinearity is tolerated, the model will not crash, but the interpretation differs.

3. Dummy Variable = One-Hot Encoding, only the default parameters differ. Econometrics defaults to drop='first' (because the model has an intercept by default), while ML defaults to drop=None (because the encoder does not know whether the downstream model has an intercept). The encoding itself is the same—the assignment of 0/1 columns is completely equivalent.

4. The choice of the base group is not arbitrary. It determines the meaning of all dummy variable coefficients—"relative to whom." Choose the most natural control state, choose a group with a sufficiently large sample size, and report it clearly in your paper.

One-sentence conclusion:

"The essence of the dummy variable trap is not that 'the coding method is wrong,' but that 'your linear model already has an implicit 1 in the intercept—it does not need another explicit 1 from the dummy variables to come out and fight it.' Econometrics drops one dummy variable; machine learning keeps all columns and adds regularization—the goal of both approaches is identical: to give the model a unique solution. Different means, same end."

8. Presentation Suggestions for Bilibili/WeChat Official Account

• Bilibili video: Consider using the metaphor of "three light switches controlling one lamp." Opening scene: one lamp (Y), three switches on the wall (East, Central, West). The three switches are wired in series—no matter how you flip the switches, as long as you know the position of two of them, the third is determined. Voiceover: "Three switches, seemingly you can control them independently—but in reality, one of them is always the 'shadow' of the other two. This is the dummy variable trap—not a profound mathematical problem, but the simple fact that 'three variables sum to exactly 1.'" Then transition to the Stata output: reg y i.region shows only two coefficients. Animate to show—the third coefficient is not "deleted," but "absorbed into the intercept." Next, show the version without the intercept—all three coefficients appear, but their meaning changes to "group means" rather than "differences relative to the base group." Finally, compare Python's One-Hot Encoding with Stata's dummy variables—the same encoding, different default parameters—"both sides are doing the same thing, except econometrics' default assumes your model has an intercept, while machine learning does not."
• WeChat Official Account: The comparison table of the three approaches (k−1 dummy variables + intercept / k dummy variables + no intercept / k dummy variables + regularization) should be made into a three-column infographic. The comparison table of dummy variables vs. One-Hot Encoding should be placed in a prominent position in Section 4. The three principles for choosing the base group should be made into cards. The code comparisons across Stata/Python/R should be presented as horizontal code blocks.
• Recommended titles:
  • Main title: 《Must Dummy Variables Be One Fewer Than the Number of Categories?—From the Dummy Variable Trap to One-Hot Encoding》
  • Alternative title: 《Why Does Stata Automatically Drop One Dummy Variable?—The Mathematical Principles of the k−1 Rule and Four Exceptions》
  • New media title: 《Dummy Variables vs. One-Hot Encoding: Econometrics and Machine Learning Have Different Defaults but Do the Same Thing》
• Key quotes:

  "The essence of the dummy variable trap: your model already hides a 'column of all ones' in the intercept—adding k dummy variables means writing the same 1 twice. Mathematically non-invertible, logically redundant."

  "Dropping one dummy variable loses no information—it simply 'pins' one group to the intercept, and all other groups are compared against it. The dropped group is not ignored—it becomes the anchor of comparison."

  "Dummy Variable and One-Hot Encoding are the same thing. The only difference: econometrics assumes you have an intercept by default, so it drops one column for you first; machine learning does not assume you have an intercept, so it keeps all columns for you first. Engineers on both sides know what they are doing—you need to know why."