Why Sample Sizes Often Vary Across Regression Models in Empirical Analysis—Causes, Red Flags, and Reviewer Concerns

I. Opening: Your Table 1 Has 8,000 Observations, but Table 4 Has Only 2,400—Reviewers Will Ask Why

Your paper contains four regression tables. Each table has a different sample size:

• Table 1 (descriptive statistics): N ≈ 8,000
• Table 2 (baseline OLS regression): N = 5,234
• Table 3 (instrumental variables 2SLS): N = 3,102
• Table 4 (mediation analysis): N = 2,456

From Table 1 to Table 4, your sample has shrunk by 70%. And nowhere in the main text have you explained any of these reductions.

The reviewer opens your paper, scans from N = 8,000 in Table 1 to N = 2,456 in Table 4—and writes the following comment:

"The sample sizes differ substantially across tables—from 8,000+ in Table 1 to fewer than 2,500 in Table 4. The authors do not explain the reason for each sample reduction, the characteristics of the dropped observations, or whether such sample selection might affect the external validity of the estimated results. Please provide a complete sample construction flow and the reason for each deletion, and discuss the potential impact of sample selection on the conclusions."

This problem is nearly universal in empirical papers—yet the vast majority of authors do not pay sufficient attention to it in their first drafts. This article helps you understand: which changes in N are "normal," which are "cause for concern," and why reviewers care so much about this number.

Core message: Variation in sample sizes across regression models is the norm in empirical analysis, not the exception—but not all changes in N are "harmless." Reasonable changes in N have three sources: (1) the proliferation of missing values across variables—newly added variables have additional missing observations, causing the sample to shrink; (2) the sequential imposition of sample restrictions—filters such as "manufacturing firms only" or "A-share listed companies only" are applied progressively across different tables; (3) the inherent data requirements of estimation methods—for example, fixed effects models automatically drop units observed only once (singletons), conditional Logit retains only units with time variation in Y, and 2SLS requires non-missing instrumental variables. Reasonable changes in N need to be explicitly tracked and explained in the paper—typically in the form of a "sample construction flowchart." Changes in N that warrant caution come in three forms: (1) the reduction in N is excessively large (> 30%–50%) and unexplained—reviewers cannot determine whether the remaining sample is still a representative subset of the original population; (2) the reduction in N is selective—the dropped observations differ systematically from the retained ones on key characteristics, which may introduce sample selection bias; (3) different models "should" have the same N but in fact do not—for example, OLS and IV with the same variable set should have the same N; if the IV sample is smaller, the IV variable itself has missing values, and the reason and possible systematic patterns need to be explained. Reviewers care about changes in N not because they are doing "arithmetic checks"—but because behind changes in N lurk three substantive threats: loss of external validity (your conclusions have shrunk from a population of 8,000 to a subset of 2,400), sample selection bias (the dropped observations are not random—they may be precisely the group with the strongest or weakest treatment effects), and a lack of research transparency (readers cannot trace your analytical path).

II. Three Reasonable Sources of Changes in N—Not All Shrinkage Is a Problem

2.1 Source 1: The Proliferation of Missing Values Across Variables—The Most Common Reason

From Table 1 to Table 2, you added 8 control variables—3 of which have missing values. In OLS, missingness in any one variable causes that row of observations to be deleted entirely (listwise deletion). N drops from 8,000 to 5,234.

This is the most common and most "innocent" type of change in N. But "most common" does not mean "no explanation needed." You need to clarify:

• Which variables caused the missingness?
• What is the proportion of missing values?
• Is the missingness random or systematic? (e.g., "small firms are less likely to report R&D expenditure—so missing R&D data are concentrated at the lower end of the size distribution")

Quick diagnosis of the source of N shrinkage in Stata:

* After regression, examine the missingness pattern of each variable
reg y x1 x2 x3
* Note the N in the output
* Then add variables step by step and observe changes in N
reg y x1           // Baseline with the largest N
reg y x1 x2        // Does missingness in x2 reduce N?
reg y x1 x2 x3     // Does missingness in x3 further reduce N?

2.2 Source 2: The Sequential Imposition of Sample Restrictions—"I Am Studying Different Populations"

In Table 2, you used the full sample (all industries). In Table 3, you state, "This paper further restricts the sample to manufacturing firms"—N drops from 5,234 to 3,800. In Table 4, you state, "This paper retains only firms that survived continuously for at least 5 years during the sample period"—N drops further to 2,800.

This type of change in N is driven by research design and is not inherently problematic—but each imposed restriction poses a threat to external validity. When you say, "The conclusions of this paper apply to manufacturing firms," you are explicitly limiting the scope of your conclusions—this is honest. When you do not explicitly state the restriction but have in fact imposed it—readers are misled into believing your conclusions apply to "all firms," when in fact they apply only to "manufacturing firms that survived for 5 consecutive years"—this lack of transparency is what reviewers dislike.

2.3 Source 3: The Inherent Data Requirements of Estimation Methods—"It Is Not That I Want to Drop Them; the Method Requires It"

This type of change in N is part of the nature of the methodology—you do not need to apologize for it, but you do need to explain clearly how it occurs:

FE models drop singletons (units with T = 1): Fixed effects models require at least two time periods within a unit to identify within-unit variation. Units observed only once (singletons) are automatically dropped in FE estimation. This can reduce N by 5%–15%, depending on how balanced your panel is.

Conditional Logit (FE Logit) retains only "switchers": For panel FE estimation with a binary dependent variable—only units whose Y changes at least once from 0 to 1 (or 1 to 0) over time enter the conditional likelihood. If 70% of units have Y = 0 in all years—the N for conditional Logit may retain only 30% of the original units. This is not the model's fault—it is a direct reflection of the "information content" of the data. But you need to tell the reviewer this number.

IV regression drops observations with missing instrumental variables: If you use external instrumental variables (e.g., "distance to port," "birth quarter"), the missingness of these variables may not be random—distance data may be coarser in remote areas, and birth quarter may be unavailable in the informal sector. The reduction in N in IV is not merely that "the sample has become smaller"—it may also change the composition of the sample.

Standard wording in a paper:

"In the conditional Logit model in Table 3, the sample is reduced to N = 1,847 (firm-year) corresponding to 412 firms that changed their export status in at least one year. The likelihood function of conditional Logit uses information only from units with time variation in Y—therefore, 256 firms whose export status never changed are automatically excluded. Appendix Table X compares 'switchers' and 'non-switchers' on observable characteristics—the two groups are similar in size and industry distribution but differ in age (non-switchers are younger)."

III. When Should You Be Concerned?—Five Red Flags for Changes in N

3.1 Red Flag 1: The Reduction in N Exceeds 30%–50% with No Explanation

If your N drops from 8,000 to 5,000 (37.5%) and then to 2,400 (another 52%)—and you do not mention it anywhere in the main text—reviewers will immediately become alert.

Reason: If 70% of the observations have been dropped from your original sample and you have not explained who the 5,600 dropped observations are—reviewers cannot determine whether your conclusions are conclusions about 8,000 people or about 2,400 people. If your conclusions hold only in those 2,400—and those 2,400 happen to be a particular type (e.g., large firms, urban residents, high-income earners)—your conclusions may be an artifact of selection bias.

3.2 Red Flag 2: The Reduction in N Is Selective—Dropped Observations Differ Systematically

Missingness is not random—missing R&D data are concentrated in small and informal firms. If your conclusion is "R&D improves firm productivity," and your final estimation sample contains only large firms with R&D data—your conclusion may not apply to small and informal firms. This is not fatal in itself—what is fatal is pretending it does. Stating clearly "who was dropped" and "why they were dropped" is not a weakness—it is a hallmark of transparency.

3.3 Red Flag 3: Different Models "Should" Have the Same N—but in Fact Do Not

Your baseline OLS regression has N = 5,234. Your IV regression (using the same set of X variables + one instrumental variable) should have the same N—because you have only added one variable (the IV) to the existing variable set. But if the IV regression has N = 3,102—you know the IV variable itself has missing values, with 2,132 observations missing (40.7%). The missingness of the IV is not random—if the IV is "industry mean (excluding the firm itself)" and some industries in your data contain only one firm—observations in these single-firm industries are missing in the IV construction, not randomly, but because of industry structure.

3.4 Red Flag 4: Adding Control Variables → N Shrinks Substantially → the Significance of the Core Coefficient Changes

In your baseline model, you have only 3 control variables, and β̂ is significant. You add 5 additional control variables—N drops from 5,234 to 3,800 (because the new control variables have missing values), and β̂ becomes insignificant.

Reviewers will question: Is it the added control variables that absorb the effect of X, or is it the 1,434 dropped observations that contained the strongest part of X's effect? These two explanations have completely different implications for your conclusions—but without tracking changes in N, you cannot distinguish between them.

Standard diagnostic approach:

• Use the baseline model's variable set, but restrict to the subsample where "new control variables are non-missing," and re-estimate the baseline model.
• If β̂ is insignificant in this same sample → sample selection is driving the change (the dropped observations happen to be those with the strongest effects).
• If β̂ remains significant in this same sample → the newly added control variables explain X's effect.

* Diagnosis: sample selection or control variables?
* Step 1: Baseline model, full sample
reg y x controls1  // N = 5,234, β̂ significant
 
* Step 2: Baseline model, but restricted to the sample where "new control variables are non-missing"
reg y x controls1 if !missing(new_control)  // N = 3,800
* → If β̂ remains significant → control variables are the driving factor (good)
* → If β̂ is no longer significant → sample selection is the driving factor (needs caution and explanation)

3.5 Red Flag 5: N Differs Dramatically Across Subgroup Regressions

Your main analysis includes subgroup regressions—"large firm group" and "small firm group." You state in the text that "the effect of X is stronger in the large firm group." But the large firm group has N = 800, and the small firm group has N = 4,200.

Reviewers will question: (a) Is your definition of the "large firm group" reasonable? (b) Why is the N of the small firm group 5 times that of the large firm group? (c) The "significance" of β̂ in the large firm group may be driven by large standard errors (small N) or extreme values—while the "insignificance" of β̂ in the small firm group may simply be due to small standard errors (large N)—you need to show this rather than relying solely on p-values.

IV. Why Do Reviewers Care So Much About N?—Not Just Arithmetic, but Three Substantive Threats

4.1 Threat 1: External Validity—"To Whom Do Your Conclusions Apply?"

N drops from 8,000 to 2,400—does the "scope of applicability" of your conclusions shrink accordingly? If your original data represent manufacturing firms nationwide, but your final analysis sample consists only of "firms that survived at least 5 years during the sample period, have complete R&D data, and exported in at least one year"—your conclusions may not hold at all for newly established firms, firms that do not report R&D, and firms that never export.

This does not mean your research is invalid—rather, you need to honestly delineate the scope of applicability of your conclusions. Changes in N are the clues for delineating this scope—reviewers track N to track the contraction of your conclusions from "everyone" to "some subset."

4.2 Threat 2: Sample Selection Bias—"The Dropped Observations Are Not Random"

If your Y is wages and 30% of the people in your sample have missing wage data—the missing people are likely not random: they may have greater difficulty finding jobs in the labor market, are more likely to be low-income earners, or are more likely to be employed in the informal sector (where income is not reported). In your final estimation sample, the representation of low-income earners is reduced—your OLS estimate of the "returns to education" may be based on an upwardly biased sample (higher-income earners are more fully employed and report their income).

Changes in N are the "entry point" for sample selection bias. What reviewers care about is not that N has become smaller—but who has become smaller, and why they have become smaller.

4.3 Threat 3: Research Transparency—"Can Readers Reproduce Your Analysis?"

The basic standard of empirical research is that readers should be able to reproduce your analysis from your description. If your paper does not mention the change in N and its reason at any step—readers cannot reproduce your "estimation sample"—because they do not know which observations were dropped, at which stage they were dropped, and why.

A sample construction flowchart solves this transparency problem—it allows readers to see your analytical path, with the cumulative deletions, restrictions, and missingness at each step from the raw data to the final estimation sample.

V. How to Present Changes in N—A Template for the Sample Construction Flowchart

5.1 Basic Structure of the Sample Construction Flowchart

Raw dataset (e.g., "China Industrial Enterprise Database 1998–2013")
    N = 150,000 (firm-year observations)
│
├── Exclude: Non-manufacturing firms
│    Reason: The research subject of this paper is manufacturing
│    Excluded: 35,000 observations
│    → Remaining: N = 115,000
│
├── Exclude: Missing key variables
│    Reason: Core variables such as industrial value added and fixed assets are not reported
│    Excluded: 12,000 observations
│    → Remaining: N = 103,000
│
├── Exclude: Micro-enterprises with fewer than 8 employees
│    Reason: The reliability of financial data for micro-enterprises is questionable (convention in the literature)
│    Excluded: 18,000 observations
│    → Remaining: N = 85,000
│
└── Exclude: Firms appearing only once during the sample period
     Reason: Fixed effects models require at least two periods of data
     Excluded: 5,000 observations
    → Final estimation sample: N = 80,000

5.2 Placement in the Paper

The sample construction flowchart typically appears at the end of the "Data" section—before the descriptive statistics. Readers first learn where your raw data come from—then see how you moved from the raw data to the estimation sample—then see the descriptive statistics of your estimation sample (Table 1).

Order matters: present the screening process first, then the characteristics of the final sample. If the order is reversed—readers first see Table 1 with N = 80,000, and then see N = 80,000 in the model footnotes—they will not ask "where did the 70,000 go?" because they have already seen it in the sample construction flow.

5.3 Noting Changes in N in Regression Table Footnotes

Even with a sample construction flowchart in place, it remains standard practice to summarize the key information about N in the footnote of each regression table:

"Note: The sample size for columns (1)–(3) is N = 80,000, including manufacturing firms that appear at least twice during the sample period. Column (4) adds an R&D expenditure variable (with a missing rate of approximately 15%), reducing the sample size to N = 68,000. Column (5) further restricts to exporting firms (N = 32,000)."

VI. Common Misconceptions

6.1 Misconception 1: "N Has Become Smaller—Reviewers Will Think My Model Is Bad—I Should Hide It"

Hiding changes in N will not make reviewers ignore them—they will calculate it themselves and then write an even longer comment: "The authors do not mention the changes in sample size in the main text—and we calculated the differences in N across tables and found that the N for the IV regression is 42% smaller than that for OLS." Rather than having reviewers point it out, it is better to present and explain it proactively in the first draft. When you explain it proactively, it is called "transparency"—when reviewers discover it for you, it is called "you need to revise."

6.2 Misconception 2: "Stata Handles Missing Values Automatically—I Do Not Need to Explain Changes in N in the Paper"

Yes, Stata automatically uses listwise deletion. But reviewers need to know: which variables caused the missingness? What is the proportion of missing values? Is the missingness systematically concentrated in a particular subgroup? Stata handles the computation for you—but it does not handle "why these observations are missing" or "whether the missingness affects your conclusions." The latter is your job as a researcher.

6.3 Misconception 3: "Changes in N Are Normal—Which Empirical Paper Does Not Have Them?"

Changes in N are normal—but unexplained changes in N are not. It is true that in most papers, N varies across tables—but good papers explicitly track the reason and magnitude of each change in the "Data" section. Bad papers pretend that N does not change. Reviewers take no more than 30 seconds to distinguish between the two.

6.4 Misconception 4: "All My Regressions Have the Same N—So I Do Not Have This Problem"

If all regressions have the same N—it may mean that you have run all analyses only on the "complete-case sample" (listwise deletion removes all observations with any missing value on any variable). Your consistency in N may have come at the cost of losing a large amount of information—you have dropped all observations with missing values on any variable. In this case, the reviewer's question will be reversed: "Why do all regressions have the same N? Did you impute missing values or delete them directly? If the latter—how many observations did you delete?"

VII. Summary

Six core takeaways on changes in sample size:

1. Changes in N are the norm—but unexplained changes in N are a red flag. Reasonable changes in N arise from the addition of variables with missing values, the imposition of sample restrictions, or the inherent requirements of estimation methods. These changes need to have names, numbers, and reasons—not silence.
2. Three reasonable sources: the proliferation of missing values across variables, sample restrictions, and the inherent requirements of estimation methods. Each corresponds to a different way of explaining and a different answer to "who was dropped."
3. Five red flags: N shrinks by > 30%–50% without explanation; the reduction is selective (dropped observations differ systematically); different models "should" have the same N but in fact do not; adding control variables reduces N and changes the significance of the coefficient; and N differs dramatically across subgroup regressions.
4. Three substantive threats that concern reviewers: external validity (do the conclusions apply to the reduced sample or the original population?), sample selection bias (the dropped observations are not random), and research transparency (can readers reproduce your analytical path?).
5. The sample construction flowchart is the standard remedy—in the "Data" section, show the cumulative deletions, restrictions, and missingness at each step from the raw data to the estimation sample.
6. Test on the "same sample"—when N shrinks, distinguish whether "sample selection" or "control variables" is the driving factor. The change in the coefficient on the key independent variable before and after the N reduction—if the coefficient is unchanged in the same sample → control variables are the driving factor. If the coefficient is no longer significant in the same sample → the dropped observations are the driving factor (needs caution and discussion).

One-sentence conclusion:

"Changes in sample size are not a technical detail to be hidden—they are the footprints of your analytical path in the data. Each reduction in N tells a story: 'This variable has 15% missing observations—because small firms do not report R&D expenditure.' 'This model retains only firms whose export status changed—because conditional Logit can only identify changes.' 'The N for this IV is 40% smaller than that for OLS—because the instrumental variable is defined only for manufacturing.' Each story is part of your research design—if you tell it, reviewers will know that you care about the relationship between your data and your conclusions. If you do not tell it—reviewers will tell the story for you, and their version is usually harsher. Good empirical work is not about keeping N constant—it is about being transparent and honest amid changes in N."

VIII. Presentation Suggestions for Bilibili/WeChat Official Account

• Bilibili video: It is recommended to use "footprint tracking" as the core visual metaphor. Opening: a detective (the reviewer) stands on an analytical path diagram—the path starts with 8,000 people, passes through "reduction points" one by one, and finally reaches 2,400 people. At each reduction point, there is a question mark—"Who fell behind here?" "Why?" "Are they random?" Narration: "When reviewers read your paper, they are not just looking at the final regression tables. They are tracking your analytical path—from the raw data to the final estimation sample, people fall behind at every step. If the people who fall behind have no names, no reasons, and no discussion—reviewers will fill in those question marks for you, and their answers are usually ones you do not want to hear." Act 1 "Three reasonable types of changes in N": three reduction points are magnified—the first (missing values in variables, annotated "R&D expenditure missingness is more common in small firms"), the second (sample restrictions, annotated "retaining only manufacturing—service firms are excluded at this point"), the third (estimation method requirements, annotated "FE models require at least two periods—firms appearing only once stop here"). Each reduction point has a brief description—"who was excluded + why." Narration: "Three reasonable types of changes—variable missingness, sample restrictions, model requirements. Each requires you to give a name and an explanation—not to hide, not to pretend it did not happen. When reviewers see your explanation—they regard it as part of the research, not as your oversight." Act 2 "Five red flags": five reduction points flashing red lights—(1) N shrinks by 50% with no explanation—red light. Beside it is written "Reviewer: Who are these 4,000 people?" (2) The dropped people differ systematically from the rest on key characteristics—red light. Beside it is written "Reviewer: Your conclusions are based on large firms, but you pretend they apply to all firms." (3) OLS and IV have different N but should be the same—red light. Beside it is written "Reviewer: The IV variable itself has missing values—is the missingness random?" (4) Adding control variables reduces N and the coefficient becomes insignificant—red light. Beside it is written "Reviewer: Are your new control variables doing the work, or are the 1,400 dropped observations doing the work?" (5) In subgroup regressions, large firms have N = 800 and small firms have N = 4,200—red light. Act 3 "Sample construction flowchart": a complete flow line from "raw data: 150,000" to "final estimation sample: 80,000," with five nodes in between—each node shows the number of excluded observations and the reason. Annotation: "This is how you turn changes in N from a 'hidden problem' into a 'displayed research design.'"

• WeChat Official Account: Present the three reasonable sources of changes in N as comparison cards with examples. Present the five red flags as warning cards—each with a simulated reviewer question "What will the reviewer ask?" Present the sample construction flowchart template as a blank template that readers can fill in with their own data. Present the diagnostic method for OLS vs. IV differences in N and the two-step procedure (same-sample test) as a practical operations card. Present the three substantive threats that concern reviewers (external validity, selection bias, transparency) as a reviewer-perspective insight card. Present the four common misconceptions as correction cards.

• Recommended titles:

  • Main title: 《Why Are Sample Sizes Inconsistent Across Tables?—Reasonable Sources of Changes in N, Red Flags, and Reviewer Concerns》
  • Alternative title: 《From Table 1 to Table 4, N Drops from 8,000 to 2,400—What Should You Explain?》
  • New media title: 《Reviewers Are Staring at Your N—Not Doing Arithmetic, but Tracking Your Analytical Path》

• Key quotes:

  "Changes in sample size are not a technical detail to be hidden—they are the footprints of your analytical path in the data. Each reduction in N tells a story—'this variable is largely missing in small firms''this model can only identify units with variation''this IV is defined only for manufacturing.' If you tell these stories—reviewers will know that you care about the relationship between your data and your conclusions. If you do not tell them—reviewers will tell them for you, and their version usually adds 'the authors do not seem to have noticed.'"

  "N dropping from 8,000 to 2,400 is not the problem—an unexplained N dropping from 8,000 to 2,400 is the problem. The reduction itself is honest—hiding the reduction is dishonest. And if the 5,600 dropped people are not random—your conclusions may have quietly shifted from 'all firms' to 'large firms,' without you even noticing."

  "Your OLS and IV should have the same N—if the N for IV is smaller, the IV variable itself has missing values. Who are the people with missing values? Why are they missing? Is the missingness random? Reviewers will ask these three questions—ask yourself these three questions before you submit."

  "The sample construction flowchart is not a formatting exercise for the appendix—it is the 'birth certificate' of your analysis. It tells readers which sample your conclusions were born from, which screening thresholds they passed through, and who was lost at each threshold. Without this chart—your conclusions have no provenance in the eyes of readers."