Unbalanced Panels and Missing Values—When to Drop? When to Impute? How to Impute Properly?

I. Introduction: There Are "Vanishing People" in Your Panel Data—Have You Noticed Them?

You are using five years of CGSS tracking survey data. The original sample has 8,000 people. You run a panel fixed effects model, and Stata tells you "Number of obs = 24,000"—but you clearly have 8,000 × 5 = 40,000 possible observations. Where did the remaining 16,000 go?

• Some people were not re-interviewed in the second year—they moved away or could not be contacted.
• Some people reported income in years one and three but left it blank in year two.
• Some people participated every year but refused to answer certain questions in certain years.

Your panel is unbalanced. Each individual is observed a different number of times. The data do not form a neat rectangle but rather a pockmarked "lunar surface."

Stata handles this automatically—xtreg by default uses only observations with complete data (complete-case analysis). But the question is: are these "vanishing people" systematically different from those who remain? If they happen to be the ones with lower income, less stable jobs, and more frequent migration—then your conclusions based on complete data hold only for "those who can be tracked, the more stable population." This is not a bias in your regression coefficients—this is your sample being silently filtered.

Core message: An unbalanced panel per se is not a problem—OLS and fixed effects estimators remain consistent and unbiased in unbalanced panels (under standard assumptions). The real problem is sample attrition—if the "missingness" of observations is not random but correlated with the model's dependent variable or error term, your estimates face sample selection bias. The choice of how to handle missing values depends on three judgments: what the cause of missingness is (MCAR / MAR / MNAR), how large the missing proportion is, and what role the missing variable plays in the model (dependent variable? key explanatory variable? control variable?). There is no one-size-fits-all rule—but there is a spectrum of options from "conservatively safe" to "fully exploiting information."

II. Unbalanced Panel ≠ Missing Data—First Clarify What You Are Facing

2.1 Unbalanced Panel vs. Balanced Panel

A panel dataset where every individual is observed in every period is a balanced panel. If different individuals are observed for different numbers of periods, it is an unbalanced panel.

Common causes of unbalanced panels:

• Natural sample attrition: respondents die, move away, refuse further participation, or cannot be contacted.
• Sample refreshment/entry: survey organizers add new samples to maintain representativeness.
• Eligibility changes: in firm-level studies, some firms go bankrupt (no longer observed) while new firms are established (new observations added).
• Rotating panels: by design, a portion of individuals is replaced each period (e.g., the CPS 4-8-4 design).
• Data merging: different data sources have different coverage along certain dimensions.

2.2 "Unbalanced" Itself Is Not the Problem—"Why Unbalanced" Is

An unbalanced panel does not by itself violate any classical assumption of OLS or fixed effects models. If your imbalance is completely random (whether an observation exists is unrelated to any variable in the model), OLS and FE results from an unbalanced panel are theoretically just as valid as those from a balanced panel.

But the problem is—imbalance is almost never "completely random." In tracking surveys, low-income individuals, migrant populations, and those in poor health are more likely to be lost to follow-up. In firm panels, bankrupt firms are no longer observed, and surviving firms are overrepresented. This non-random "selective disappearance" means your sample is silently filtering out specific types of individuals—and your regression is unaware that this filtering has occurred.

III. The First Gate for Handling Missing Data: Determining the Type of Missingness

Before taking any remedial action, you need to ask: why are these data missing? This is crucial for the strategy you subsequently adopt.

3.1 Three Missingness Mechanisms (Rubin, 1976)

MCAR (Missing Completely At Random)

The probability of missingness is unrelated to any variable (including Y and X). It is purely a matter of chance.

• Example: A few pages of a questionnaire are randomly blown away by the wind. Test tubes in a lab are randomly knocked over.
• Basis for judgment: There is no theoretical reason to believe missingness is related to any variable in the model.
• Treatment: Listwise deletion will not introduce bias—it only sacrifices efficiency (standard errors increase due to reduced sample size).

MAR (Missing At Random)

The probability of missingness may depend on variables already observed in your data, but conditional on these observed variables, missingness is unrelated to the missing variable itself.

• Example: Younger people are more likely than older people to refuse to answer income questions. But "whether to refuse" depends only on age (which you observe in the data); conditional on age, there is no systematic difference in income between refusers and respondents.
• Basis for judgment: You can explain missingness using variables you already have—conditional on these variables, missingness is unrelated to the missing variable itself.
• Treatment: Deletion will introduce bias due to "filtering driven by observed variables" (e.g., deleting young people creates an age-skewed sample). Multiple imputation is the correct choice under MAR.

MNAR (Missing Not At Random)

The probability of missingness depends on the value of the missing variable itself, even after controlling for all observed variables.

• Example: High-income individuals are more likely to hide their income because their income is high. Even if you control for age, gender, and education—high-income individuals are still systematically more likely to be missing.
• Basis for judgment: The cause of missingness is the value of the missing variable itself. This is the most difficult case.
• Treatment: All standard methods (including multiple imputation) cannot guarantee unbiasedness under MNAR. You need selection models (Heckman selection) or pattern-mixture models for sensitivity analysis.

3.2 A Judgment Framework—Which Type Does Your Missingness Belong To?

In empirical papers, you usually cannot prove whether missingness is MCAR or MAR—you can only argue why you believe it is closer to one or the other, and provide robustness evidence.

Standard practice:

1. Compare the distributions of key variables between the complete sample and the full sample—if the differences are small, this supports MCAR.
2. Use a probit/logit model to predict "whether missing"—if observable X variables significantly predict missingness, then at least it is not MCAR.
3. In robustness checks, report results using both deletion and imputation—if the core coefficients are stable, the conclusions are insensitive to the missingness mechanism.

IV. When to Drop?—Deletion Strategies and Their Costs

4.1 Listwise Deletion (Complete-Case Analysis)

Operation: Keep only observations with no missing values on any variable (Y and all X). This is the default behavior in Stata and most software.

When is it reasonable?

• Missingness is completely random (MCAR)—deletion introduces no bias, only a loss of efficiency.
• The missing proportion is small (< 5%)—regardless of the missingness mechanism, a small amount of missingness is unlikely to materially change conclusions.
• Missingness is concentrated in control variables (rather than key variables), and these controls are theoretically weakly correlated with the key X.

When is it unreasonable?

• The missing proportion exceeds 10% and is not MCAR.
• Missingness is systematically concentrated in certain types of observations (e.g., low-income individuals are more likely to be missing → deleting them → core coefficients are based on a sample that has "removed the poor").
• Missingness involves the key explanatory variable or the dependent variable—each deletion removes a potentially critical piece of information.

An underappreciated cost of listwise deletion: if your regression has 10 control variables, each with small amounts of missingness on different observations—the combinations of missingness can lead to a loss of 20–30% of the sample. Everyone is missing a little on some variable, and the result is large-scale deletion. This is the most inefficient aspect of "listwise deletion."

4.2 Pairwise Deletion

Operation: When computing each statistic (e.g., a correlation matrix), use all available pairs of observations—without requiring every observation to be complete on all variables.

Rarely used in regression analysis—because regression requires a complete X matrix and cannot use different samples for different β coefficients. Pairwise deletion is mainly used for correlation matrices in descriptive statistics.

4.3 When You Should Not Drop—Identifying Attrition Bias

In panel data, if an individual disappears completely after period t (no longer appears in any subsequent survey), dropping them may introduce attrition bias. Common methods for identifying attrition bias:

• Generate a dummy variable attrited_i = 1 if the individual is missing in any subsequent period, = 0 if always present.
• Regress/run a Logit of attrited_i on baseline (first-period) characteristics—if baseline variables significantly predict attrition → attrition is not random.

* Define attrition
bysort id: gen last_obs = _n == _N
bysort id: gen attrited = (last_obs == 1 & wave < max_wave)
probit attrited x1 x2 y_baseline   // Baseline characteristics predict attrition

If baseline Y significantly predicts attrition—those who attrit have different initial Y values from those who remain—your panel is systematically filtering out a certain type of person. In this case, fixed effects estimates using only the complete panel may be biased.

V. When to Impute?—Imputation Strategies and Their Pitfalls

5.1 Simple Imputation—Fast, but Easy to Get Wrong

Mean/median imputation: Fill missing values with the sample mean of that variable.

• ❌ Almost never recommended for regression analysis. Mean imputation artificially compresses the variance of the variable (missing values are assigned the most "average" value in the full sample), underestimates standard errors, and inflates significance. It also implicitly assumes that "missing people happen to be exactly at the mean"—with no theoretical basis whatsoever.

Last Observation Carried Forward (LOCF): Fill the current period's missing value with that individual's previous-period observation.

• ⚠️ Common in medicine/epidemiology ("if a patient drops out of a trial, their last health status is considered representative of subsequent status"). Rare in empirical economics—because it assumes that "variables do not change over time," yet economic variables almost always do.
• If a variable changes slowly and missingness is incidental (e.g., one page of a questionnaire was accidentally skipped in one round), LOCF may be reasonable.

Linear interpolation: If observations exist for periods t−1 and t+1, fill period t with their average.

• ✅ A fairly natural approach when variables change smoothly (e.g., BMI, age, GDP) and the number of missing periods is small.
• ❌ Meaningless when variables can jump discretely (e.g., whether unemployed, whether married).

5.2 Multiple Imputation (MI)—The Current Gold Standard

Core idea: Rather than guessing "what the missing value is," draw multiple times from the predictive distribution of missing values conditional on observed data, generate multiple "complete" datasets, run the same regression on each dataset, and finally combine the estimates across datasets using Rubin's Rule.

Operational steps (three stages):

• Stage one (imputation): Use MCMC (Markov Chain Monte Carlo) or chained equations (MICE) to draw missing values from the posterior distribution given observed data. Draw M times (typically M = 20–50), generating M complete datasets.
• Stage two (analysis): Run the regression independently on each complete dataset, obtaining M sets of coefficients and standard errors.
• Stage three (combining): Use Rubin's Rule to average the M coefficients (point estimates) and combine the "within-imputation variance + between-imputation variance" of the M standard errors (total standard error).

The between-imputation variance (variation across imputations) captures the additional uncertainty introduced precisely because you do not know what the missing values truly are. This is the key advantage of MI over mean imputation—it does not just give you a "best guess"; it honestly reports the additional uncertainty your guessing has introduced.

Stata implementation:

* Stage one: declare data and imputation
mi set wide                  // Data in wide format
mi register imputed x1 x2    // Declare variables to be imputed
mi impute chained (regress) x1 x2 = y z1 z2, add(20)
* Stage two: analysis
mi estimate: reg y x1 x2 z1 z2

MI is valid under the MAR assumption—it assumes that "conditional on all observed variables, missingness is unrelated to the true value of the missing variable." If this assumption fails (MNAR), MI cannot guarantee unbiasedness either.

5.3 Panel-Specific Imputation—Exploiting the Time Dimension

Panel data provide an information source that cross-sectional data lack: observations of the same individual in other periods.

Good panel imputation strategies exploit this time dimension:

• If missingness is intermittent (someone occasionally misses one period but has data before and after) → linear interpolation or more flexible longitudinal MI.
• If missingness is monotone (once gone, never returns—panel attrition) → requires special treatment. In this case, end-of-sample missingness is correlated with certain unobservable characteristics of the individual (dying patients drop out of clinical trials; firms on the verge of bankruptcy stop reporting data).

In the case of attrition, MI uses an imputation model (filling missing values based on observed characteristics)—but the problem is that if the trend of attriters "getting worse" cannot itself be predicted by your observed variables, MI cannot capture it.

VI. Testing for and Addressing Panel Attrition

Panel attrition is the most intractable problem in unbalanced panels—because it is usually not MAR but MNAR (attrition is related to unobservable deterioration at the individual level).

6.1 Testing for Attrition Bias

Method 1: Compare baseline characteristics of stayers and attriters

In regression papers, add a baseline balance table—compare whether individuals who ultimately remain in the sample and those who ultimately exit differ significantly on key variables at the initial period. No significant differences → weak evidence supporting random attrition.

Method 2: Test whether "attrition × time" affects key coefficients using interaction terms

Generate a dummy variable attrited_i, run the baseline regression on both the full sample and the retained sample—if the core coefficients do not differ significantly between the two groups, this supports the view that attrition bias is small.

Method 3: Compare core conclusions across different samples

• Use the balanced panel (keeping only individuals observed in all periods)
• Use the unbalanced panel (all available observations)
• If the core coefficients are similar in both → attrition bias may not be severe in this study.

6.2 Methods for Addressing Attrition

Method 1 (conservative): Use only the balanced panel for the baseline specification and the unbalanced panel for robustness.

Method 2 (identification): Heckman selection model—use a probit equation to predict "whether surviving in the panel," generate the Inverse Mills Ratio, and include it in the main equation to correct for selection bias. However, this requires at least one variable that affects attrition but does not directly affect Y (an exclusion restriction)—such a variable is usually very difficult to find.

Method 3 (bounding analysis): Even if attrition bias cannot be corrected, you can conduct a "worst-case" bounding analysis—"even if the Y of attriters declines by XX units per year, does the qualitative direction of the core conclusion remain unchanged?"

VII. A Practical Decision Framework

What type is your missingness?
├── MCAR (completely random, proportion < 5%) → Listwise deletion, report in a footnote
├── MCAR (proportion > 5%) → Consider MI to recover efficiency
├── MAR → Multiple imputation (MI), report coefficient comparisons before and after MI
└── MNAR → 
    ├── Panel attrition? → Heckman selection model / bounding analysis
    └── Intermittent missingness? → Compare deletion vs. imputation vs. complete sample in robustness

Things you should do in all cases:

1. Report the proportion and pattern of missingness (which variables have more missingness? Is missingness monotone or intermittent?)
2. Compare core coefficients from deletion vs. imputation (place in robustness checks)
3. Honestly discuss in the methods or data description section the likely direction of bias introduced by missingness

VIII. Common Misconceptions

8.1 Misconception 1: Assuming Stata's Default Handling Is Correct

Stata automatically deletes observations with missing values—it does not tell you how many were deleted or who they were. In descriptive statistics and regression output, the reduction in observation count is the only visible clue, but many people do not even look at the Number of obs column.

Before running regressions, run misstable summarize—it tells you the missing count for each variable and how the joint pattern of missingness causes how many observations to be discarded.

8.2 Misconception 2: Mean-Imputing Missing Variables and Pretending Nothing Happened

Mean imputation artificially compresses the variance of the variable → standard errors are underestimated → t-values are inflated → it becomes easier to obtain "significant" conclusions. This is equivalent to using missing values to manufacture significance—one of the issues reviewers are most sensitive about.

8.3 Misconception 3: Multiple Imputation Is "Magic"—Done It, Done Right

MI is valid under the MAR assumption. If your data are MNAR (e.g., high-income individuals refuse to answer because their income is high), MI uses the patterns of "low-income individuals" to predict the missing values of "high-income individuals"—systematically underestimating the missing values. MI is not a panacea—it is only as good (or as bad) as the imputation model you use.

IX. Summary

Five core takeaways for handling missing values in unbalanced panels:

1. Unbalanced ≠ problematic. The cause of attrition is the problem—if missingness is correlated with the model's dependent variable or error term, estimates face selection bias.
2. Determining the type of missingness is the first step. MCAR → deletion is unbiased but loses efficiency. MAR → multiple imputation. MNAR → selection models or bounding analysis.
3. "Imputing" has more opaque costs than "deleting"—easier to make mistakes without realizing it. Mean imputation, LOCF, and single-regression imputation all artificially compress uncertainty. Multiple imputation is the only standard method that honestly reports imputation uncertainty.
4. The best strategy is transparency and robustness. Report missing proportions, compare deletion and imputation results, and honestly discuss the likely direction of attrition bias—let readers judge for themselves.
5. For most empirical papers, listwise deletion is sufficient—provided you honestly discuss how it has changed your sample composition. If your core conclusions maintain the same sign and significance under both MI and listwise deletion, reviewers will find it difficult to challenge the sensitivity of your conclusions to the missing-value handling method.

One-sentence conclusion:

"Missing values in an unbalanced panel, if handled poorly, mean you are making inferences with 'filtered data'—and you are not even aware of this filtering process. Missingness is not a matter of 'a few rows of data missing'—it inserts a silent, unexamined filter between your research conclusions and the real world. Multiple imputation is currently the most honest alternative to this filter—but it still requires an assumption you may never be able to test: conditional on everything you have observed, missingness is unrelated to what is missing. If you cannot even believe this assumption—you should at least tell readers in your robustness checks: if I am wrong, how far could the conclusions deviate?"

X. Presentation Suggestions for Bilibili/WeChat Official Account

• Bilibili video: Suggest using "vanishing people" as the narrative theme. Opening: a neat grid (balanced panel—every individual observed in every period), rows neatly arranged. Then observations gradually dim and disappear one by one—some disappear for one period and return (intermittent missingness), others disappear permanently (attrition). Voiceover: "People have vanished from your panel data. Stata automatically ignores them. But the question is—did the vanishing people leave randomly, or did they leave systematically?" Act One "Three Types of Missingness": three scenario cards labeled MCAR, MAR, MNAR—MCAR: a die randomly decides which rows disappear. MAR: younger people are more likely to disappear—but you know who the younger people are. MNAR: high-income individuals hide their income because their income is high—even if you know everything else about them, they will still hide. Act Two "Delete vs. Impute": two buttons—"Delete" erases incomplete observations (sample frame shrinks, red box labeled "possible selection bias"), "Impute" fills missing positions with dashed lines (multiple colored dashed lines converging from different angles—MI's multiple sampling process). Act Three "The Secret of MI": a missing value has multiple candidate values drawn from the distribution—multiple complete datasets are generated, the same regression is run on each, and finally the coefficients are combined (Rubin's Rule formula expands). Within-imputation variance + between-imputation variance are highlighted—voiceover: "MI does not just fill in a best guess—it honestly says: 'Because I guessed, my uncertainty is larger than the original data—this extra portion of uncertainty, I call it between-imputation variance.'"
• WeChat official account: Create an infographic core for the definitions and judgment framework of the three missingness mechanisms (MCAR/MAR/MNAR three-column comparison: definition, examples, available methods). Present the method spectrum of deletion vs. imputation (from conservative to fully exploiting information) as a horizontal gradient color band. Present the three stages of multiple imputation (imputation → analysis → combining) as a vertical flowchart, with Rubin's Rule formula annotated. Present the three testing methods for panel attrition as cards. Present the practical decision framework (tree diagram from missingness type to treatment plan) as a poster-level infographic. Create code cards for the Stata command quick reference (misstable summarize / mi set / mi impute chained / mi estimate).
• Recommended titles:
  • Main title: "Missing Values in Unbalanced Panels—When to Drop? When to Impute?"
  • Alternative title: "The Vanishing People in Panel Data—Attrition Bias, Multiple Imputation, and Honest Uncertainty"
  • New media title: "The 16,000 Observations Stata Silently Deleted—Did You Notice That People Have Vanished from Your Panel?"
• Key quotes:

  "An unbalanced panel itself is not the problem—the problem is whether those 'vanishing people' are systematically different from those who 'stayed.' If so, your conclusions hold only for 'those who can be tracked, the more stable population.'"

  "Mean imputation does not tell you 'I guessed'—it just fills in a number and then pretends that number is real. Multiple imputation tells you 'I guessed, and because I guessed, my uncertainty is amplified by this additional uncertainty.' Guessing without saying so is deception. Guessing and saying so is honesty."

  "Missingness is not 'a few rows of data missing'—it inserts a filter between your sample and the real world. Your task is not to pretend this filter does not exist, but to study what it filters out, and then leave a transparent discussion of this filtering in your conclusions."

  "Multiple imputation is the best approach under the MAR assumption—but if your missingness is MNAR (the missing people happen to be 'worse off' precisely on the variable they are missing), MI is also powerless. In the worst case—use bounding analysis to tell readers: even if I make the most pessimistic assumptions about missingness, would the qualitative direction of my core conclusions change?"