Panel Data and Fixed Effects: Choosing Between FE and RE with TWFE in Practice

Structure of This Article

Principles: Differences and choices among FE / RE / Pooled OLS
Intuition: When to use fixed effects
Code: Complete Stata code for panel data analysis

Layer 1: Principles

The Panel Data Model

Panel data contain both a cross-sectional dimension $i$ and a time dimension $t$ :

Y_{it} = X_{it}'\beta + \alpha_i + \lambda_t + \varepsilon_{it}

where:

$\alpha_i$ : unit fixed effects (time-invariant unobserved heterogeneity)
$\lambda_t$ : time fixed effects (common time shocks)
$\varepsilon_{it}$ : idiosyncratic error term

Comparison of Three Estimation Methods

Method	Assumption	Applicable Scenario
Pooled OLS	$\text{Cov}(\alpha_i, X_{it}) = 0$	No unit heterogeneity
Random Effects (RE)	$\text{Cov}(\alpha_i, X_{it}) = 0$	Unit effects uncorrelated with explanatory variables
Fixed Effects (FE)	Allows $\text{Cov}(\alpha_i, X_{it}) \neq 0$	Unit effects may be correlated with explanatory variables

The Hausman Test

H = (\hat{\beta}_{FE} - \hat{\beta}_{RE})' [\text{Var}(\hat{\beta}_{FE}) - \text{Var}(\hat{\beta}_{RE})]^{-1} (\hat{\beta}_{FE} - \hat{\beta}_{RE}) \sim \chi^2(k)

$H_0$ : RE is consistent and efficient (i.e., $\text{Cov}(\alpha_i, X_{it}) = 0$ )
If $H_0$ is rejected: use FE

Practical advice: In causal inference research, FE should almost always be used. The core assumption of RE (unit effects uncorrelated with explanatory variables) is rarely credible in most economic settings.

Clustered Standard Errors

In panel data, the error term $\varepsilon_{it}$ typically exhibits serial correlation within units. This calls for clustered standard errors (cluster-robust SE):

\hat{V}_{cluster} = (X'X)^{-1} \left(\sum_{g=1}^{G} X_g' \hat{u}_g \hat{u}_g' X_g \right) (X'X)^{-1}

Principles for choosing the clustering level:

Cluster at the highest level at which the treatment variable varies
The number of clusters $G$ should be sufficiently large (generally $G \geq 50$ )
When $G$ is too small, consider the wild cluster bootstrap

Layer 2: Intuition

The Intuition Behind Fixed Effects

Fixed effects are essentially a "within transformation" (group-mean demeaning) operation:

\tilde{Y}_{it} = Y_{it} - \bar{Y}_i

This eliminates all time-invariant unit characteristics (both observed and unobserved), using only within-unit variation over time to identify the effect.

Example: Studying the impact of education spending on economic growth

Without FE: wealthy provinces have both high education spending and fast growth, leading to biased estimates
With FE: compare how changes in education spending within the same province across years affect changes in growth

Layer 3: Stata Code

// ═══════════════════════════════════════════════
// Panel Data and Fixed Effects Analysis
// ═══════════════════════════════════════════════
 
clear all
set seed 44444
 
// Simulate panel data
local N = 100
local T = 10
set obs `=`N'*`T''
gen id = ceil(_n/`T')
bysort id: gen t = _n
 
// Unit fixed effects (correlated with x!)
gen alpha_i = rnormal(0, 2) if t == 1
bysort id: replace alpha_i = alpha_i[1]
gen x = 0.5*alpha_i + rnormal(0, 1)
gen y = 1 + 2*x + alpha_i + 0.3*t + rnormal(0, 1)
// True coefficient β = 2
 
xtset id t
 
// ═══ Pooled OLS (biased) ═══════════════════
reg y x, cluster(id)
est store pooled
 
// ═══ Random Effects ═══════════════════════════
xtreg y x, re
est store re
 
// ═══ Fixed Effects ═══════════════════════════
xtreg y x, fe cluster(id)
est store fe
 
// ═══ reghdfe (recommended) ═════════════════════
reghdfe y x, absorb(id t) cluster(id)
est store twfe
 
// ═══ Hausman Test ═══════════════════════════
hausman fe re
// If p < 0.05, use FE
 
// ═══ Comparison of Results ═════════════════════
esttab pooled re fe twfe, ///
    se star(* 0.10 ** 0.05 *** 0.01) ///
    title("Panel Data Estimation Comparison") ///
    mtitles("Pooled" "RE" "FE" "TWFE") ///
    note("True β = 2")

References

Wooldridge, J. M. (2010). Econometric Analysis of Cross Section and Panel Data. MIT Press.
Cameron, A. C., & Miller, D. L. (2015). A Practitioner's Guide to Cluster-Robust Inference. Journal of Human Resources, 50(2), 317-372.