What Is One Actually Doing When Conducting Empirical Research?

I. Introduction: A Familiar Yet Unexamined Term

"Is this conclusion empirically supported?" "We need to conduct empirical research." "This paper is purely theoretical and lacks empirical testing."

In academic discourse, "empirical research" is a high-frequency term. Yet few pause to ask: When a researcher says they are "doing empirical research," what exactly are they doing?

Running regressions in Stata is empirical research; having subjects fill out questionnaires in a psychology lab is empirical research; randomly assigning patients to different drugs in a hospital is also empirical research—can these really be the same thing?

This article helps you clarify three things:

1. What fundamentally distinguishes empirical research from theoretical research and speculative research?
2. Among empirical studies themselves, how do the practices of psychology, economics, and medicine differ?
3. What are the complete steps of an empirical study in economics?

II. Three Research Paradigms: Empirical, Theoretical, and Speculative

Before discussing specific operational steps, we must first clarify where empirical research sits within the broader research landscape.

2.1 Speculative Research: Relying on Logic and Insight

Speculative research is humanity's oldest mode of inquiry. It relies on the researcher's introspection, intuition, and logical deduction, without the need for systematic collection of external evidence.

Typical forms: The political philosophy of pre-Qin thinkers; Adam Smith's discussion of the division of labor in The Wealth of Nations (he did not conduct data analysis but reasoned from observation and inference); most doctrinal interpretations in legal scholarship.

Core characteristics:

• Data is not required; the quality of reasoning depends on insight and logical rigor.
• Conclusions may be highly insightful but cannot be systematically "falsified."
• Suitable for addressing "what ought to be" questions and conceptual analysis, but inadequate for answering "what is" questions.

2.2 Theoretical Research: Building "If...Then..." Logical Chains with Models

Theoretical research constructs formal frameworks to derive conclusions. In the context of economics, this typically means building mathematical models.

Typical forms: Market equilibrium models in economics, string theory in physics, equilibrium analysis in game theory.

Core characteristics:

• Focuses on "given assumptions, what necessarily follows," without touching data.
• Produces testable hypotheses, not final conclusions.
• The criterion for evaluation is logical consistency, not agreement with real-world data.

2.3 Empirical Research: Answering Questions with Systematic External Evidence

What distinguishes empirical research is this: it requires the researcher to step outside their own mind and systematically collect and analyze evidence from the external world.

Here, "evidence" can be behavioral data from the laboratory, statistical data from national censuses, transcripts of interview recordings, or user click logs from a company's backend—the key is that both the collection methods and the analytical methods must be systematic and replicable.

One Table to Understand the Three Paradigms

In one sentence: speculative research says "I think it should be this way," theoretical research says "if the assumptions hold, then it must be that way," and empirical research says "let us see what is actually the case."

III. Under the Same Banner, Different Battles: A Comparison of Empirical Research Across Three Disciplines

Although all claim to "do empirical work," the operational approaches and focal concerns of different disciplines can differ enormously. Here we compare three representative disciplines: psychology, economics, and medicine.

3.1 Empirical Research in Psychology: Controlled Experiments + Analysis of Variance

Empirical research in psychology takes the laboratory experiment as its dominant paradigm. The core logic is: manipulate one variable, observe changes in another, and test whether the observed change exceeds what random fluctuation would produce.

Typical steps:

1. Recruit subjects (usually first-year students taking introductory psychology)
2. Randomly assign to experimental and control groups
3. Manipulate the independent variable (e.g., show the experimental group a violent video and the control group a neutral video)
4. Measure the dependent variable (e.g., scores on an aggression scale)
5. Test between-group differences using t-tests or analysis of variance (ANOVA)

Core model:

where $T_i$ is the treatment indicator variable (0 or 1), and $\tau$ is the experimental treatment effect—this is the most basic statistical model in psychology experiments. More complex designs incorporate mediating and moderating variables, forming the framework of "mediation-moderation analysis."

Core concerns:

• Internal validity: Is the observed effect truly caused by the manipulation?
• Whether randomization succeeded: Are the experimental and control groups truly comparable on baseline characteristics?
• Effect size: Is the effect statistically significant—and how large is it?

A classic example: The Stanford Prison Experiment (Zimbardo, 1971). Students were randomly assigned to play the roles of "guards" and "prisoners," and the effect of role on behavior was observed. This experiment perfectly demonstrates the power of empirical research in psychology—through randomization, the philosophical question "does role affect behavior" was transformed into an operable empirical question. Of course, its ethical problems and subsequent challenges to internal validity also illustrate the boundaries of this type of research.

3.2 Empirical Research in Economics: Observational Data + Causal Identification

Empirical research in economics (i.e., econometrics) faces a fundamental predicament: economists can almost never run experiments.

You cannot randomly raise wages for one group and lower them for another to estimate the elasticity of labor supply; you cannot randomly have one country join the WTO and another stay out to study the effects of trade liberalization. The data economists obtain are almost entirely observational—people have already "self-selected" in their real lives.

This determines the core proposition of empirical research in economics: How can we identify causal relationships when randomization is impossible?

Typical steps (what we call the "five-piece causal inference toolkit"):

1. Pose an economic question that can be tested with data
2. Find an identification strategy—this is usually the most valuable part of the entire paper
3. Estimate the causal effect using an econometric model
4. Conduct a series of robustness checks to rule out alternative explanations
5. Discuss the economic significance of the estimated results

Core model matrix (research design takes priority over specific models):

• Instrumental variables (IV): Find a third-party variable that affects Y only through its effect on X. Classic example: Angrist used draft lottery numbers as an instrument for "military service" to estimate the effect of service on lifetime earnings.
• Difference-in-differences (DID): Compare the differential change in outcomes between the treatment group and the control group before and after a policy. The Card & Krueger minimum wage study discussed in the previous article is a canonical example.
• Regression discontinuity (RDD): Near a certain cutoff, treatment assignment is "as-if random." For example, students scoring a few points above or below the college entrance exam cutoff have nearly identical ability, yet one group goes to college and the other does not—this provides an excellent opportunity to estimate the returns to college education.
• Fixed effects models: Control for time-invariant omitted variables. For instance, when studying whether democracy promotes economic growth, including country fixed effects can absorb time-invariant factors such as geography and culture.

Core concerns:

• Identification: Can your estimate be interpreted as causal? This is the soul of economics. An empirical paper with poor identification is worthless no matter how high the R².
• Endogeneity: Is your key independent variable correlated with the error term? If so, OLS estimates are biased and inconsistent.
• Economic significance vs. statistical significance: An effect being statistically significant does not mean it is economically important. A variable may be significant at the p < 0.001 level, yet have a coefficient of only 0.0001—which may be economically negligible.

3.3 Empirical Research in Medicine: Clinical Trials + Evidence Hierarchy

Medicine is chosen as the third comparative discipline because it forms a striking contrast with economics: medicine can conduct randomized controlled trials (RCTs), while economics generally cannot.

The core paradigm of medical empirical research—the RCT:

1. Recruit eligible patients
2. Randomly assign to treatment and control groups (this is the key difference!)
3. Double-blind (neither patients nor doctors know who is in which group)
4. Follow up for a period and measure pre-specified outcome indicators
5. Estimate the treatment effect using intention-to-treat (ITT) or per-protocol (PP) analysis

The medical evidence hierarchy:

Core concerns:

• Whether randomization and blinding were rigorous: Any breach of randomization downgrades the entire study.
• Clinical significance vs. statistical significance: A drug may lower blood pressure by 2 mmHg with p < 0.001—statistically perfect—but is such a reduction clinically meaningful?
• External validity: Trials are conducted on highly selected "ideal patients." Can the results be generalized to the diverse patient populations of the real world?

3.4 Comparison Table of Key Differences Across Three Disciplines

IV. A Complete Step-by-Step Checklist for Empirical Research in Economics

Having understood the cross-disciplinary landscape, we now zoom back in to focus on the specific operations of empirical research in economics. A standard empirical economics research project typically proceeds through the following seven steps:

Step 1: Pose an Economic Question

Not every question is an economic question. "How has income inequality in China changed over the past two decades?"—this is an economic question. "Can education reduce inequality?"—this is a causal question that can be answered with empirical methods.

A good empirical question typically satisfies three criteria: important (worth answering), feasible (data are available to answer it), and tension-filled (intuition may be wrong).

Step 2: Define and Operationalize Economic Variables

Translating abstract concepts into measurable indicators is far more difficult than it appears.

• You want to study "human capital" → Do you use years of schooling? Standardized test scores? Or years of work experience?
• You want to study "health" → Do you use self-rated health (subjective)? BMI (objective but crude)? Or medical expenditures (subject to selection bias)?
• You want to study "institutional quality" → Do you use the World Bank governance indicators? The strength of property rights protection? Or the efficiency of judicial contract enforcement?

The choice of variables itself embodies a theoretical stance. Choosing different proxy variables can lead to completely different empirical conclusions, which is why a good empirical researcher spends considerable time justifying "why I use this indicator to measure this concept."

Step 3: Specify the Econometric Model

This was the core content of the previous article; here we provide a brief review. An econometric model is essentially a hypothesized mathematical structure that describes the data-generating process linking the dependent and independent variables:

• $Y_i$: the variable you want to explain
• $X_i$: the key independent variable of interest
• $Z_i$: control variables (you acknowledge they affect Y, but they are not your current focus)
• $\varepsilon_i$: contains everything you did not measure, do not know about, or cannot measure
• $\beta_1$: what you truly want to estimate—the causal effect of X on Y

Step 4: Choose an Identification Strategy

This is the most distinctive step in empirical economics—in other disciplines, you rarely see the term "identification strategy" occupying the central position in a paper.

The identification strategy answers this question: Why do you claim that $\beta_1$ is a causal effect rather than merely a correlation?

Common identification strategies and their applicable scenarios:

A regression without an identification strategy is merely descriptive statistics with some numbers attached.

Step 5: Estimate Model Parameters

This is technically the "simplest" step—with modern statistical software, you only need to write one line of code:

reg ln_wage education experience exp_squared, robust

But simple does not mean unimportant. At this step, you need to handle:

• The choice of standard error estimation methods (heteroskedasticity-robust standard errors, clustered standard errors, bootstrap standard errors)
• Treatment of missing data
• Detection of outliers
• Diagnosis of multicollinearity

Step 6: Test Statistical Significance

Statistical significance answers: What is the probability that the effect we observe is purely due to sampling error?

• p < 0.05: If the true effect is zero, the probability of observing our current result (or a more extreme one) is less than 5%.
• 95% confidence interval: We are 95% confident that the true parameter lies within this interval.

Key reminder: statistical significance ≠ economic importance. In large samples, even an effect so small as to be meaningless can be statistically significant. This is why the final step is needed.

Step 7: Assess Economic Significance

Economic significance answers a different question: Is this effect important in the real world?

For example, suppose you estimate that "each additional year of education increases wages by 0.5%." This effect may be statistically significant (p < 0.01, with a sufficiently large sample), but economically—if one more year of schooling earns only 0.5% more, education investment is almost not worthwhile.

Assessing economic significance typically involves:

• The magnitude of the coefficient itself (elasticities, semi-elasticities)
• Comparison with baseline levels (relative to the average wage, what does 0.5% mean?)
• Comparison with estimates in the existing literature (what have other studies found using different data?)
• Counterfactual simulations (if the policy changes, how much would the overall economy be affected?)

V. Conclusion: The Soul of Empirical Research—Eliminating Alternative Explanations

Looking back across the three disciplines and the seven-step process, you will find that all the effort in empirical research points in the same direction:

Systematically rule out alternative explanations so that the conclusion "X causes Y" becomes more credible than any alternative narrative.

• Psychology uses randomization to rule out individual differences as an alternative explanation.
• Economics uses identification strategies to rule out reverse causality, omitted variables, self-selection, and other alternative explanations.
• Medicine uses RCT + double-blinding to rule out placebo effects and assessor bias as alternative explanations.

And all the statistical tools—t-tests, regression, instrumental variables, analysis of variance, survival analysis—are essentially different forms of the same thing: separating the "signal within the signal" in the data.

VI. Finally: A Map of "Doing Empirical Research Is..."

If someone next time asks you "what exactly are you doing when doing empirical research," you can hand them this map:

Doing empirical research = posing a falsifiable question
            + finding or generating data that can answer the question
            + choosing a research design that can rule out alternative explanations
            + estimating the effect of interest with a statistical model
            + distinguishing between "statistically significant" and "practically important"
            + honestly reporting your assumptions, limitations, and uncertainties

And the ultimate virtue of empirical research is not "proving that you are right," but rather—making your conclusions vulnerable to being overturned by facts. The process of overturning is itself the progress of science.

VII. Presentation Suggestions for Bilibili/WeChat Official Account

• Bilibili video: In the "three-discipline comparison" section, consider using three side-by-side screens showing visual imagery of a psychology laboratory, an economist examining charts, and a doctor conducting a clinical trial; then merge them into a single frame at the end with the text "Eliminate Alternative Explanations."
• WeChat official account: The three-discipline comparison table is well suited for an infographic; the seven-step process is best presented as a vertical flowchart with an icon for each step.
• Recommended titles (for both official account and video):
  • Main title: "What Is One Actually Doing When Conducting Empirical Research?"
  • Alternative title: "Psychology, Economics, Medicine—Three Ways of Fighting Under the Same Banner of 'Empirical'"
• Key quotes (suitable for cover cards or closing frames):

  "The soul of empirical research is not proving that you are right, but making your conclusions vulnerable to being overturned by facts."