Why Does OLS Produce Just One Number, Yet We Call It \"Unbiased\" and \"Efficient\"?

1. Introduction: A Single Number—Where Do Expectation and Variance Come From?

Anyone who has studied introductory econometrics has experienced this cognitive moment—

You run a regression:

reg wage education

Stata outputs a number: $\hat{\beta}_1 = 0.08$. The coefficient on years of education is 0.08. This is a number, just like 3.14 or 42—a definite, concrete number.

Then the textbook tells you:

• The OLS estimator is unbiased: $\mathbb{E}[\hat{\beta}_1] = \beta_1$
• The OLS estimator is efficient: among all linear unbiased estimators, it has the smallest variance
• The variance of the OLS estimator is $\text{Var}(\hat{\beta}_1) = \frac{\sigma^2}{\sum (X_i - \bar{X})^2}$

You sit there with a nagging question in your mind:

Wait—$\hat{\beta}_1$ is a number, 0.08. How can a number have an expectation? How can a number have a variance? Isn't variance a concept that describes the dispersion of a random variable?

This is not a failure of understanding on your part. It is a sign that your intuition is too good. You vaguely sense that the textbook is talking about two different things while using the same symbol $\hat{\beta}_1$—sometimes it refers to "the specific numerical value you computed from one sample," and sometimes it refers to "the computational formula itself." And these two things have entirely different properties.

This article aims to resolve this confusion. It requires three concepts: population regression function vs. sample regression function, estimator vs. estimate, and the true meaning of unbiasedness.

Core message: The OLS estimator is a random variable—because it is a function of the sample, and the sample is randomly drawn. Each time you draw a different sample, the $\hat{\beta}_1$ you compute will differ. "Unbiased" means that if you repeatedly draw samples and compute a $\hat{\beta}_1$ each time, the average of these $\hat{\beta}_1$ values will converge to the true $\beta_1$. The 0.08 in your hand is merely one realization of this random variable.

2. Two Worlds: Population Regression Function vs. Sample Regression Function

2.1 Population Regression Function (PRF)—Real but Invisible

In the world of the population, the true relationship between variables is described by the Population Regression Function (PRF):

$Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$

Here $\beta_0$ and $\beta_1$ are true, fixed, unknown parameters. They represent the essence of the relationship between X and Y in the population—embedded in the underlying logic of how the world operates, and you can never observe them directly. Just like physical constants, the gravitational constant G is a fixed value that you cannot obtain through any single measurement—you can only estimate it through experimental data.

2.2 Sample Regression Function (SRF)—What You Can See

What you can see is only a sample drawn from the population. Based on this sample, you use the OLS formula to compute a fitted line:

$\hat{Y}_i = \hat{\beta}_0 + \hat{\beta}_1 X_i$

This is the Sample Regression Function (SRF). $\hat{\beta}_0$ and $\hat{\beta}_1$ are not the true parameters—they are estimates of the true parameters based on this particular sample.

2.3 The Crucial Step: When the Sample Changes, $\hat{\beta}_1$ Changes Too

Now consider a thought experiment—

You draw another random sample from the same population (same sample size, but containing different individuals). You apply the OLS formula to this new sample. What do you get?

You get a different $\hat{\beta}_1$. Perhaps the first time it is 0.08, the second time 0.085, the third time 0.076.

The sample is random → the $\hat{\beta}_1$ computed from the sample is also random.

This is the key to understanding everything. $\hat{\beta}_1$ is not a fixed number—it is a function of the sample:

$\hat{\beta}_1 = f(\text{sample}) = \frac{\sum_{i=1}^{n} (X_i - \bar{X})(Y_i - \bar{Y})}{\sum_{i=1}^{n} (X_i - \bar{X})^2}$

Precisely because the sample is randomly drawn, the output of $f(\text{sample})$ is also random—it is a random variable. And a random variable naturally has an expectation, a variance, and a distribution.

The 0.08 in your hand is merely "one realization" of this random variable on one particular sample.

3. Estimator vs. Estimate—A Badly Underappreciated Distinction

This distinction is the foundation of the entire logic of econometric inference, yet it is often glossed over in teaching.

• Estimator: A formula or rule that tells you how to compute a numerical value from any sample. It is a random variable because it maps a random sample to a numerical value.

  Example: $\hat{\beta}_1 = \frac{\sum (X_i - \bar{X})(Y_i - \bar{Y})}{\sum (X_i - \bar{X})^2}$ is the OLS estimator.

• Estimate: The specific numerical value obtained by applying the estimator formula to one particular sample. It is a fixed number.

  Example: The 0.08 you obtain after running the regression on the CHFS 2019 data is the value of the estimator on the "CHFS 2019 sample."

The same symbol $\hat{\beta}_1$ in textbooks is used to refer to both of these things—this is the root of the cognitive confusion. When the textbook writes $\mathbb{E}[\hat{\beta}_1] = \beta_1$, the $\hat{\beta}_1$ refers to the estimator (a random variable), not the estimate (a fixed number).

4. What Does Unbiasedness Really Mean?—Not "Your Estimate Is Correct"

Now we can precisely understand "unbiasedness":

$\mathbb{E}[\hat{\beta}_1] = \beta_1$

This does not mean "the 0.08 in your hand equals the true value." Unbiasedness is a statement about the estimator (the formula itself), not about the estimate (the specific 0.08).

Its precise meaning is:

If you repeatedly and infinitely draw samples from the population, each time drawing n observations and each time computing a $\hat{\beta}_1$ using the OLS formula, then the average of these infinitely many $\hat{\beta}_1$ values equals the true $\beta_1$.

In one sentence: The OLS estimator—this "formula"—does not systematically aim left or right; it aims at the bullseye. Although each arrow (each specific estimate) may land left or right, if you shoot infinitely many arrows, the average position of the arrow impacts is the bullseye.

"Unbiased" does not guarantee that the specific estimate in your hand is close to the true value. It only guarantees that "if I make a long-term commitment to your ability to repeatedly sample, you will not be systematically led in the wrong direction." How far the single 0.08 in your hand is from the true value—that is a question answered by the standard error and confidence interval.

5. Monte Carlo Simulation—"Seeing" Randomness with Your Own Eyes

Nothing demonstrates "why the estimator is a random variable" more intuitively than simulation. Although we cannot run code here, you can follow along with a "thought simulation":

Set up the real world: Suppose the true population regression function is $Y = 1 + 2X + \varepsilon$, where $\varepsilon \sim N(0,1)$. So the true $\beta_1 = 2$.

Steps:

1. Randomly draw 100 observations from the population (n = 100), and use the OLS formula to compute the first $\hat{\beta}_1$.

   • Result: $\hat{\beta}_1 = 2.03$.

2. Randomly draw again 100 observations (a second sample, different from the first), and compute OLS again.

   • Result: $\hat{\beta}_1 = 1.94$.

3. Repeat the drawing 1000 times. Each time is an independent random sample, and each time you compute a $\hat{\beta}_1$.

4. Now you have 1000 values of $\hat{\beta}_1$: 2.03, 1.94, 2.07, 1.98, 2.01, ...

Now, plot these 1000 values as a histogram. What do you see?

You will see them densely distributed around 2, forming a bell-shaped curve approximating a normal distribution—with the center at 2. The average of these 1000 values is approximately 2.001 (very close to 2).

This is unbiasedness manifesting before your eyes—not "a particular $\hat{\beta}_1$ equals 2," but "the average of many $\hat{\beta}_1$ values equals 2." And the dispersion of these 1000 values themselves (i.e., their standard deviation) is the standard error of the OLS estimator—it measures "how much the estimate fluctuates from sample to sample."

6. Efficiency—Why Is OLS BLUE?

Once you understand that "the estimator is a random variable," efficiency follows naturally.

The OLS estimator is not the only estimator—you could use other formulas to compute a slope from the sample. For instance, you could connect the first and last data points to estimate the slope, or you could use median regression.

Efficiency states: Among all linear unbiased estimators, the OLS estimator has the smallest variance. Mathematically, for any other linear unbiased estimator $\tilde{\beta}_1$:

$\text{Var}(\hat{\beta}_1^{\text{OLS}}) \leq \text{Var}(\tilde{\beta}_1)$

This is the Gauss-Markov Theorem—OLS is BLUE (Best Linear Unbiased Estimator).

"Best" here does not mean most accurate—it means "smallest variance": that is, among all unbiased linear estimation methods, OLS produces estimates with the smallest sampling fluctuation. Of course, this conclusion depends on several key conditions holding—the error term being homoskedastic and free of autocorrelation.

7. Summary: A Cognitive Upgrade from "One Number" to "A Random Variable"

One sentence to close:

"The 0.08 in your hand is just a number—it has no expectation and no variance. But the formula that produced this number, because it gives different answers each time it faces a different random sample, is a random variable. Unbiasedness, efficiency, variance—these all describe properties of that formula, not properties of your specific number. Once you understand this, you have truly entered the logic of econometric inference."

8. Presentation Suggestions for Bilibili/WeChat Official Account

• Bilibili video: Consider using animation to show the process of "repeated sampling → obtaining 1000 $\hat{\beta}_1$ values → forming a normal distribution." You could start with a blinking dot representing one sample, and after blinking 1000 times, the dots cluster around 2 to form a bell-shaped curve. This provides the most intuitive visual impact.
• WeChat article: The three-row concept comparison table (true parameter vs. estimator vs. estimate) is recommended as a core infographic. The Monte Carlo simulation description should be accompanied by a schematic "histogram of 1000 estimates."
• Recommended titles:
  • Main title: 《Why Does Regression Produce Only One Number, Yet We Call It "Unbiased" and "Efficient"?》
  • Alternative title: 《That 0.08 Has No Expectation or Variance—What Has Expectation and Variance Is the Formula That Produced It》
• Key quote:

  "Unbiasedness does not guarantee that the estimate in your hand is close to the true value—it only guarantees that the method producing this estimate will not systematically go astray."