The Phantom Variance Fallacy

Presenting random variables as deterministic values

I have a simple question about two football teams:

• Team A averages 3.3 yards per play
• Team B averages 5.0 yards per play

Which team wins?

In this idealized model, it looks like Team B. In reality, it’s Team A, but it’s due to a piece of information that was not given—the distribution of yards per play.

Football has a very clear risk of ruin during a drive: four downs without 10 yards of progress and you turn the ball over to your opponent. Thus, the shape of the distribution is key to determining the effectiveness of a play type. We can imagine that Team A always runs the ball with a very low variance result, and Team B always throws the ball with a very high variance result.

This situation is analogous to many real-world situations. Perhaps the most illustrative is to pick up nickels in front of a steamroller [1]. There are many opportunities, in financial markets, betting markets, criminal enterprise, and elsewhere where you are offered a seemingly endless supply of wins. Of course, there is invariably some hidden variance, a steamroller or otherwise, ready to ruin your day at some unexpected time.

Roger Lowenstein’s When Genius Failed [2] is an excellent book on this topic. It tells the story of how hidden risk brought down Long-Term Capital Management.

The fallacy is to present a random variable as if it were a deterministic value. Sam Savage calls the broader mistake “the flaw of averages” [3]. This usually happens unintentionally, but skilled rhetoricians use it frequently. You can use this bit of mathematical sleight of hand to create seemingly valid, but completely incorrect conclusions.

In fact, even my original question was flawed. When I said Team A wins, I meant that Team A wins 77.9% of games in this simplified model where Team A’s yards per play are determined by a normal distribution, \(N(3.3, 0.8^2)\), and Team B’s from a bimodal mixture: 70% \(N(0.5, 0.45^2)\) and 30% \(N(15.5, 2^2)\). Obviously, this verbose phrase doesn’t have a “headline” quality most people look for when making strong statements.

This insight is not new—“but I arrived at it independently” [4]. Sam Savage describes the broader statistical error as “the flaw of averages”: plans based on average inputs often fail because averages hide uncertainty [3].

---

Appendix: A simplified football model

This is deliberately a simplified model of football. Each team gets 10 drives, every drive starts at its own 25-yard line, and a drive lasts at most 14 plays. Normal downs apply: gain 10 yards within four plays to reset the downs. A touchdown scores 7 points. On fourth down at or beyond the opponent’s 35-yard line, the team kicks a 3-point field goal; it also kicks if the 14-play limit ends in field-goal range. We ignore punts, field-goal misses, penalties, turnovers, clock effects, and extra-point variation.

Team A’s yards per play follow a low-variance normal distribution, \(N(3.3, 0.8^2)\), truncated at zero. Team B is bimodal: 70% of plays come from \(N(0.5, 0.45^2)\) and 30% come from \(N(15.5, 2^2)\), also truncated at zero.

import numpy as np
import plotly.graph_objects as go
rng = np.random.default_rng(42)

def team_a_play(size=None):
    return np.maximum(rng.normal(3.3, 0.8, size), 0)

def team_b_play(size=None):
    shape = () if size is None else size
    boom = rng.random(shape) < .30
    return np.where(boom, np.maximum(rng.normal(15.5, 2, shape), 0),
                    np.maximum(rng.normal(.5, .45, shape), 0))

a_plays = team_a_play(200_000)
b_plays = team_b_play(200_000)

Probability distributions for yards gained on one play.

Team A mean: 3.30; Team B mean: 5.03

def drive(play):
    position, series_yards, down = 25., 0., 1
    for _ in range(14):
        if down == 4 and position >= 65:
            return 3
        gain = float(play())
        position += gain
        series_yards += gain
        if position >= 100:
            return 7
        if series_yards >= 10:
            series_yards, down = 0., 1
        else:
            down += 1
            if down > 4:
                return 0
    return 3 if position >= 65 else 0

def games(play, n=20_000):
    return np.array([sum(drive(play) for _ in range(10)) for _ in range(n)])

a_scores, b_scores = games(team_a_play), games(team_b_play)

The resulting probability distributions for points scored per game.

Finally, pair one simulated Team A score with one simulated Team B score. This assumes independent offenses and ignores defensive matchups.

Team A win probability: 77.9%
Team B win probability: 17.7%
Tie probability: 4.4%

Team B gains more yards per play on average. Team A still wins more often in this model because its lower-variance plays sustain drives more reliably.

This result depends on the chosen distributions and simplified rules; it does not imply that low-variance offenses generally outperform explosive ones.

References

[1] RidgeHaven Capital, “Picking Up Nickels In Front Of A Steam Roller” - Seeking Alpha

[2] Lowenstein, Roger, “When Genius Failed: The Rise and Fall of Long-Term Capital Management” - Penguin Random House

[3] Savage, Sam, “The Flaw of Averages” - Harvard Business Review

[4] Hughes, Sarah, “Mad Men: season one, episode four” - The Guardian