Deleting the companies that died flattered this strategy at 4 of 4 basket widths — by +1.16 to +2.49 CAGR points a year, with a drawdown 4.51 to 6.62 points shallower than the truth. But the effect has no fixed size and not even a fixed direction: on a leveraged, concentrated version of the same strategy it ran backwards, costing 2.68 points. Survivorship bias is real and it is not the reason your backtest and your account disagree.
Granted, not measured: that survivorship bias exists, that it usually flatters, and that every practitioner warning you about it is right to.
Tested: how much it is worth, in CAGR points and drawdown points, on one strategy we built ourselves and had already believed.
The survivorship verdict-flip vs spy holds at >= 3 of 4 basket widths.
It held at 1 of 4. The prediction was falsified. The bias moved the number at every width and moved the verdict at only one, which is a smaller claim than the one we registered — and the one we are left with.
| Parameter | Locked value |
|---|---|
| strategy | 12-1 momentum: 252-day return, 5-day skip, monthly re-rank, equal-weight daily hold of the prior day's top-N |
| universe | point-in-time top-500-liquid US common stocks (min $1M dollar-volume, $5 price floor) |
| window | 2003-09-10 to 2026-06-22 — 5,731 trading days, 262 rebalance months, 21.7 effective years |
| widths swept | 5, 9, 15, 25 |
| the one variable | whether the universe keeps names that later delisted, or is intersected with today's active flag |
| frictions | none — no stop, no leverage, no transaction cost |
| # | Pre-registered goalpost | Measured | Pass |
|---|---|---|---|
| Pre-registered goalpost 1 — the verdict flip holds at ≥3 of 4 widths | 1 of 4 | FALSIFIED | |
| Pre-registered goalpost 2 — the survivor-only arm understates drawdown | held on the plain basket, reversed on the inherited arm | FALSIFIED | |
| Pre-registered goalpost 3 — delisting attrition falls monotonically as the as-of date approaches today | 38.2% → 32.4% → 25.4% | CONFIRMED | |
| Pre-registered goalpost 4 — the census reproduces from the store | at least 6,529 of 11,827 | CONFIRMED |
3 of 6 pre-registered predictions across both labs were wrong — and on this lab's own four, two were. Both counts are true; they have different denominators, and a count without its denominator is the defect this channel exists to catch.
| Figure | Value | What it means |
|---|---|---|
| Plain basket, 9 names — survivorship-free | 7.49%/yr | |
| Same basket — survivor-only universe | 9.58%/yr | |
| What deleting the dead was worth | +2.09 pts/yr | |
| Range across all 4 widths | +1.16 to +2.49 pts/yr | |
| Drawdown, made to look shallower by | 4.51 to 6.62 pts | |
| Inherited leveraged arm — the same bias, backwards | -2.68 pts/yr | |
| SPY over the same window | 9.07%/yr, -56.47% drawdown |
The sweep across basket widths is the stress test: the same question asked four times at 5, 9, 15, 25 names. The sign of the effect survived all four. Its size did not — it moves by a factor of roughly two across widths, which is why this report gives a range and never a single number.
Both arms reproduced exactly on re-run (CAGR and drawdown deltas both 0.0), in 82.1s of a registered 2700s budget.
SPY, identical in all 8 cells: $65,896.78 from $10,000, 9.07%/yr, -56.47% max drawdown. It is the same instrument in every row, so any difference between rows is the basket, not the market.
The ghost bound. Dead names occupy 0.000% to 0.724% of basket-day slots. That is a bound on how much of the effect can be mechanical, not a second measurement of the effect.
The best objection is that our survivorship-free arm is itself flattered. It is. When a held name's data ends, its return goes NaN and is dropped — there is no force-exit at a loss and no cost charged. So the true gap between the two universes is at least what we report, never exactly it. Every delta on this page is a floor.
The second objection is that active tracks symbols, not companies. Also true. A ticker change, a merger and a bankruptcy all read identically in this flag — UTX, which merged, sits beside LEH, which did not. So the honest phrase is symbols that stopped, not companies that went to zero.
There is no practitioner under test here. The strategy is ours, we built it, and we had already believed it — which is the only reason we can open every drawer of it for you. The claim on trial was our own.
sim_hold returns endpoints — a final value, a CAGR and a max drawdown per cell — not a row per fill. So what follows is every cell the experiment produced, 4 widths × 2 universes = 8 cells, and there is no per-trade file behind it to publish. Saying so is cheaper than implying otherwise. Check our arithmetic: the CSV below is the same file the figures above are computed from.| # | width | universe | final $ | CAGR | max DD | beats SPY | dead names held | ghost-days | basket-day slots | ghost share |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 5 | survivorship-free | $10,085.13 | 0.04% | -96.76% | no | 148 | 165 | 27,320 | 0.604% |
| 2 | 5 | survivor-only | $17,219.86 | 2.53% | -91.16% | no | 0 | 0 | 27,320 | 0.000% |
| 3 | 9 | survivorship-free | $47,995.13 | 7.49% | -89.43% | no | 233 | 356 | 49,176 | 0.724% |
| 4 | 9 | survivor-only | $72,839.31 | 9.58% | -82.81% | yes | 0 | 0 | 49,176 | 0.000% |
| 5 | 15 | survivorship-free | $172,550.64 | 14.02% | -72.62% | yes | 327 | 522 | 81,960 | 0.637% |
| 6 | 15 | survivor-only | $215,334.37 | 15.18% | -68.11% | yes | 0 | 0 | 81,960 | 0.000% |
| 7 | 25 | survivorship-free | $282,940.19 | 16.64% | -65.58% | yes | 461 | 779 | 136,600 | 0.570% |
| 8 | 25 | survivor-only | $381,926.78 | 18.26% | -60.12% | yes | 0 | 0 | 136,600 | 0.000% |
Survivorship bias is real, and it is smaller than its reputation. On this strategy it is worth +1.16 to +2.49 CAGR points a year — enough to matter, nowhere near enough to explain why a backtest and a live account disagree.
The direction is not a law. We found a case where it reverses, on a leveraged version of our own strategy. Anyone quoting you a single number for survivorship bias is quoting a number that depends on what they were running.
And the check is cheap. Ask what list the backtest was allowed to buy from. If the answer is today's list, you already know which way the number leans.
Everything we make is free — the reports, the ledgers, all of it. The only thing that costs us real money is the market data.