--- title: "What a backtest cannot tell you" description: "An equity curve answers one question accurately. Here are the five things it quietly assumed on your behalf, and how to read the number it prints." date: 2026-09-06 updated: 2026-09-19 tags: [backtesting, basics] author: "Allen Yan" image: https://sweetvaluelab.com/blog/img/7322610f4944b91c.png lang: en canonical: https://sweetvaluelab.com/blog/what-a-backtest-cannot-tell-you --- A backtest answers one question accurately: for this fixed set of holdings, over a window that has already finished, how far down did it go and what was left at the end. It answers that well. The trouble starts afterwards. The number prints, and a second sentence attaches itself to it: so the strategy works. Then a third: so it will keep working. Then a fourth: so I would have sat through that. None of those three are in the output. Here are the five things that curve assumed on your behalf. ![An equity curve with its drawdown shaded, above a strip of the five assumptions the curve depends on](https://sweetvaluelab.com/blog/img/4ae70ad1f81258de.svg "Schematic. The curve records one thing: what this basket did between these two dates. The −34% drawdown it walked through lasted eighteen months, and the five numbered items are what the line quietly took for granted.") ## 1. It assumes you held on Every backtest walks through its drawdowns and assumes you walked through them too: a third of the money gone over eighteen months, with everyone around you explaining why this time is different. The equity curve has no field for the month you capitulated, and that month is usually the whole distance between the printed result and the one you got. So read the depth of the worst drawdown, but read its length harder. The S&P 500 peaked in October 2007 and took five and a half years to reach that level again. A CAGR will not tell you that. It is one number. ## 2. It assumes the window was representative A decade ending today contains one particular level of rates and one particular rotation in style. Drop the same holdings into 2000 to 2010 and plenty of strategies invert. The honest reading of a strong result is "this survived this regime". What happens in a different one is not in the window. The defence is cheap: run the same basket over several start dates and several lengths. If the conclusion only holds in one of them, it is a statement about that stretch of market. ## 3. It assumes you picked without hindsight This is the dangerous one, and it is completely invisible in the output. Select today's winners and test them backwards, and the selection already contains the answer. "These ten stocks did well over ten years" is true and useless. "A rule I could have written down in 2015 would have picked them" is a claim you can check. The whole problem sits between those two sentences. Funds work the same way. What you can backtest is what survived. The mediocre ones closed or merged years ago, and their records went with them. ## 4. It assumes frictions are zero Backtests here are gross: no commissions, no bid-ask spread, no dividend withholding tax, and no capital gains tax on what rebalancing sells. In a taxable account the tax alone can eat a meaningful part of an income strategy's edge, and it lands every year instead of waiting until the end. The more a strategy turns over, the more the paper number flatters it. Buy and hold loses almost nothing. Monthly rebalancing loses a great deal. ## 5. It assumes the dates were not luck Start and end dates are leverage on their own. Shift a ten-year window six months across a crash and the one that came out ahead can swap places with the one it beat. A conclusion that depends on a specific start date is a fact about that date. ## So what is it good for? Quite a lot, as long as you read it as description: - **Sizing the risk you are signing up for.** Worst drawdown, how long it lasted, volatility. Those hold up across regimes far better than returns do. - **Comparing structures.** [SPY vs QQQ](https://sweetvaluelab.com/tools/portfolio-backtest/compare/spy-vs-qqq) is a question about concentration and sector weight, the same concentration [market breadth](https://sweetvaluelab.com/blog/how-to-read-market-breadth) counts one day at a time, and both were rules that existed before the window opened. - **Seeing what the dividends did.** The distance between taking the cash and reinvesting it is arithmetic about the past (see [total return vs price return](https://sweetvaluelab.com/blog/total-return-vs-price-return)). - **Killing an idea.** A strategy that fails in-sample will not be rescued out of sample. ## How the one here is computed Everything the [portfolio backtest](https://sweetvaluelab.com/tools/portfolio-backtest) computes is history: split-adjusted daily prices and the dividends that were really paid, with no model in between. The basket is marked to market every trading day, 252 observations a year instead of 12, so a drawdown is what happened between month-ends and not what the month-end pair suggests. Each dividend is applied as cash on its ex-date at the matching split-adjusted per-share amount. A dividend-adjusted price series is never used, because that counts the income twice. With reinvestment on, a dividend buys fractional shares at that day's close; with it off, the cash sits idle. Weights set the opening trade and then drift, which is what a buy-and-hold holder lived through. No rebalancing, no contributions, no taxes, no commissions. The window is the overlap of every holding's price history, and the tool shows the start date it actually used, plainly, when a holding's history is shorter than the window you asked for. That silent, always-flattering kind of error is usually how it gets in. A dividend cut is reported only when both the annual total per share and the average payment per ex-date fell, so a payment that merely moved across a year boundary stays out of the cut column. The [FIRE calculator](https://sweetvaluelab.com/tools/fire-calculator) is a different animal. It is a Monte Carlo simulation that runs entirely in your browser: monthly compounding, each month's return drawn independently around the mean you pick with annualized volatility scaled by the square root of twelve, and spending stated in today's money and inflated inside the simulation. Use it to work out what you would have had to sit through. Whether the next decade rhymes is a separate question.