From backtest to decision
A backtest describes the past, not the future. Learn what a good result actually proves, what it cannot promise, and how to decide what to do next. Education only, not advice.
Part of the Backtesting and Research track on Agenticks. About 10 minutes, written for a advanced reader.
You ran the test. The equity curve climbs, the numbers look healthy, and now there is a decision in front of you: do nothing, keep researching, paper trade, or risk real money. This lesson is about reading a result honestly so that decision is grounded in what a backtest can actually tell you. Here is the uncomfortable truth that good researchers make peace with early. A backtest describes the past and only the past. It replays a fixed set of rules over history that already happened. That is genuinely useful, but it is not a forecast, and treating it like one is how careful work turns into overconfidence.
A backtest is a description, not a prediction
The result tells you how your rules would have behaved on data that already exists. It does not tell you how they will behave on data that has not happened yet. Everything you decide next should respect that gap.
So what does a good result genuinely give you? A few real things. It tells you the idea has structure: across many trades, the rules did something other than break even minus costs. It gives you a sample size large enough to start trusting the numbers, assuming you actually have one. It shows you the maximum drawdown, the worst peak-to-trough drop the strategy put you through, which is the pain you would have had to sit through to collect those returns. And it shows you the shape of the ride through the equity curve, not just the destination. Those are facts about the past. They are worth a lot. They are also the ceiling of what the test can honestly promise.
What a backtest does not tell you matters just as much. It does not tell you the future will rhyme with the history you tested. Markets shift: volatility regimes change, a behavior that paid for three years can quietly stop. It does not tell you whether your great-looking curve came from a real edge or from luck and overfitting, where the rules were tuned so tightly to old data that they memorized noise. And unless you modeled them, it does not include the costs that show up live: slippage, fees, partial fills, and the very human urge to override the rules at the worst moment. A result can be true about the past and still be a bad basis for a live decision. Both can hold at once.
Aim for calibrated confidence, not certainty
The right output of a backtest is not yes or no. It is a more or less believable case. A big sample, a survivable drawdown, and a result that holds on data you did not tune on all push the case up. A tiny sample, a brutal drawdown, or a curve that only shines on the exact window you built it on push it down.
So how do you actually turn a result into a next step? Think of it as a ladder, not a light switch. First, sanity check the sample and the costs. If there are too few trades, or you never modeled fees and slippage, you do not have a decision to make yet, you have more work to do. Next, see whether the edge survives data the strategy never saw, out-of-sample and ideally forward testing on fresh data as it arrives. Forward testing is the fairest check there is, because that data did not exist when you wrote the rules. Only after that does paper trading or small live size make sense, and even then you are still gathering evidence, not collecting a guarantee. The goal of every rung is the same: replace hope with one more piece of real evidence before more money is on the line.
A strategy shows a strong, profitable equity curve over five years of history. What is the most honest thing this result proves? How those exact rules behaved on that exact past data That is precisely what a backtest measures. It is a faithful description of the past, which is useful, but it stops there.
- The rules had structure on past data
- A measured fact about the history you tested
- The worst drawdown you would have endured
- The deepest drop the test put you through
- Next year will look like the test period
- A claim about the future the test cannot support
- The edge is real, not luck or overfitting
- A claim one in-sample run cannot prove alone
Put the steps in the order that moves a good-looking backtest toward a real decision.
- Check the sample size and whether costs were modeled
- Test the same rules on data they were never tuned on
- Forward test on fresh data as it arrives
- Only then consider paper trading or small live size
past confidence future
You can read a result without fooling yourself
You know what a good backtest does and does not prove, and how to turn it into a next step instead of a leap of faith.
Common questions
- Does a good backtest mean the strategy will work in the future?
- No. A backtest describes how rules behaved on past data. It can show structure and a plausible edge, but it cannot promise that the future will look like the past. The honest read is a probability, not a guarantee.
- What is the single best thing to do after a backtest looks good?
- Test the same rules on data the strategy has never seen, ideally going forward in real time. Forward testing on new data, on paper or small size, is the fairest check there is because that data did not exist when you built the rules.
- If the backtest is profitable, why not just go live?
- Because a single profitable curve can come from luck, overfitting, or costs the test ignored. Live trading adds slippage, fees, and emotion. A sensible decision weighs the whole picture and the worst case, not just the final number.
Terms defined in this lesson
Continue
Sources