Sample size and why ten trades mean nothing
Why a small sample of trades lies, how luck dominates tiny backtests, and roughly how many trades you actually need before win rate and expectancy mean something.
Part of the Backtesting and Research track on Agenticks. About 9 minutes, written for a advanced reader.
You run a quick backtest, see eight wins out of ten trades, and feel like you found something. Stop there for a second. Ten trades is not a result. It is a rumor. The problem is sample size, the number of trades behind a number. When the sample is tiny, luck does almost all the talking. A strategy with no real edge and a strategy with a genuine one can produce the exact same ten-trade run, and from those ten trades alone you cannot tell which is which.
Small samples are dominated by luck
Flip a fair coin ten times and getting eight heads is not rare, it happens often. Now imagine that coin is your strategy. Eight wins out of ten can come from a real edge or from a lucky streak, and ten flips is nowhere near enough for the randomness to cancel out. The smaller the sample, the more the result is just noise wearing a costume.
Here is why a small sample lies. Every strategy produces a spread of possible outcomes, not one fixed number. Over many trades that spread tightens and the true tendency shows through. Over a few trades the spread is enormous, so a single short run can land almost anywhere inside it. That means a tiny sample can flatter a bad idea and bury a good one. A losing strategy can win its first ten trades. A winning strategy can lose its first ten. Neither short run is evidence, because backtesting on a thin slice of history measures luck at least as much as it measures the rules.
More trades shrink the noise
As the number of trades grows, the random ups and downs start to average out and the real signal, if there is one, has room to appear. This is why a 60 percent win rate over 12 trades is meaningless and the same 60 percent over 600 trades is worth a careful look. The rate did not change. The amount of luck baked into it did.
A strategy wins 8 of its first 10 trades. What can you honestly conclude from that alone? Almost nothing, because 10 trades is far too small a sample to separate a real edge from luck Right. A run that good happens by chance often enough that 10 trades cannot tell a real edge from a lucky streak. You need many more trades before the win rate means anything.
So how many trades do you actually need? There is no single magic number, and anyone who hands you one is overselling. The honest answer is that it depends on two things: how big the edge is, and how noisy the results are. A large, obvious edge shows through in fewer trades. A small edge buried in violent swings can need thousands of trades before it separates from the noise, and some never do. As a rough habit, a few dozen trades is barely a hint, a few hundred starts to be worth reading, and even a large sample only lowers the odds you were fooled. It never removes them. This is also where a Monte Carlo simulation earns its keep. By reshuffling the same trades many times, it shows the full range of curves luck could have produced from your sample. A wide range over few trades is the math telling you, plainly, that you do not have enough data yet.
How many trades depends on the edge
Think of it as a ratio. The bigger the edge relative to the swing in results, the fewer trades you need to see it. A strong, steady edge can reveal itself in a few hundred trades. A faint edge inside wild variance can need many thousands, which is exactly why a small-sample backtest of a marginal idea is close to worthless.
- Sample size
- How many trades sit behind a result
- Win rate
- The share of trades that ended profitable
- Expectancy
- Average win or loss per trade over many trades
- Monte Carlo
- Reshuffles trades to show the range luck allows
Sample size also explains a trap you have probably already fallen into: cherry-picking. If you scroll a chart, find ten clean-looking setups, and count the result, you did not measure an edge. You measured your own ability to spot good-looking spots after the fact. Hindsight makes every chart look obvious, and ten hand-picked examples are a tiny, biased sample on top of an already tiny one. The fix is the same as the cure for small samples. Write the rule clearly, let it run across the whole history, and take every trade the rule produces, not just the pretty ones. Then look at how many trades there actually were, because that count decides how much the expectancy and win rate are worth. A clean equity curve drawn from forty trades is a sketch; the same curve drawn from four hundred is closer to a measurement.
Put these steps in the order that protects you from a small-sample illusion.
- Write the entry and exit as a fixed rule
- Run the rule across the full available history
- Count how many trades the rule produced
- Trust the metrics only if the sample is large enough
sample luck win rate trades
You can now smell a thin sample
You know why ten trades lie, why luck dominates a small sample, and why the trade count, not the win rate, decides how much a backtest is worth.
Common questions
- How many trades do I need before a backtest means anything?
- There is no single magic number, but a handful of trades tells you almost nothing. Most people start to take a result more seriously somewhere past a few hundred trades, and even then only with caution. The right number depends on how big the edge is and how noisy the results are: small edges and wild swings need far more trades to show through.
- Why can ten trades look amazing and still mean nothing?
- With ten trades, luck dominates. A coin can land heads eight times out of ten by pure chance, and a strategy with no real edge can string together a great short run the same way. A small sample does not have enough trades for randomness to average out, so a fluke and a real edge look identical.
- Does a bigger sample guarantee the strategy will work?
- No. A larger sample makes win rate, expectancy, and drawdown more trustworthy, but it never makes them certain. The past is still the past. A big honest sample lowers the odds you were fooled by luck; it does not promise the future will match it.
Terms defined in this lesson
Continue
Sources