A backtest can answer whether a fixed rule would have worked on recorded historical data. It cannot prove that the rule was available at the time, that the prices were obtainable or that the same pattern will continue.
The biggest danger is not a coding error. It is repeatedly changing the system after seeing the result until historical noise looks like a discovery.
Define the hypothesis before the score
A useful hypothesis explains why the feature might matter and defines the eligible races, selection rule, stake and price basis. For example, a rule based on field size and an existing model rating is testable; choose whichever filters make the most profit is not a stable rule.
Record the definition. Small wording changes such as before non-runners versus final field can materially change which races qualify.
Separate discovery, validation and forward proof
Use an early period to explore and a later untouched period to validate the final rule. Do not shuffle racing rows randomly because yesterday and tomorrow are not interchangeable: models, markets and data coverage change over time.
After validation, freeze the rule and collect live forward selections. Forward proof is slower, but it exposes operational effects that a clean historical table misses.
Prevent future-data leakage
Only use information that existed when the bet would have been placed. A final SP, settled result, corrected rating or post-race comment cannot be an input to a pre-race selection. Leakage often produces spectacular results because the future has quietly entered the feature set.
Time-stamp source snapshots where possible. If a field's availability is uncertain, exclude it or mark the test as exploratory.
Model realistic execution
State whether returns use an advised price, SP or BSP and whether commission is deducted. Apply non-runner, void and dead-heat rules. If the strategy targets early prices, do not score every bet at a price that appeared briefly when the selection process could not have captured it.
Liquidity and stake limits also matter for large claimed returns. A level-stakes proof is a clean starting point, not evidence that any bankroll could scale without friction.
Measure robustness, not just the best total
Inspect results by year, course, race type and price band, but do not keep slicing until one subgroup looks perfect. A credible rule should have a plausible mechanism and should not collapse when one winner or one meeting is removed.
Report strike rate, LSP, ROI, maximum drawdown, longest losing run and the number of variants tried. The last number makes the cost of searching visible.
Know when to abandon the system
A failed out-of-sample result is useful information. Resist adding another filter immediately. First ask whether the original mechanism was wrong, the data changed or the discovery result was noise.
If you do revise the rule, it becomes a new hypothesis and needs a new untouched period. That discipline is less exciting than a perfect backtest and much more valuable.
Questions
Frequently asked questions
How many bets does a horse racing backtest need?
There is no magic count. More varied settled selections narrow uncertainty, while short or highly filtered samples remain fragile. Show the count and confidence around the result rather than declaring a universal threshold.
Can I optimise a system on all available history?
You can explore it, but then you have no untouched history left to test whether the rule generalises. Reserve a later period or move to a frozen forward test.
This article explains a method; it does not guarantee a return. Read the responsible gambling guidance and never chase a loss.