Quant Trading Academy, Module 1, Part 2

Backtesting is powerful and easy to misuse. You do not need to build backtests, but you do need to understand what they measure and how to interpret them.

What Backtesting Is

A backtest applies trading rules to historical data and measures what would have happened if they had been followed. A backtest is a simulation of the past, not a record of actual trades.

It answers: "If this strategy had been applied to this data, what results would it have produced?" It does not directly answer: "Will it make money going forward?"

The Three Enemies of a Reliable Backtest

Overfitting
Overfitting happens when a strategy is tuned so precisely to past data that it memorises history instead of finding a real pattern. Test enough variations and one version will often look exceptional by chance. The antidote is simplicity and robustness.

Look Ahead Bias
Look ahead bias happens when the backtest uses information that would not have been available at the decision point. If a signal uses a candle close and enters at that same close, it is cheating. You only know the close after the candle ends.

Data Snooping Bias
Data snooping happens when the same dataset is reused until good results appear. If you test many ideas and show only the winners, those winners may just fit quirks in that sample. Many published factors fail out of sample for this reason.

The Gold Standard: Out of Sample Testing

The best defence is out of sample testing: build the strategy on one period, then test it on a separate unseen period.

If a strategy is built on 2018 to 2022 data and works on 2023 to 2024 data, that second result is meaningful. A stronger method is walk forward testing, where the strategy is repeatedly re-optimised on rolling windows and tested on the next unseen window. Strategies that survive both are more likely to reflect edges.

The Honest Limitations of Any Backtest

Even a backtest has limits. Costs are easy to underestimate. Slippage, spreads, and poor fills matter. Market impact is invisible: larger positions can move price against you. Historical data has gaps and anomalies. Markets evolve. Participants, structure, and regulation change. Backtests are useful, but not guarantees.

What Quant's Backtests Are Telling You

Quant's setups come from systematic backtesting designed to reduce these failure modes. A +EV label reflects a pattern with positive expectation across historical data, not just a configuration tuned until it looked good.

The backtest is a calibrated prior, not a certainty. Under similar conditions, the setup has historically produced positive outcomes over a meaningful sample. Live performance will differ. Individual setups will lose. A bad stretch is not immediate proof of decay. The real question is whether it still looks like normal variance or whether conditions have changed systematically.

When to Question a Signal

Do not abandon a signal just because it has had a bad run. But following any signal forever, regardless of evidence, is just as wrong.

Threshold should be statistical, not emotional. Ask: has market structure changed, is underperformance materially worse than historical drawdowns, and is there a logical reason the signal may have stopped working?

If no, maintain discipline. If yes, do a systematic review, not a panic exit. Quant continuously monitors signal health.

Key Takeaways

A backtest shows what would have happened historically, not live results.
The main enemies are overfitting, look ahead bias, and data snooping bias.
Out of sample testing is the best validation tool.
All backtests have limits: costs, market impact, imperfect data, and market evolution.
Quant's +EV signals are meaningful and rigorous, but not guarantees.
Question a signal systematically, not emotionally.