How to Avoid Overfitting When
Building a Gold Trading Bot

Start by articulating a mechanical reason why the strategy should work in the market — before writing a single line of code or touching a single parameter. For XAUUSD specifically: "London session opens create increased volatility as European institutional flow enters the market. Range breakouts during the first 30 minutes of the London open have historically produced directional moves because of concentrated liquidity arriving simultaneously." That is a structural hypothesis. Parameters (how far the breakout must extend, what SL distance, which EMA to use as trend filter) are then set to operationalise the hypothesis — not to maximise historical profit. The rule: if you cannot explain why the strategy should work without referencing historical data, you are curve-fitting from the start.

The minimum credible backtest for a XAUUSD EA covers at least 3 years and spans at least two distinct volatility regimes. The mandatory inclusion years: 2020 (COVID panic — extraordinary spike volatility), 2022 (aggressive Fed tightening — consistent trending with strong directional bias), 2023–2024 (post-tightening consolidation — range conditions). An EA that performs well in only one of these regimes is not robust — it is optimised for that regime. Test data should cover at least 500+ trades to provide statistical significance for win rate and expectancy calculations.

Reserve at least 30% of your total data set from the optimisation process. If you have 5 years of data, optimise on 3.5 years and hold out 1.5 years completely untouched. Run the final parameter set on the holdout data without any further optimisation. The holdout result is the only honest assessment of the EA's performance outside its training window. If the holdout result is within 30% of the in-sample result, the EA shows signs of robustness. If the holdout result shows a significant drop or losses, the EA was overfit to the training data regardless of how good the backtest looks.

Walk-forward testing is the gold standard for preventing overfit in EA development. Divide your 3+ years of data into rolling windows: optimise on window 1, test on window 2 (no optimisation). Optimise on windows 1+2, test on window 3. Continue for all windows. A robust EA passes 75%+ of forward windows (profitable in at least 3 of 4). If the EA passes backtest optimisation beautifully but fails every forward window, the strategy has no generalisable edge — it was pattern-matched to the historical sequence. Walk-forward testing in MT5: Strategy Tester → Enable Walk Forward → set In-Sample percentage to 70%.

Each optimised parameter needs sufficient data to validate. The rule of thumb: 1 parameter requires approximately 100–200 trades of data to validate statistically. At 8 parameters and 1,000+ trades, you have minimal statistical validation per parameter. Above 8 optimised parameters with the same data, you are borrowing statistical confidence from one parameter's data to validate another — a form of overfitting by committee. The discipline: list every parameter you plan to optimise. For each one, ask: "Is this value constrained by structural logic or am I optimising it freely?" If the answer is "freely," consider whether the parameter can be derived from structural logic instead.

Any gold EA must be tested on 2020 COVID crash data (March 2020: gold dropped 200+ USD in one week before reversing sharply) and 2022 FOMC surprise cycle data (gold fell from $2,000+ to $1,600 over 9 months on aggressive rate hikes). If the EA catastrophically fails on either of these data periods, the strategy is fragile. "Catastrophically fails" means: stops triggered at distances far outside their intended range, drawdowns exceeding 40%+ in those periods, or consistent loss across all stress months. Some drawdown during stress periods is acceptable — the strategy should survive and recover, not excel.