10 Common Backtesting Mistakes to Avoid in Prop Trading

Backtesting is a vital process for any forex prop trader who wants to develop and improve his trading system. It allows you to test your ideas and assumptions on historical data and see how they would have performed in the past.

It can help you identify the strengths and weaknesses of your trading idea, optimize your parameters, and increase confidence in your trading decisions.

However, backtesting is not a foolproof method. There are many pitfalls and mistakes that can lead to inaccurate or misleading results and ultimately affect your trading performance.

In this blog post, we will discuss ten of the most common backtesting mistakes that some prop traders make and how you can avoid them:

1. You don’t take enough trades when testing your system

One major mistake that some prop traders make is not taking enough trades when backtesting. They experiment with just a few trades and conclude they have a solid system.

This is not ideal. It results in a lack of statistical significance, reliability, and robustness of the system.

If you only test your system on a small sample of data, you may not capture the full range of market conditions, scenarios, and events that can affect your system. You may also overfit your system to the specific data that you used and fail to generalize to other data sets.

To avoid this mistake, you should test your system on a large and diverse sample of data that covers different market phases, cycles, and trends.

You should also test your system on different markets, timeframes, and instruments to see how it performs in different environments. This way, you can tell if the system will be “reliable” or not.

2. You quit your system when the test results are poor or not what you expected from a few initial trades

Another mistake that some prop traders make is to quit when the results aren’t immediately great. Backtesting is a trial-and-error process. It is unlikely that you will find a profitable system on your first try.

It takes time, patience, and persistence to test, tweak, and improve your system and to find the optimal settings and parameters for it.

Therefore, you should not give up on your system too soon or be discouraged by the initial results. Instead, you should analyze the results carefully and identify the areas that need improvement.

You should also compare the results with your expectations and see if they are realistic or not.

3. You don’t have a written plan

One of the most important steps in backtesting is to have a written plan. A written plan defines the objectives, rules, and parameters of your system and serves as a guide for your testing process.

It helps you to stay consistent, focused, and disciplined and avoid making arbitrary or emotional decisions.

4. You don’t account for your emotional state

Not accounting for your emotional state in backtesting is another big miss. Backtesting is not the same as live trading because it does not involve the same level of stress, pressure, and emotions that live trading does. When you backtest, you do not deal with the fear, greed, doubt, and excitement that live trading entails.

However, your emotional state can have a significant impact on your trading performance, and it can affect your ability to follow your system, manage your risk, and execute your trades. Therefore, you should not ignore your emotions when backtesting, but rather try to simulate them as much as possible.

One way to do this is to test your system under optimal conditions, when you are calm, focused, and confident.

Another way is to test your system under suboptimal conditions when you are tired, distracted, or stressed.

This will help you to see how your system performs in different emotional states and how you can cope with them.

5. You change or tweak your system in the middle of backtesting

This is adding, removing, or modifying the rules, indicators, or parameters of your system based on recent results or performance. It is a form of curve fitting, and it will contaminate your data and invalidate your results.

Curve fitting is the process of creating a system that fits the data perfectly but does not work well in the future or on other data sets. It is a form of overoptimization, and it can lead to false confidence, unrealistic expectations, or poor performance.

To prevent this mistake, you should not change your system in the middle of a test, but rather test each system separately and compare the results.

You should also use the split-sample method, where you divide your data into two parts: an in-sample part (where you develop and optimize your system) and an out-of-sample part (where you validate and verify your system).

This will help you avoid overfitting and test the robustness of your system.

6. You use bias to positively or negatively justify your system

A pre-set bias to prove or disprove your system can mess up the efficiency of your backtesting. This can happen due to confirmation, hindsight, or survivorship bias.

It can make you manipulate, ignore, or select the data, trades, or results that support or reject your hypothesis. And it makes you overlook or disregard the ones that contradict or challenge it.

To fix this situation, you want to have an objective and neutral attitude to test the system in backtesting and avoid a preset bias

You should test the system as it is, not as you want it to be. You should also use the scientific method and follow these steps:

• Formulate a clear and testable hypothesis
• Collect and analyze relevant and reliable data
• Evaluate and interpret the results and findings
• Draw and communicate the conclusions and implications
• Repeat and refine the process

7. You don’t do enough analysis after testing & you have a bad reporting system

Backtesting is not only about generating numbers and statistics but also about interpreting and understanding them. It is not enough to look at the win rate and return of your system. You must look at other metrics and factors that can affect your trading performance.

Some of the metrics and factors that you should analyze after testing are:

• The risk-reward ratio and the expectancy of your system
• The drawdown and the recovery factor of your system
• The Sharpe ratio and the Sortino ratio of your system
• The profit factor and the R-squared of your system
• The trade distribution and the equity curve of your system
• The sensitivity and the stability of your system

To do this analysis, you need to have a good reporting system that can generate and display these metrics and factors in a clear and comprehensive way.

A good reporting system can help you visualize, compare, and evaluate your results, and to identify the strengths and weaknesses of your system.

8. You only test on one fx, stock, or commodity pair and assume it will work in all pairs or markets

Another serious mistake that traders make is to only test their system on one market or instrument and assume that it will work in all markets or instruments. This is a form of generalization that can lead to poor performance.

Different markets and instruments have different characteristics, dynamics, and behaviors. They may not respond in the same way to the same system or strategy.

Therefore, you should not test your system on only one market or instrument. Test it on multiple markets or instruments, and see how it performs in different environments. You should also customize your system to fit each market or instrument and adjust your parameters, rules, and indicators accordingly.

This will help you increase your adaptability. It will also help you to exploit the opportunities and advantages of each market or instrument.

9. You overoptimize your system by adding more conditions or indicators

Some prop traders may overoptimize their system by adding more indicators or conditions as a result of perfectionism, complexity, or overconfidence.

This can also lead to overfitting, curve fitting, or data mining. It can make your system too complicated, fragile, or specific. To avoid this mistake, you should try to keep your system simple and robust.

You should not add more indicators or conditions than necessary, and only use the ones that have a clear and logical rationale and purpose. You should also use the principle of parsimony, or Occam’s razor, which states that the simplest explanation or solution is usually the best.

You should also use cross-validation, walk-forward testing, or Monte Carlo simulation, to test the robustness and stability of your system.

10. You believe that live trading results will be exactly the same (100%) as your backtesting results

It is misleading to think or believe that live results will be exactly the same as backtesting results. This can lead to disappointment, frustration, and failure.

Backtesting is not a guaranteed image of future performance, but rather a simulation of past performance. It does not account for all the variables, uncertainties, and changes that can occur in the real market.

Some of the factors that can affect live trading results are:

• Slippage and commissions

• Market liquidity and volatility

• Data errors and gaps

• Technical issues and glitches

• Human errors and emotions

In conclusion, avoiding these common backtesting mistakes will help you build a solid and more dependable system for prop trading success. It will help you backtest your strategy efficiently.