Price Action Lab Blog

Premium Market Analysis

Trader education, Trading Strategies

When Performance Is too Good to Be True

Look-ahead bias is common  in backtests by novice strategy developers but results affected by it are frequently displayed in social media and financial blogosphere.

I wrote the article “Look-Ahead Bias and How to Detect it” three years ago after visiting the website of a quant hedge fund and realizing that they were actually marketing results affected by bias, as my analysis showed.

Look-ahead bias is a serious error. In a nutshell it is caused by entry signals that are affected by knowledge of future conditions. In real trading the future is unknown but the bias in backtests may result in unrealistic performance figures.

What is unrealistic? In my opinion, any backtest results with annual return in access of 10% and Sharpe ratio greater than 1 must be carefully scrutinized for the presence of bias. Below is a recent example from my own work. A few days ago, after programming a strategy (late at night) I obtained the following equity curve.

Annualized return is about 39% and Sharpe is about 3.2. I did not rush to post this in Twitter or in any other social media platform. The first thing that came to my mind was that something was wrong. Especially when equity curve starts going vertical, this is strong indication that something is wrong. Here are some things to watch for:

  • The equity curve is exceptionally smooth
  • Annualized return is >> 10
  • Sharpe ratio is >> 1
  • Equity curve shows periods of vertical rise

I looked at the code and in the beginning I could not find anything wrong. But after spending about two hours trying a few things I discovered the source of the look-ahead bias. Below is the actual equity curve.

Annualized return is about 4.5% and Sharpe is 0.41. This is more reasonable and what is expected from an “average intelligence’ hypothesis that one was.

I suspect that look-ahead bias has been used purposely to impress people. But the non-intentional aspect of it has probably affected some quant hedge funds that delegate strategy development work to programmers who are very good in what they do but have little experience with backtesting and strategy analysis.

It boils down to this:  If the backtest performance appears too good to be true, it is with high probability affected by some form of bias, such as:

  • Look-ahead
  • Data-snooping
  • Selection of survivors
  • Over-fitting

After all, markets are not very generous in rewarding backtests, as anyone with skin-in-the-game knows.

Contact us if you have any questions.


Disclaimer:  All performance results are based on backtests. No part of the analysis in this blog constitutes trade recommendation. The past performance of any trading system or methodology is not necessarily indicative of future results. Read the full disclaimer here.

All backtests were performed in Amibroker

Copyright Notice

Now,

Leave a Reply

Theme by Anders Norén