Price Action Lab Blog

Premium Market Analysis

DLPAL software

Performance Mean-Reversion of Machine Designed Strategies

Performance mean-reversion is common in the case of machine designed strategies.

Strategies suggested by the data often fail in real trading even after passing all validation tests. One reason for that is that validation is a weak method when strategies are developed via multiple comparisons. Some convoluted schemes have been proposed to adjust results for the impact of data-mining bias but the side effect is high Type-II error, or missed discoveries. After all, the proposed solutions do not deal with the root of the problem but only act as a filter of bad strategies.

When we started developing the first machine learning algos for design of trading strategies in the late 90s we knew we had to deal with this problem effectively. Since we have improved our methodology significantly. Apparently, both academia and practitioners are still using the wrong remedies. Our solution is proprietary and part of what constitutes the search engine of DLPAL software. Despite the recent changes in market dynamic, the strategies developed by DLPAL tend to be more robust than those developed by other engines.

Below we include two examples of how performance decays. Our metric is the Profit Factor PF. If PF drops to 1, then this means the strategy is breaking even. Normally good strategies need PF > 1.2 but performance also depends on payoff ratio and win rate.

Market: SPY ETF
In-sample: 01/29/`1993 – 12/31/2012
Out-of-sample: 01/02/2013 – 07/19/2019
Software for strategy identification: DLPAL S
Long-only strategies
Performance metric for mean-reversion: Profit Factor

Note that we will be looking only at long-only strategies for the following reasons: The out-of-sample coincides with a bull market in equities although there are minor corrections. Long strategies that fail in a bull market are worse than short that are expected to fail anyway. Therefore, short strategy failures cannot provide indication of robustness under given market conditions. If many long strategies fail, this probably means that the identification algorithm is not robust.

Case 1:  All positions: 3% profit target – 2% stop-loss

DLPAL S workspace

We select the Extended search and set the % Profitable for short to  > 100% to exclude short strategies. Minimum trades is set to 100 (Trades > 99). Below are the results.

Each line in the results is a strategy that satisfies the performance parameters specified on the search workspace.  Index and Index Date are used internally. Trade on is the entry point, in this case the Open of next bar. P is the success rate of the strategy, PF is the profit factor, Trades is the number of historical trades, CL is the maximum number of consecutive losers, Type is LONG for long strategies and SHORT for short strategies, Target is the profit target, Stop is the stop-loss and C indicates whether % or points for the exits, in this case it is %. Last Date and First Date are the last and first date in the data file.

The software identified seven strategies that fulfilled the criteria specified on the search workspace. Note that usually all strategies from the results are used to develop a final system but that also depends on methodology used.

All Profit Factor PF values are above 1.5 as expected by design.

Next we use Test Strategies tool to test the strategies in the out-of-sample. Below are the results.

It may be seen that no PF dropped below 1 although the value for the highlighted strategy is close to the break-even value. Although there was some mean-reversion there was also improvement in the case of two strategies whose PF increased in the out-of-sample. Overall these results indicate that the search engine is robust. But due to the low number of strategies the conclusions may not be significant. Below we look at an example with larger sample of strategies.

Case 12:  All positions: 2% profit target – 4% stop-loss

DLPAL S workspace

As in previous Case 1, we select the Extended search and set the % Profitable for short to  > 100% to exclude short strategies. Minimum trades is set to 100 (Trades > 99). Below are the results.

Each line in the results is a strategy that satisfies the performance parameters specified on the search workspace.  Index and Index Date are used internally. Trade on is the entry point, in this case the Open of next bar. P is the success rate of the strategy, PF is the profit factor, Trades is the number of historical trades, CL is the maximum number of consecutive losers, Type is LONG for long strategies and SHORT for short strategies, Target is the profit target, Stop is the stop-loss and C indicates whether % or points for the exits, in this case it is %. Last Date and First Date are the last and first date in the data file.

In this case the software identified 21 strategies that fulfilled the criteria specified on the search workspace. All Profit Factor PF values are above 1.5 as expected by design.

Next we use Test Strategies tool to test the strategies in the out-of-sample. Below are the results.

Only three strategies have PF < 1 in the out-of-sample as marked on results but only the highlighted one actually broke down while and the other two are close to break-even. This is a highly satisfactory result.

Conclusion

Although our specific tests indicated that the strategy identification algorithm is robust, developing final strategies for trading is not trivial and requires effort and often innovation to avoid common pitfalls. Anyone who thinks they can buy software, run it and it will generate a strategy that will make them good money is probable naive or even misguided and these apply also to anyone who makes these claims about software. Tools require skilled users for significant results.

You can request a free demo of DLPAL S  here. For more articles about DLPAL S click here.


If you have any questions or comments, happy to connect on Twitter: @priceactionlab

Subscribe via RSS or Email, or follow us on Twitter.

Disclaimer:  No part of the analysis in this blog constitutes a trade recommendation. The past performance of any trading system or methodology is not necessarily indicative of future results. Read the full disclaimer here.

Copyright Notice

Leave a Reply

Theme by Anders Norén