Bootstrap Tests of the Predictive Power of the Patterns Discovered by Price Action Lab

A question that is often asked is whether the price patterns discovered by Price Action Lab have real predictive power or their performance is random. This is an important and valid question. In this study we will consider three patterns from the search results for QQQQ ETF for 3% profit target and 3% stop loss and we will run tests to determine their significance. The search period of the results was from 01/03/2000 to 11/10/2010. Price Action Lab discovered 121 patterns with Profit Factor > 1.5  as shown in the figure below. The patterns are sorted according to their number of trades.

Each line in the results corresponds to a price pattern that satisfies the performance parameters specified by the user.  P is success rate of the pattern, PF is the profit factor, Trades is the number of historical trades, CL is the maximum number of consecutive losers, Type is LONG for long patterns and SHORT for short patterns , Target is the profit target,  Stop is the stop-loss and C indicates whether % or points for the exits. Last Date and First Date are the last and first historical data file.

Next, we backtest the first pattern on the list by right-clicking on that line and then selecting backtest. This is a pattern that generated 97 short positions (this number includes an open position). The following is a screenshot of the output of the backtest:

It may be seen that the profit factor of this pattern is equal to 1.60 and the average win to average loss ratio is equal to 1.03. Next, we copy and past the results in Excel and we keep only the far right column that lists the P/L of each trade. We then output this column to a .txt file, which is available for review and download by clicking here. The file is made available so that the readers can run their own tests should they wish so. Next, a bootstrap test is run using the returns in the text file. The aim of the test if to determine whether this particular pattern has any predictive power.

The p-value calculated by the bootstrap test is  0.013205. This is much less than the threshold value of 0.05 for statistical significance. This value can be interpreted to mean that there is moderate to strong evidence against randomness in the data tested.

We next repeat the analysis for the second pattern in the list, which is a long pattern with 87 trades and profit factor equal to 1.68 . The text file of the returns can be downloaded by clicking here. The p-value calculated by the bootstrap test in this case is  0.01054. This value is also much less than 0.05, the threshold value, and this means that there is moderate to strong evidence against randomness in the data tested.

Not all patterns fair as well as the first two we considered. For example, the first of the last two patterns in the list (not shown on the screenshot), a long patterns with 30 trades and profit factor equal to 1.68 (less than that of the first pattern considered and equal to that of the second) has returns that when analyzed with the bootstrap test generate a p-value of  0.07301, which means that there is no evidence against the hypothesis that its performance is due to luck and it is highly probable that it has very low or no predictive power.

Advanced users of Price Action Lab may use the bootstrap test or any other statistical test of significance to assist them in selecting which patterns have higher probability of remaining profitable in the future. The power of Price Action Lab is in its ability to discover a large number of candidates for the such tests quickly and efficiently.