The complete process of discovering price patterns for SPY using Price Action Lab, testing them in out-of-sample and then generating a trading system is presented here. It is shown that Price Action Lab can find patterns that result in a statistically significant and profitable system in an extended out-of-sample period.
Trading SPY is an efficient and liquid market. Some have argued that if a market is efficient, then it does not allow profiting through the exploitation of price patterns. In support of their claim they have presented studies that quite often, in our opinion, are very basic and even naive. Here we present just one example of machine generated SPY price patterns that as a group has maintained an exceptionally large profit factor over a very long out-of-sample testing period.
The results presented here can be confirmed by anyone using a demo version of Price Action Lab. Actually, the demo version was used to generate the results of this study.
Step 0: Download non-adjusted daily SPY data from Yahoo and create two data files, one is the in-sample with prices ranging from 01/29/1993 to 12/31/2008 and the other will be used as the out-of-sample with prices ranging from 01/02/2009 to 04/04/2012. These files are saved at two different locations as SPY.txt.
Step 1: Create a target and stop file
In this example we use 2.5% for the profit-target and stop-loss. A T/S file is created with the values as shown above and then saved as 2p5.trs
Step 2: Create a search workspace
The workspace is created by selecting the following:
a) The T/S file we created by the name 2p5.trs
b) The in-sample SPY data file SPY.txt.
c) The trade parameters: % is marked to indicate that the values in the selected T/S file (2p5.trs) stand for percentages based on the open of the next bar, which is selected under Inputs. The Delay input is kept marked off. (Using delay inputs will be the subject of another post).
d) Under search parameters we input 70 for the percent minimum win rate for patterns that generate long signals and 68 for the percent minimum win rate for patterns generating short signals. We also input 1.30 for the minimum profit factor, 29 for the minimum number of trades and 10 for the maximum number of consecutive losers.
e) The date range in the data file is shown under File date Range. In this case it corresponds to the in-sample range. The Search Range is left to 500. This means that Price Action Lab will demand that all patterns that are found to satisfy the performance criteria set in (a) – (d) must have at least one historical trade in the most recent 500 bars. Finally, the Extended search option is checked and we run the workspace.
Step 3: In-sample results
Price Action Lab will run for an interval of time depending on computer CPU speed but in this particular case it will complete the search after about 15 minutes on the average. The output should look like the one below:
Each line in the above results corresponds to a price pattern that satisfies the performance parameters specified by the user. Index and Index Date are used internally to classify patterns. Trade on is the entry point, in this case the Open of next bar. P is the success rate of the pattern, PF is the profit factor, Trades is the number of historical trades, CL is the maximum number of consecutive losers, Type is LONG for long patterns and SHORT for short patterns , Target is the profit target, Stop is the stop-loss and C indicates whether % or points for the exits, in this case it is %. Last Date and First Date are the last and first date in the historical data file.
It may be seen from the results that Price Action Lab found 13 patterns, 10 long and 3 short, which satisfy the performance criteria specified on the workspace for the in-sample.
Edit on 05/03/2013: Added equity graphs generated by v4.0
It may be seen from the above backtest that in the in-sample and for no multiple positions, the profit factor of this system of 13 patterns was 2.34 and the win rate 69.32%. The payoff ratio was equal to 1.04, a little higher than expected which is good.
One could argue that these patterns are random artifacts of survivorship bias, i.e. they survived by chance alone and they have no predictive power. This is the reason we used an in-sample to search for the patterns. We will now use the out-of-sample to see how these patterns performed in a regular backtest. Please note that:
- Price Action Lab does not look at the out-of-sample when searching the in-sample. This is very important because if a program looks at the out-of-sample and then selects patterns from the in-sample that also worked well in the out-of-sample this is an extremely bad (and even deceiving) practice that is known as data-snooping. Unfortunately, there are some programs that claim machine generation of trading systems where the user is required to provide the combined in-sample and out-of-sample file and that raises questions about their internal operation which is not disclosed.
- Price Action Lab does not find systems that later allow varying their parameters for fitting their performance in the out-of-sample. All patterns found are parameter-free, eliminating an important but serious concern dealing with curve-fitting performance in the out-of-sample. Again, some other programs that claim machine generation of trading systems is not very clear what they do in the out-of-sample with the parameters obtained in the in-sample.
Step 4: Out-of-sample testing
Price Action Lab makes it very easy to test in the out-of-sample all patterns in the in-sample results. Just click on Test Patterns and then select the location of the out-of-sample data file, as shown below:
The results are obtained shortly after making the selection and confirming it:
It may be seen from the above example test that the pattern performance parameters changed due to the out-of-sample backtest. As a result, one long pattern and two short patterns have a profit factor less than 1. Three patterns failed in the out-of-sample but 10 remained profitable and some have a much higher profit factor than in the in-sample. The profit factor of this group of 11 price patterns can be easily calculated using the backtesting fuction of Price Action Lab applied to each pattern, adding gains and losses separately and then dividing the two. In this case it was found that the profit factor is equal to 2.30. This is a very significant result on a total of 97 trades in the out-of-sample. This amounts to about three position trades per month on the average for this particular system. The frequency may be different for other systems in other markets, as also the proportion of long and short patterns.
Edit on 05/03/2013: Added equity graphs generated by v4.0
It may be seen from the above backtest that in the out-of-sample and for no multiple positions, the profit factor of this system of 13 patterns was 1.30 (as opposed to a 3.30 profit factor when all positions were taken) and the win rate was 55.71% (64.65% when all positions are taken). The payoff ratio was equal to 1.03, a little higher than expected which is good.
Note that the raw returns drawdown shown on the out-of-sample equity graph above scales to an actual maximum drawdown of about -12% for 100K initial equity and a fully invested system at each trade.
The forward performance of this system after the out-of-sample period is reported here.
Step 5: Add the systems to system tracking or generate code
The patterns Price Action Lab discovered, all or any selection of them, can be add to the system tracking module of the program for receiving notifications when a signal is generated or, if the user wishes, code can be generated for implementing a system in some other popular platform.
Comments and FAQ
In this particular example the program found more long patterns than short. This is normal because short-patterns have in general a low win rate. One could decrease the win rate for short-patterns on the search workspace to see what happens. Price Action Lab provides a foundation for performing such experiments very efficiently. This is where its real value is found.
How long will these patterns remain profitable? This is unknown. However, the real value of Price Action Lab is that the search can be repeated again after one year, for example, and the systems can be upgraded. This is not like buying a black-box and have to live with it.
Do I really need the out-of-sample test since it reduces available data for the in-sample? Actually, some people believe that out-of-sample testing is not needed after it is used to confirm that a certain process of finding trading systems leads to significant results. Thus, if it is shown than Price Action Lab can get significant patterns in an out-of-sample as long as the one in this post, then the process can be eliminated all together and the available data can be used as an in-sample to increase search flexibility. Instead of doing out-of-sample tests, the search is rerun every six months and the system is rebalanced.
Disclosure: no relevant position at the time of this post.