The timing ability of a machine designed trading system for SPY is analyzed using random data. The performance of a system designed on random data is compared to that of a system designed using actual data. The hypothesis that the system that was developed on actual data was based on spurious correlations can be rejected and thus this provides strong evidence that the system has timing ability.
The SPY2p5 system was machine (re)designed at the end of 2012 using all available actual daily data. Its performance during 2013 came close that that of buy and hold. However, one could claim that the system was based on spurious correlations and it performed well in a strong bull market because of chance. If that were actually the case and the timing of the signals plays no important role then a random system, like one developed using random data, would also perform well on actual data. This study is organized in the following steps:
1. Random data are generated with the same number of bars as SPY from inception to 01/31/2012.
2. A system is machine designed by Price Action Lab using the random data with the same parameters as the SPY2p5 system.
3. The performance of the two systems, the random and actual, is analyzed
Random data for SPY were generated in Amibroker using the Mersene Twister random number generator function mtRandomA(). Out of many possibilities, a random chart was chosen to reflect an overall upward bias in SPY as shown below:
The top window of the chart shows the random data series and the bottom shows the actual SPY data.
Below is a screenshot of the search workspace used for the machine design with the random data, which apart from the choice of data file is identical to the one used for the actual data:
The in-sample was from 01/29/1993 to 12/31/2012 and the out-of-sample is the year 2013.
The results consisted of 19 long patterns and 36 short patterns. This is the in-sample equity curve of the system (one share per position – points test) developed on random data for single positions (no multiple open positions):
Below is the out-of-sample equity curve. As expected, the performance is close to break-even. Specifically, the win rate is 50% and the CAR is -2.52%:
We should be more worried if the performance came out positive. The results confirm that the patterns were spurious correlations fitted in the in-sample and naturally break-down in the random out-of-sample.
Next, since the two systems, the actual and the random, have different numbers of long and short patterns we need to normalize them. The actual system has 62 long and 7 short patterns and the random system 19 long and 36 short. For comparison purposes we will use 19 randomly chosen long patterns from the actual system to go along the 7 short and only 7 randomly chosen short patterns from the random system to go along the 19 long. In this way and for comparison purposes only both systems have the same number of long and short patterns. Below is the equity performance of the system developed on actual data for 2013 after the normalization:
The win rate is 62.50%, the profit factor is 1.72 and CAR is 18.02%. Next is the equity performance of the random system:
As expected again, the performance is random with win rate 50% and a CAR of just 1.27%. profit factor is slightly above 1 at 1.07.
Finally, it is interesting to note the performance of only the long patterns in the random and actual in-sample for the random system. Below is the equity performance of the long patterns in the random in-sample and it is expected to be positive because of selection:
Win rate is 65.77% and CAR is 10.77%. Next is the equity performance of the random long patterns in actual in-sample data:
As was expected, the performance is random with win rate of 50.71% and CAR equal to -1.41%. Therefore, no spurious correlations, even in the presence of a long-term bias, can match the performance of timing signals. If there is no timing involved, then the system performance breaks down.
Price patterns and trading systems developed based on them have timing ability if properly data-mined and used. Developing final trading systems from price pattern constituents is not a trivial task and requires a lot of hard work and devotion. There is nothing ready to be served on a plate that will offer immediate gratification to a trader. Any edge consists of 3 parts: (a) the tools used, (b) the knowledge of the developer and (c) the amount of hard work put. The example in this article showed that price patterns can have timing ability but not all price patterns do. Selecting the proper market, timeframe and risk/reward parameters is essential for success.
Disclosure: no relevant positions.
Charting program: Amibroker