Suppose a trader has purposely used a random system to trade SPY since its inception. What was the probability to make annually more than the buy and hold return? Or suppose a trader has worked hard to develop a trading system that generated a positive return over the same period. Was her system intelligent or it was tossing a coin? This post offers two examples that try to answer these questions.
The consciously random trader
Suppose a trader had a good random number generator, like the Mersene Twister I used for the purposes of this analysis (part of Amibroker), and when trading in SPY started back on January 29, 1993, she decided that she would like to test her luck, run the generator for numbers in the interval [0,1) and go long at the close if the outcome p is greater than 0.5 and short if it is less than 0.5. Then, repeat the same procedure before every market close and if the outcome is a reverse entry, then exit and reverse position, otherwise keep the position open. Commission is 1 cent per share. Had this trading any chance of being profitable?
A posteriori, a simulation of this random system shows that trading randomly on purpose had probability of about 32.50% to have generated a positive CAR (compound annual return) from SPY inception to 05/02/2013. Thus, about 33% or all random traders would end up with some profit. But the probability of realizing a CAR above the buy and hold return of 6.55% (no dividends) was only 2.59%. Even smaller at 0.68% was the probability of realizing the buy and hold return of 8.55% that includes dividend reinvestment. However, a large number of consciously random traders, about 33% of them, would have realized some kind of a profit. Here I must argue that the market seems to reward consciously random traders in great proportions but appears to be ruthless with those that think they poses an edge when they do not actually. Below is a graph summarizing the statistics I just talked about:
The above graph shows the distribution of CAR for 20,000 simulation runs of the random SPY trading described above in the period specified and some relevant statistics. Percentages above specific CAR values are also shown. These percentages are also p-value estimates and can be used in hypothesis testing. For example, if a trader thinks that his trading method must be significant at the 97.5% level, it must have generated a CAR in excess of 6.50% in the same period. A CAR below that does not provide evidence against the null hypothesis that the trading method has no intelligence and it may be essentially random.
Random trading (as opposed to consciously trading randomly)
Many traders put a lot of hard work and spent countless hours in developing and testing trading systems. Unfortunately, most are fooled by randomness and many of the systems they come up with are mere artifacts of curve-fitting and data-mining and selection bias, to name just a few factors that contribute to the randomness of results. However, most of those traders think their systems have intelligence. But do they? Simulation can be used to answer this question by comparing the results of those systems to results obtained from random systems.
Some traders often select a period of time when a security had a low buy and hold return and try to develop a trading system that will exceed that. However, exceeding buy and hold returns in backtesting and not in actual trading does not imply that the system has any intelligence. The trader may be fooled by randomness and selection bias. Below I show an example for SPY. The random trading described before is now repeated from the start of 1999. The buy and hold return with dividend reinvestment since January 1999 for SPY is about 3.82%. It turns out that close to 10% of random traders made more than the buy and hold return in a simulation with 20,000 runs. Significance at the 95% level would require a system with a CAR of about 6% or higher. However, whereas traders with actual performance records can use the 95% significance level to test their results for randomness, in the case of backtesting results a significance level of 99%, or much higher than that, should be used to account for all the other factors that can result in system performance degradation. Even then, one cannot know whether outstanding backtesting results are due to outstanding curve-fitting or due to system intelligence. This can be shown only in forward testing. If during forward testing the adjusted CAR falls below the one required for significance then the system may be in trouble. Hey, I never said trading and trading system development are easy and whoever tell you that is selling you hopes and dreams. Here are the results for this second example:
The graph shows that since 01/05/1999 to 05/02/2013, nearly 10% of conscious random trading over-performed buy and hold with dividend reinvestment. I do not think that markets can get any more generous rewarding in excess of buy and hold return 1 out of 10 traders who consciously trade randomly. But those who claim to have a system that is intelligent, must show much higher returns. Otherwise it is better to get a coin and start tossing it before the market close and just in time before MOC orders are accepted by the broker. If you have a trading system that has generated a CAR of 4% trading SPY in the above period then you may be no better than a consciously random trader the only difference may be that you are doing it unconsciously and you are fooled by randomness.
Disclosure: no relevant position at the time of this post and no plans to initiate any positions within the next 72 hours..
Charts created with AmiBroker – advanced charting and technical analysis software. http://www.amibroker.com/”