Simple Monte Carlo analysis tools are often used to assess the risks of trading strategies and to determine appropriate capitalization levels. However, simple trade reshuffling algorithms can produce misleading results in many cases and fool their users.
There are several Monte Carlo analysis tools available to traders. Some of these tools are even distributed for free via popular forums and blogs. Although these tools can be useful in certain cases, they can also produce misleading results when the following conditions are true:
- Strategy is over-fitted on historical data and/or the win rate is too high
- There is forced long/short symmetry in trade generation
- Longer-term trend following
- Trades that depend on the outcome of certain other trades
- Trade generation uses equity curve feedback
- Strategy dynamics depend on position sizing
- The trade sample is very large
- Strategy is developed via data-mining
Basic Monte Carlo Method
Given a vector of n trade results (P/L):
T[0] = {T1, T2, T3, …,Tn}
then k new vectors may be generated by reshuffling the original trade results. There are now k+1 vectors of trade results, and corresponding equity curves, which can be used in accessing the potential of the strategy, its risk, drawdown, etc.
Rationale and assumptions
The rationale behind the method is that T[0] represents a particular order of trades and naturally the order could be different. Reshuffling of results is employed to arrive at different possibilities. The main assumption here is that all outcomes in T[0] are independent in the same way that one coin toss is independent from another. If the independence condition is violated, then this particular method of assessing risk does not apply and may produce misleading results.
The independence assumption is directly violated in cases 2, 4, 5 and 6 above and indirectly but in a significant way in case 3. Below are a few examples for some of the cases, 1-7.
Case 1
Curve-fitted strategy and/or high win rate
The results of a long-only 50-200 golden cross strategy in S&P 500 are shown below:
This strategy has 100% win rate in SPY unadjusted since inception. This strategy may be considered curve-fitted. Monte Carlo will not work for this or for similar systems even with many more trades if they are curve- fitted because all new T[k] vectors generated by reshuffling will not deviate too far from curve-fitting conditions. This can also be proven mathematically.
Case 2
Forced long/short symmetry
Many trading strategies employ forced long/short symmetry. Long/short symmetric strategies have several advantages but also some disadvantages but an analysis is beyond the scope of the article. The important point here is that most Monte Carlo algorithms break the symmetry that is present by design. As a result, any inferences made from them are false. Below is the equity curve for a 50-200 cross long/short symmetric system in SPY unadjusted data since inception:
Monte Carlo analysis will break the long/short symmetry of this strategy and produce false results.
Case 3
Longer-term trend following
Actually, both strategies in Cases 1 and 2 above were trend-following ones. If a strategy does longer-term trend following in daily or weekly timeframes, then the Monte Carlo method is likely not to be appropriate. Below is the equity curve of a strategy I use that simulates trend-following by generating long-only, short-term trades along trends. Naturally, it also generates many losing trades along downtrends.
This strategy produced large drawdown levels during the dot-com crash and financial crisis bear markets of 2000 and 2007, as shown on the above chart. However, reshuffling the trades in many cases will produce a certain sequence T[k] that has losing trades from both those downtrends. In essence, this would amount to expecting a 100% downtrend in SPY! This is unrealistic and the distribution of returns will cause a highly skewed median drawdown. In turn, both risk of ruin and required initial capital may be overestimated, seriously limiting the profitability of the strategy in the future.
Cases 4, 5 and 6
The non applicability of the simple Monte Carlo tools available to retail traders as part of other programs, at a price, or for free arises in these cases from the fact that trades are not independent by design. Reshuffling of trades assumes independence and, as a result, any conclusions made do not conform to reality. Note that strategies with trades that depend on the outcome of previous trades or on the rate of equity growth are common and actually offer an edge in some cases. In addition, position size algorithms must be considered as an integral part of a trading strategy, otherwise the results may be misleading. It is interesting that some developers, or promoters, of the simplistic Monte Carlo approach argue that it must be used before applying position size. They argue that traders calculate capital requirements and risk of ruin without accounting for complete strategy dynamics. In this case, the results can greatly overestimate or underestimate these parameters.
Case 7
Large trade samples
In the presence of a large number of reshuffles k and when the number of trades is large, there is high portability of generating trade sequences T[k] with drawdown levels equal to the initial capital. Remember that the net profit of a strategy is equal to sum of winners minus sum of losers:
net profit = sum of winners – sum of losers
Next, consider a strategy with $100,000 initial capital but tested over a long period of time so that the sum of winners is $400,000 and the sum of losers is $200,000. The profit factor of this system is 2 but the sum of losers is twice the initial capital. Considering trade sequences in the results with drawdown levels close or greater than $100,000 may be unrealistic. These sequences are outliers and often an artifact of the long backtest. In reality, they have probability near 0 but if one does many reshuffles a great number of such sequences will emerge due to the ordering of trades. In other words, the probability of all sequences of trades T[k] is not the same but it is generally a function of the path the system follows and of discrete time. This effect is not evident with a small number of trades but becomes dominant when the number increases.
Case 8
When a strategy is developed via data-mining, Monte Carlo analysis is not useful in determining whether it is robust or random. However, this is exactly what many users and developers of data-mining programs do in many cases. The reason for the non applicability of this method is that when Monte Carlo analysis becomes part of the data-mining process, it loses it effectiveness as a validation tool due to data-snooping and selection bias. Although Monte Carlo analysis can be used to estimate probabilities of future drawdown levels, assuming it is applicable, it cannot be used as a validation tool. Amateur quant traders use Monte Carlo analysis for the purpose of validating strategies discovered via data-mining bias.
Summary
Simple Monte Carlo analysis tools can be useful in certain cases and provided their users understand the limitation and assumptions. One can safely assume that in the case of more advanced trading strategies that have an edge these tools do not apply without major modifications or may not apply at all. They can serve as a training tool for aspiring quant traders to understand the relationship between drawdown, initial capital and risk of ruin but not for realistic calculations of these parameters except in some simple cases. Actually, methods for determining risk of ruin, initial capital and maximum expected drawdown are quite involved and are considered an integral part of a trading edge by many quants. Simple reshuffling methods cannot fulfill this purpose.
If you found this article interesting, I invite you follow this blog via any of the methods below.
Subscribe via RSS or Email, or follow us on Twitter
If you have any questions or comments, happy to connect on Twitter: @mikeharrisNY
Charting and backtesting program: Amibroker
Technical and quantitative analysis of major stock indexes and 34 popular ETFs are included in our Weekly Premium Report. Market signals for position traders are offered by our premium Market Signals service
