Premium Market Analysis, Trader Education, Software, and Trading Strategies

Price Action Strategies, Technical Analysis

An Example of Selection Bias in Pattern Analysis

The 5-day winning streak in S&P 500 and SPY is a pattern easy to spot on a daily chart but any inferences about the future direction of the market based on only this pattern are based on selection bias. There may be other patterns on the chart that are as, or maybe even more, important as the 5-day winning streak pattern. Chartists and most analysts who rely on backtesting visually identified patterns often commit this notorious fallacy.

Identifying pattern visually on the charts and trying to tarde based on them was a practice followed in the middle of the 20th century. Inferences were made based on what some authors have asserted without any sort of quantitative details. Then in the late 1980s backtesting came along and some traders attempted to quantify the impact of the patterns that were visually identifiable. Problems arose however when either the patterns could not be objectively described or when the selection bias was too large because of the presence of other not as easily identifiable patterns that implied different conclusions. When these notorious problems were realized many abandoned visual chart analysis for other methods, like for example algorithmic trading. It is peculiar though that even nowadays there are scores of analysts who pick one easily identifiable pattern on a chart, backtest it on only one symbol and then try to make inferences about the market direction being ignorant of the fact that in most cases any trade samples obtained in this way are nowhere near those required to guarantee the statistical significance of the claims made.

An example of such malpractice is the 5-day winning streak in SPY as of yesterday’s close, shown on the chart below:


The Winning Streak Indicator of 6 days on the bottom pane is at 100% meaning that during the last 6 days the market registered successive higher closes. However, analyzing this pattern alone in SPY is not sufficient for making inferences about the future direction of the market because of the small sample of trades, although the results may be positive for a large selection of exists. In addition, there are other patterns that may be important. Visual identification leads nowhere when trying to answer these questions and the automation of the process is required.

Specific aspects of this task may be accomplished by using the scan function of Price Action Lab, which is a software program that identifies short-term price pattern formations by analyzing price action in a completely automated way. In the case of SPY, a workspace was setup with non-adjusted daily data since inception, 2% profit target and stop-loss and the remaining parameters set as shown:


PAL was instructed to look only at 6-bar patterns for this example only. The results are as follows:


In the screenshot above each line corresponds to one exact price pattern. P is the win rate, P1 is the 1-Bar win rate, Trades is the number of historical trades, CL is the maximum number of consecutive losers and Target and Stop the values of the profit target and stop-loss. C indicates the type of target and stop-loss, in this case it is a percentage added to the entry price, shown under Trade On as the open of next bar.

It may be seen that a total of 9 distinct 6-bar price patterns were identified. Pseudo code generation will show that the first pattern with Index 1 is the 6-day closing winning streak. This pattern has 93 non-overlapping trades in the history of SPY, win rate equal to 58.06% and 1-bar win rate equal to 0%, meaning that at no instance the profit target was hit at the following day after its formation. This is the first indication of lack on momentum in the case of this pattern.

 A backtest of this pattern shows an equity curve that is not impressive at all:


Only after 2007 this pattern generated an upward slopping equity curve. One cannot just claim from aggregate results that this is a pattern with a bullish bias because it all depends on prevailing market conditions. The robustness of this pattern to variations in the exits is shown below:


It may be seen that for small profit-target and stop-loss the performance is negative and the Robustness Index is only at 68.42, meaning that only 68.42% of the variations show positive expectation. This is another indication of a not so strong bullish bias.

Then, a portfolio backtest on Dow-30 stocks with adjusted data since 01/2000 shows disappointing results for this pattern as it was profitable only in 14 stocks and the portfolio profit factor barely stayed close to 1:


The conclusion is that what one sees on a chart – because that is easy to do – and backtests on a single security may be an artifact of randomness. This 6-day winning streak in SPY has no statistically significant bullish bias and although the market may continue to rise there is no causality here, i.e. this pattern does not imply a continuation of the uptrend but it is rather a random formation or outcome of it.

Determining the market bias

In order to determine the market bias all possible patterns must be accounted for and put in a proper ensemble. Then that gives the directional bias or probability. This can be accomplished with the p-indicator function of Price Action Lab. The results for SPY for different target and stop-loss values are as follows:


Explanation of the results

The probability of an up move and of a down move for a particular target/stop pair are the long success rate and the short success rate of the p-indicator, P-long and P-short, respectively, as shown on the screenshot above. P-long is a measure of the probability that a long position initiated at the next open will result in a profit for the corresponding profit target and stop-loss values.  Similarly, P-Short is a measure of the probability that a short position initiated at the next open will result in a profit for the corresponding profit target and stop-loss values.  A more useful measure in this case is the difference between the long and short probabilities, shown as P-delta, which is a measure of the bias, or tendency, of the market to move towards the long or short direction. The parameter S represents a significance measure of the calculated numbers with a zero value indicating no significance, values around 5 indicating moderate significance and values above 8 indicating high significance.

It may be seen that for profit target and stop-loss pairs of 1% and 2% the directional bias is negative and that may mean that after such pattern there is profit taking. The directional bias rises to above 54% for 5% profit target and stop-loss. Note that these results were adjusted for SPY trend bias (detrended).

Related post 

Disclosure: no relevant positions.
Charting program: Amibroker