"Mining Fool's Gold" By Grant McQueen and Steven Thorley Financial Analysts Journal March 15, 1999

The authors' paper urges investors to view all "foolproof" investment strategies and trading rules with some degree of skepticism. They remind readers of the pitfalls of data mining and how to avoid them. Their analysis subscribes to The Random Walk Theory of stock prices, which deems that securities prices cannot be forecast. This questions the ability of experts to consistently pick winners and is strong evidence in favor of indexing.

"Successful" investment strategies, even those that have been "successful" for 24 years, may turn out to be fool's gold, not a golden chalice," they write.

They are referring to data mining, which is the practice of finding forecasting models by searching through databases of variables for correlations, patterns, or trading rules. After searching enough variables, say a hundred, a researcher will find, just by chance, about five that are statistically significant at the 95% confidence level. The problem lies when the final pattern is proclaimed significant without providing the number of unsuccessful mining attempts.

They present the Motley Fool's "Foolish Four" investment strategy as a classic example of data mining. The "Foolish Four" portfolio is formed by taking the five lowest-priced among the ten highest dividend yielding Dow stocks, dropping the lowest priced stock, and giving the second-to-the-lowest stock twice the weighting (40% weight) of the rest.

It is a formula for market timing that has worked well over the past years. From 1973 to 1996, the portfolio outperformed the DJIA by a 12% margin but had a substantially higher standard deviation.

The authors, with the intention of evaluating the pitfalls of data mining, tweaked the "Foolish Four" model and came up with the "Fractured Four" portfolio that beat the DJIA by almost 19% a year on average. One can probably come up with numerous other models that beat the "Fractured Four", but what predictive value would they have? Almost none if they are a product of data mining.

The paper provides the following guidelines to investors to challenge the integrity of any investment strategy that promises to deliver superlative returns:

• Look out for a high number of variables
• Make sure there exists a plausible reason why the strategy worked
• Test the strategy out-of sample. If it worked from years 19aa to 19bb, then it should also work from years 19xx to 19yy
• Adjust for risk, transaction costs and taxes. The strategy may lose almost all its apparent benefits

Even if a trading strategy does become popular, there will be investors flocking to take advantage of it before the price runup is expected to occur and then you'll have others trying to beat them and so on.

So if we resign to the fact that securities prices cannot be forecast then it would be hard to make the case for the ability of pros to consistently outperform the market. Knowing this, investing in low-cost index funds seems to a lucrative proposition in comparison to actively managed stock funds that charge high fees that eat into investors' return.

Review By Rahul Seksaria, Assistant Editor