Can an efficient market exhibit irrational pricing?
The answer is Yes. The answer given here is based on a re-reading of Robert Shiller’s 2005 book, Irrational Exuberance.
Let us first define terms. An efficient market is one where all profitable trades have been exploited; in the textbook sense, it is one where the price observed is the market-clearing price, or where the demand and supply curves intersect. Irrationality is a defect in the perceptions of market participants; they may be subject to excesses of optimism or pessimism, which may cause them to buy more or less than they would if they had been rational.
The concept of fundamentals also needs to be defined. The “fundamental” can be distinguished from the market-clearing price. The former is the price that turns out ex post to be correct after all uncertainty has been removed. The latter is the “discovered” price in a market. One can think of a stock market as one where traders bet on the future earnings of a particular company; and the fundamental price is the statistically best (unbiased) estimate of the present value of such future earnings. The fundamental is unknowable with certainty but can be given an estimate that abstracts from the effects of excessive optimism or pessimism. (The so-called “value investors” are said to trade on the basis of fundamentals.)
Of course, ex ante, optimism or pessimism is not immediately obvious; they become known or explainable only at some later point in time. Nonetheless, there is enough in the history of financial bubbles and crashes that gives markers of whether there is an excess of greed or fear. Greed is when just about every trader thinks the market is an easy party; fear is when there is palpable uncertainty as to when a bear market will end. What cannot be predicted, as has been pointed out some observers, is the point at which greed turns to fear, or when a bubble bursts to become a crash. This means one can at least avoid riding a bubble in its late stages; and one can also perhaps avoid “buying too soon” in a crash.
Consider the market then as a price discovery mechanism. The conventional efficient markets theory suggests that no individual can consistently do better than the collective wisdom of market traders. It is a zero-sum game that pits every trader against every other trader in “divining” the correct price. The individual trader with persistently incorrect views will lose to the market as a whole. The “correct” price here is the one that clears the market, taking into account the different views taken by all participants. (There is of course no guarantee that the actual or observed price is correct; the actual price is one that simply reflects one particular trade between two participants at a point in time.)
But what happens if there is generalized irrationality in the sense of excessive optimism or pessimism?
For example, such a generalized excess of optimism would drive the (correct) price “too high” relative to the fundamentals. It is possible that value investors could not pull the price down back to fundamentals because of uncertainty (the optimists may turn out to be right) or because of limits on short sales. Nonetheless, over time, the optimists and pessimists will lose the game of price discovery to the value investors. And yet, the market is still efficient in the sense that all will get to be at the best they can do (no one can be made better or worse off; they have exploited all possible voluntary trades based on their individual views on the correct price) even if the market-clearing price differs from fundamentals.
This means that an efficient market can have irrational pricing, which by definition is a market price that results from a generalized excess of optimism or pessimism.
It has been said that one way to riches in the stock markets is to “sell too soon.” That way, one can take the money and run. Every Ponzi operator, legitimate or otherwise, follows this strategy. (Shiller has a discussion of “naturally occurring Ponzi processes.”) This also means that a trader who sold at the very top was just plain lucky; so was the trader who bought at the bottom. It doesn’t mean that lucky is bad. It just means that luck is rare and not easily replicated.