When the stock market crashes, do the professionals and everyday investors see the same future? A pair of economists at MIT set out to answer that question using a source of financial forecasts that has been sitting quietly on library shelves and microfilm reels for nearly 70 years.
In an NBER working paper, David Thesmar and Emil Verner of the MIT Sloan School of Management report that expectations from a well-known independent equity research firm behave very differently from the expectations of individual investors and professional economic forecasters. When retail investors get excited about stocks, sophisticated analysts tend to get cautious. And when the two groups disagree most sharply, trading volume spikes.
The puzzle behind the project
For decades, finance researchers have debated why stock prices swing so much more than the dividends companies pay. One classical view holds that swings reflect rational, shifting views about risk: investors demand higher expected returns when stocks look cheap. Under this view, expectations of future returns should be high when prices are low relative to earnings.
But when researchers began asking real investors what they expected, the answers didn’t fit. Individual investors, it turned out, tend to feel most optimistic right after prices have already gone up, when stocks look expensive. That has pushed many economists toward behavioral explanations, in which investors extrapolate recent trends rather than think like textbook rational agents.
Thesmar and Verner wanted to know what a more sophisticated group of forecasters thought, and whether their beliefs looked different. To find out, they needed data that stretched further back than the surveys most researchers rely on.
Digitizing seven decades of analyst forecasts
Their source was Value Line, an independent equity research firm that has published forecasts on roughly 1,500 companies per year since 1956. Value Line analysts publish projections of each firm’s earnings per share three to five years out, along with a target stock price at the same horizon. Warren Buffett has reportedly subscribed for years; Charlie Munger once called Value Line’s charts “a human triumph.”
For the recent period (1987 to 2024), Thesmar and Verner used Value Line’s own digital database. For the earlier decades, they physically scanned bound volumes and microfilm records held in various libraries, then ran the images through optical character recognition software to extract forecasts. In total, they digitized 38,696 investment reports covering more than 92,000 firm-year observations. They then verified the digitized numbers against Compustat and against overlapping years in Value Line’s own digital files.
From these forecasts, the researchers backed out an implied expected return for each firm and each year, then aggregated the firm-level numbers into a national time series. They compared this “Value Line expected return” against eight other surveys that capture the views of individual investors, professional forecasters, and other finance professionals.
Four patterns emerged
The first finding is that Value Line’s expected returns look very different from those of other investors. Value Line’s forecasts move around a lot more than most survey-based measures, and they are only weakly, sometimes negatively, correlated with the expectations of individual investors. When retail investors surveyed by researchers like Robert Shiller or Graham and Harvey feel most bullish, Value Line’s implied expected returns tend to be lower, and vice versa.
The second finding is about valuation. When stocks are cheap relative to earnings, Value Line’s expected returns are high, matching the pattern that classical rational models predict. A statistical relationship between the cyclically adjusted earnings-to-price ratio and Value Line’s expected return produced an R-squared of 65%, meaning the analyst forecasts track valuation swings closely. None of the other eight expected-return series showed a comparable positive correlation. Expected returns from surveys of individuals tended to move in the opposite direction: high when prices were high.
Digging into what drives Value Line’s expectations, the authors found that the firm’s analysts appear to assume valuation ratios will drift back toward long-term averages. Roughly 87% of the variation in their expected returns comes from this “repricing” component, not from expected earnings growth. Individual investors, on the other hand, appear to extrapolate from recent stock returns: strong past performance makes them more optimistic, weak performance more pessimistic. Notably, past earnings growth didn’t do much to explain any group’s expectations, once past returns were accounted for.
The third finding is that Value Line’s expected returns actually forecast future returns. Across the nine survey-based series the researchers tested, Value Line’s was the only one that reliably and positively predicted future realized market returns. The predictive power was strongest at longer horizons: at the five-year horizon, the R-squared reached 19% for nominal returns. Predictability was weaker for excess returns (returns above the risk-free rate), but the direction was consistent, and it was stronger in the more recent half of the sample.
The fourth finding concerns trading volume. When the gap between Value Line’s expected return and the average expected return of individual investors widened, trading volume on the market rose, and so did volatility. In quarterly data from 2001 onward, the correlation between belief disagreement and trading volume was 0.66. The authors read this as evidence that both groups of investors are actually trading against one another, not just holding views in isolation.
How the researchers interpret the pattern
Thesmar and Verner frame their results around a model of “heterogeneous beliefs” in which two types of investors coexist. Naive investors extrapolate recent returns; sophisticated investors have close-to-rational expectations but face limits on how much they can bet against the naive crowd. In this setup, when naive investors get bullish and push prices up, sophisticated investors know that future returns will probably be lower and lean the other way. Their beliefs are, by construction, negatively correlated with those of naive investors.
The authors argue that Value Line looks a lot like the sophisticated investors in this model, while individual investors resemble the naive ones. They interpret the volume finding as evidence that both groups matter for prices: if only sophisticated views mattered, disagreement wouldn’t generate trading.
The researchers also push back on a recent strand of literature that attributes most stock market fluctuations to expectations about future cash flows rather than expected returns. Using Value Line’s own numbers, expected earnings growth over the next four years explains essentially none of the variation in the price-to-earnings ratio. About 59% of the variation comes from expected returns, and the remainder comes from the expected long-term valuation multiple. The authors interpret this as showing that the “cash-flow story” depends heavily on whose expectations you’re using.
Caveats worth noting
The researchers caution against reading their variance decomposition as a causal statement. The accounting identity they use holds for both rational and non-rational beliefs, so the finding that expected returns “explain” most valuation movements doesn’t by itself prove that discount-rate shifts cause price swings.
They also note that Value Line’s forecasts are not fully rational. In the early decades of the sample, especially the 1960s and 1970s, Value Line’s expected returns appear to have been too sensitive to valuation ratios, meaning the analysts overshot when stocks looked cheap or expensive. Predictability of returns using Value Line’s forecasts is also weaker for excess returns than for nominal returns, and in the sample studied, conventional predictors like the CAPE ratio show similarly weak performance on excess returns.
Finally, the data reflect one particular research firm. Whether other sophisticated market participants share Value Line’s approach to valuation, or whether Value Line’s own methodology (which literally involves drawing a “value line” through past valuation ratios) built in a mean-reversion assumption from the start, is a question the authors flag rather than resolve.
