A Deeper Look: Why are "Expert" Predictions and Polls Wrong So Often?
Why are “Expert” Predictions and Polls So Wrong?
How investors should receive data, polls, and consensus expectations
On Friday, June 5 , the Department of Labor reported that non-farm payrolls added 2.5 million jobs during the month of May. But “consensus expectations” were for May’s numbers to show a loss of about 9 million jobs. That 11.5 million “miss” is more than the population of New York City and Houston combined.
Then on Tuesday, June 16 , the U.S. Commerce Department reported that U.S. retail sales surged a whopping 17.7% in May, as consumers returned to shopping and spending. But economists surveyed by Bloomberg had expected only an 8% increase. That 17.7% gain represented approximately $485 billion, meaning the expert economists were off by about $266 billion, which almost as much as Amazon’s 2019 revenues.
On Thursday, June 18, the Philadelphia Fed Index for June turned positive as it came in at a positive 27.5 from a negative 43.1 the previous month. Economists polled by MarketWatch expected a reading of negative 20. That’s not a little “miss.”
During quarterly earnings season, it seems as if most companies routinely beat “consensus expectations” when it comes to reporting earnings. Ever wonder why? Are consensus expectations pegged too low so that it will make companies look better? Or are the “experts” that incompetent?
Remember the polls that were so very wrong about Brexit, leading to a massive global market selloff as markets had priced in an expected outcome based on those polls (leading to trillions of dollars being wiped out)? Remember the polls leading up to our 2016 Presidential election?
These are just a few recent examples, but there are countless more that have us wondering how accurate estimates from experts truly are, whether consensus expectations are helpful at all and if we can rely on polls to make investing decisions.
The short answer: take all expert opinions, polls and “consensus expectations” with a big fat grain of salt. Maybe two grains.
How Do Such Big Misses Happen?
It came as a shock to almost everyone when the May employment numbers were released. How did so- called experts miss predicting the numbers by 11.5 million? Are the numbers that unreliable or is there some deep-conspiracy that we don’t know about? No and no.
First off, the government agencies collecting data are having a challenging time given COVID-19. Just read the disclosure from the Bureau of Labor Statistics:
“Data collection for both surveys was affected by the coronavirus (COVID-19) pandemic. In the establishment survey, approximately one-fifth of the data is collected at four regional data collection centers. Although these centers were closed, about three-quarters of the interviewers at these centers worked remotely to collect data by telephone.
Additionally, BLS encouraged businesses to report electronically. The collection rate for the establishment survey in May was 69 percent, slightly lower than collection rates prior to the pandemic. The household survey is generally collected through in- person and telephone interviews, but personal interviews were not conducted for the safety of interviewers and respondents. The household survey response rate, at 67 percent, was about 15 percentage points lower than in months prior to the pandemic.”
Now that does not of course account for why so- called experts were so wrong in their predictions. Here is one explanation that makes intuitive sense: statisticians rely on data, interpret trends and then make predictions. And most of the time, statisticians stay within a certain range of predictions and don’t stray too far outside the lines. In other words, their predictions usually follow a predictable and linear line.
Think about this for a second: in March the U.S. lost over 1.4 million jobs and in April that number was a staggering 20 million. And while states were beginning to open up, how many could have really predicted a gain for May? If you really think about it, we should not be surprised when huge and rapid swings are missed.
Consider those Retail Sales numbers reported on June 16th as additional evidence of how huge and rapid swings are very difficult to predict. How many statisticians would have predicted these massive jumps:
- Clothing and clothing-accessories stores jumping 188%;
- Furniture and home-furnishing sales jumping 90%;
- Stores focused on sporting goods, hobbies, musical instruments and books jumping 88%;
- Electronics and appliance stores jumping 51%; and
- Motor vehicle sales jumping 44%.
In the cold light of dawn, no one would have predicted the largest retail sales gain since 1992. If asked, everyone might have hoped for such results, but no one would have predicted such an outlier – the recent data just did not suggest that happening.
Nonetheless, the huge retail sales numbers for May bode well for an economic recovery since consumer spending represents close to 70% of our GDP.
Consensus Earnings Estimates & Surprises
Each quarter, public companies open their books and disclose to the world how they performed during the past three months. It can be a time of great stress and great jubilation for shareholders. The move in a company’s share price following an earnings announcement often stems from whether the company beats or misses the expectations set by Wall Street analysts. And the collection of expectations from Wall Street experts form what is called “consensus expectations.”
But how excited should we get when the “experts” are wrong about corporate earnings expectations and companies end up surpassing consensus estimates?
Take Alphabet for example – Google’s parent company. Shares rose 9% in extended trading after the tech giant reported earnings results (second quarter of 2019, not recently) that beat expectations set by Wall Street. In fact, Alphabet reported adjusted earnings per share of $14.21 on revenue of $38.94 billion, versus the $11.30 earnings per share on revenue of $38.15 billion that Wall Street expected.
Then there was the time that Starbucks jumped nearly 6% after beating expectations. And the time that Tesla dropped 10% after missing consensus earnings.
The truth is, during earnings season, volatility often ramps up and individual stocks can have some of their biggest moves of the year on the days they report.
But the bigger issue for individual investors is that maybe these earnings “surprises” aren’t really surprises after all.
Consider this statistic from research firm FactSet:
- 63% of S&P 500 companies beat earnings estimates in the first quarter of 2020
- That 63% number was the lowest percentage of companies beating earnings estimates since the third quarter of 2012
Further, FactSet reports that 75% of technology stocks in the S&P 500 beat earnings estimates and that 85% of Consumer Staple stocks in the S&P 500 beat estimates too.
The truth is that most companies regularly beat earnings expectations. So maybe you should rethink whether or not you should rely on this measure when you make investing decisions.
The Challenge with Polls
Most of us assume polls are mathematics and as such, there is no room for interpretation since numbers can’t lie. And while the conclusions are not usually wrong, often times the conclusions can be a bit misleading. As such, it’s important to understand how those results were reached as well as what entity is doing the polling.
Consider polls trying to determine the winner of the 2020 Presidential race. One poll might only query registered voters, another poll might only query adults aged 20-30, another poll might query only adults without regards for their voting status and another might poll only women that go to Lifetime Fitness on the second Tuesday morning of each month between the hours of 9 am and noon. Yes, that’s hyperbole, but unless you read the fine print, you wouldn’t know.
While none of these methodologies are wrong per se, each will have limitations and, to a degree, will influence results. And more often than not, the conclusion drawn from a certain poll focuses on a screaming headline and not the methodology underneath. For polls to be more useful, it’s important that we understand their methodologies as that will in turn help us better understand their limitations.
What Can Investors Believe?
The answer to that question, of course, is not easy. The best advice might be to receive such data, consensus estimates and polls with a healthy dose of skepticism. Further, you should:
- Assume there are going to be mistakes, as happened when the Department of Labor admitted that there were mistakes made in the data collection for March and April;
- Consider the source and ask if there might be some sort of hidden agenda. And while some might suggest that there is manipulation everywhere – including from U.S. Government agencies – that’s not so likely. The potential for manipulation is higher, however, with some more obscure organizations that clearly have a well-known bias;
- Rely on your instincts. Call it the smell test or the eye test. Use your common sense for determining whether something feels authentic or not.
What Would Mark Twain Say?
Finally, investors should rarely make investing decisions based solely on recent data sets or polls or consensus anythings. And if you feel tempted to do otherwise, talk it over with your financial advisor first so that you can discuss how statistics are manipulated to support certain points. You might also rely on the words of Mark Twain when he said:
"There are three kinds of lies: lies, damned lies, and statistics.”
But, it wasn’t Mark Twain that came up with that phrase. Nor was it British prime minister, Benjamin Disraeli. It was attributed to an unknown writer in 1891 and then later to Sir Charles Wentworth Dilke.
At least that’s what Wikipedia says is true...
Sources: Bloomberg.com; marketwatch.com; bls.gov; census.gov; philadelphiafed.org; factset.com; wikipedia.org
