You probably have read those types of reports from big consulting companies or big players in the communications industry, usually coming out at the end of the year as their flood of trend reports or future forecast insights. We all read them, some of us probably align our activities to the things they find out from those reports. But there is a big gotcha with those reports, either conducted by experts and based on their opinion or a qualitative survey among industry insiders: all of them carry immense biases related to their task of giving an opinion, and the authors further have a motivation, given their role in the industry, to even manipulate the results and statements of the report. But let’s dig deeper into the concept of biases first.

A bias is an error with a direction

A random error goes in both directions. Ask enough experts and the errors cancel out. A bias goes in one direction. Ask more experts who share the bias and you get a more confident wrong answer. Research on structured group judgment makes the point: aggregating similarly biased judgments reinforces the bias at group level (Bolger, Rowe and colleagues). Averaging helps with something else. Simple averaging works almost entirely by reducing noise, so a shared bias stays (Satopää, Tetlock, Mellers and colleagues).

Seven biases appear in the research on expert judgment and qualitative interviewing. We picked them because each one hits trend reports directly.

Experts know less than they think

Philip Tetlock followed forecasts from 284 experts over about two decades. His data supported the real-world robustness of overconfidence, hindsight bias and self-serving counterfactual reasoning (Philip Tetlock and colleagues). Experts who weighed both sides of an argument were less overconfident and got less media attention than experts who argued one confident line (Philip Tetlock).

Daniel Kahneman and Gary Klein asked when expert intuition deserves trust. Two conditions must hold. The environment is regular enough to learn from, and the expert has had practice with accurate, timely feedback. Felt confidence is not a reliable sign of accuracy (Kahneman and Klein). A five-year trend forecast meets neither condition, in our view.

Hindsight bias adds a second problem. People feel they knew an outcome all along (Neal Roese and Kathleen Vohs). A report whose forecasts nobody scores leaves this bias unchecked.

Incentives shape the answer

Security analysts offer a clean test. A study of roughly 12,000 analysts at 600 brokerage houses found optimistic analysts, relative to the consensus, had better career outcomes, even with accuracy held constant. For analysts covering stocks their own house underwrote, careers depended less on accuracy and more on optimism (Harrison Hong and Jeffrey Kubik).

Sponsorship shows the same pattern. A Cochrane review of 75 papers found drug and device studies funded by manufacturers reported favorable efficacy results more often (risk ratio 1.27) and favorable conclusions more often (1.34) than studies with other funding. Standard risk-of-bias checks did not explain the gap (Andreas Lundh and colleagues). This is medicine, not marketing. The mechanism is the same: the author gains when the result points one way.

Disclosure alone does not fix this. In experiments, people did not discount advice from conflicted advisors enough, even when the conflict was disclosed. Disclosure sometimes made matters worse (Daylian Cain, George Loewenstein and Don Moore).

Optimism builds hockey-stick forecasts

Decision makers treat their own problem as unique. They anchor on their plan and their scenarios, and they neglect the statistics of similar cases. Daniel Kahneman and Dan Lovallo call this the inside view. Overly optimistic forecasts result (Kahneman and Lovallo). Bent Flyvbjerg explains inaccurate forecasts of costs and demand with two causes: optimism bias and strategic misrepresentation. The first is an error. The second is a choice (Bent Flyvbjerg).

Gartner’s hype cycle shows how this looks in trend work. A critical review compared Gartner’s reported progression of technologies with empirical evidence and found incongruences (Ozgur Dedehayir and Martin Steinert). We honestly doubt any forecast growing steeply from a small base without friction.

A number in the question moves the answer

Experienced judges read a criminal case file. They then rolled dice to set a sentencing demand. The random number pulled their sentences toward the figure (Birte Englich, Thomas Mussweiler and Fritz Strack). Experience did not protect them. A survey question with a number inside plants an anchor in the same way. “Will AI replace 30 percent of agency work by 2030?” puts 30 in the expert’s head before any thinking starts.

Groups pull answers toward the majority

Solomon Asch showed people give wrong answers to match a group. A 2023 replication with 210 participants, mostly students, found a 33 percent error rate in the classic line task. Paying for correct answers lowered the rate to 25 percent. Social influence stayed. On political statements, 38 percent conformed (Axel Franzen and Sebastian Mader).

Sushil Bikhchandani, David Hirshleifer and Ivo Welch modeled the mechanism. People who observe those before them follow the behavior of the group and set aside their own information. The result is fads and fragile consensus. Delphi exists because free-flowing groups suffer from dominant individuals and pressure to conform (Fergus Bolger and George Wright). A workshop where experts speak before everyone has written down an answer sets up exactly this problem.

Interviewers find what they expect

Confirmation bias means seeking and reading evidence in ways favoring existing beliefs (Raymond Nickerson). In qualitative research, this shows up in the analysis. Researchers seek data confirming their hypotheses and overlook inconsistent data (Joanna Smith and Helen Noble). Take a researcher writing a report on “the rise of X”. The interview questions focus on X, the quotes come from experts who agree, and the rest stays out.

Question format adds to the problem. Jon Krosnick’s satisficing theory explains why respondents who take shortcuts lean toward agreeing. An agree/disagree format presents the researcher’s hypothesis as the answer to accept.

Interviewees give acceptable answers

Social desirability bias is the tendency to present reality in line with what seems socially acceptable. Nicole Bergen and Ronald Labonté documented how a research team detected the bias in interviews and focus groups in rural Ethiopia. Data collectors spotted cues in the nature of responses and in word choice. Rapport, careful questions, training and regular debriefs reduced the bias. The authors expect the bias to persist, but they show how to reduce the effect.

Their setting was health research, not marketing. We apply the mechanism, not the numbers. Experts speak for an employer, a reputation and a position. We expect the same pressure on an agency CEO asked whether agencies will shrink.

Why expert research is more exposed

Four features raise the exposure.

  • The opinion is the data. No outside measurement checks an expert’s view. In a panel of 15, three experts with the same belief make up 20 percent of the result.
  • Feedback arrives late. Forecasts resolve years later, if anyone checks. Intuition needs fast feedback to be reliable (Kahneman and Klein).
  • The experts have stakes. Employers, products and reputations sit behind the answers (Hong and Kubik, Lundh and colleagues).
  • The researcher’s beliefs shape the question, the method and the reading of the data (Smith and Noble). Overconfident experts get more media attention, so a panel picked by prominence tends to select them (Tetlock).

What works

No method removes these biases. The evidence below shows what reduces them.

  • Collect first answers alone and anonymously. Delphi does this and the IDEA protocol extends the approach. Experts research alone, give private estimates with reasons, see the spread of the group, discuss and consider the opposite, then give a second anonymous estimate. The estimates are aggregated mathematically (Anca Hanea, Victoria Hemming and colleagues). A review of Delphi studies found Delphi groups beat statistical groups in 12 studies against two, and standard interacting groups in five against one, with ties in both comparisons. The review found no consistent edge over other structured procedures (Gene Rowe and George Wright).
  • Ask for probabilities, then score them. In 33 applications of Roger Cooke’s classical model, fewer than one third of individual experts were statistically accurate. Combinations weighted by performance on calibration questions were more accurate and more informative than equal weighting (Abigail Colson and Roger Cooke). Tetlock’s forecasting tournaments found training, teaming and tracking improved accuracy (Barbara Mellers and colleagues). A 2024 reanalysis found the effects shrank or even reversed in its best-fit models.
  • Take the outside view. Ask how comparable forecasts or projects turned out before you trust the inside story (Kahneman and Lovallo, Flyvbjerg). In practice: pull out the trend report from three years ago and check the calls.
  • Run a pre-mortem. Gary Klein’s method asks a team to assume the plan failed and explain why. The method builds on research on prospective hindsight (Deborah Mitchell, Jay Russo and Nancy Pennington). Explaining alternative outcomes also reduces hindsight bias (Roese and Vohs).
  • Remove conflicts, do not only disclose them (Cain, Loewenstein and Moore). Recruit a diverse group of experts instead of the most prominent ones (Hemming and colleagues).
  • Write neutral questions. Avoid agree/disagree formats (Krosnick) and numbers inside questions (Englich and colleagues). Train interviewers to spot cues of socially desirable answers and debrief after each session (Bergen and Labonté).
  • Triangulate. Combine methods, data sources, investigators and theories (Nancy Carter and colleagues). Use a second coder for interviews, peer debriefing and a documented decision trail (Helen Noble and Joanna Smith).

How to read the next trend report

Ask five questions before you act on a report.

  • Who paid for the report, and who sells what the report recommends?
  • How did the authors select the experts? By prominence or by diversity?
  • Did experts answer alone before any group discussion?
  • Are the forecasts probabilities with dates, or adjectives such as “soon” and “massive”?
  • Did the publisher score last year’s forecasts?

To sum up: expert opinion is evidence of what experts believe. Treat the opinion as evidence of the future only when the method earns the trust. Reports built on independent answers, probabilities and scored track records deserve your attention. Reports built on prominent voices and confident adjectives deserve your doubt. Evaluate the method before you follow the trend. FoMo is not a strategy.

If you like to dig deeper, we highly recommend “Superforecasting” by Philip Tetlock and Dan Gardner.

Sources

  • Bolger, F., Rowe, G. et al. (2023). Virtuous opinion change in structured groups. Judgment and Decision Making, 18, e25. https://doi.org/10.1017/jdm.2023.22
  • Satopää, V., Salikhov, M., Tetlock, P., Mellers, B. (2021). Decomposing the Effects of Crowd-Wisdom Aggregators: The Bias-Information-Noise (BIN) Model. SSRN. https://doi.org/10.2139/ssrn.3781405
  • Tetlock, P. E. (2005). Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press.
  • Tetlock, P. E. et al. (2014). Forecasting Tournaments: Tools for Increasing Transparency and Improving the Quality of Debate. Current Directions in Psychological Science. https://doi.org/10.1177/0963721414534257
  • Kahneman, D., Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515-526. https://doi.org/10.1037/a0016755
  • Roese, N. J., Vohs, K. D. (2012). Hindsight bias. Perspectives on Psychological Science, 411-426.
  • Hong, H., Kubik, J. D. (2003). Analyzing the Analysts: Career Concerns and Biased Earnings Forecasts. Journal of Finance, 58(1), 313-351. https://doi.org/10.1111/1540-6261.00526
  • Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., Bero, L. (2017). Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews, MR000033. https://doi.org/10.1002/14651858.MR000033.pub3
  • Cain, D. M., Loewenstein, G., Moore, D. A. (2005). The Dirt on Coming Clean: Perverse Effects of Disclosing Conflicts of Interest. Journal of Legal Studies, 34(1), 1-25. https://doi.org/10.1086/426699
  • Kahneman, D., Lovallo, D. (1993). Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking. Management Science, 39(1), 17-31. https://doi.org/10.1287/mnsc.39.1.17
  • Flyvbjerg, B. (2008). Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice. European Planning Studies, 16(1), 3-21. https://doi.org/10.1080/09654310701747936
  • Dedehayir, O., Steinert, M. (2016). The hype cycle model: A review and future directions. Technological Forecasting and Social Change, 108, 28-41. https://doi.org/10.1016/j.techfore.2016.04.005
  • Englich, B., Mussweiler, T., Strack, F. (2006). Playing dice with criminal sentences: The influence of irrelevant anchors on experts’ judicial decision making. Personality and Social Psychology Bulletin, 32(2), 188-200. https://doi.org/10.1177/0146167205282152
  • Franzen, A., Mader, S. (2023). The power of social influence: A replication and extension of the Asch experiment. PLOS ONE, 18(11), e0294325. https://doi.org/10.1371/journal.pone.0294325
  • Bikhchandani, S., Hirshleifer, D., Welch, I. (1992). A Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades. Journal of Political Economy, 100(5), 992-1026.
  • Bolger, F., Wright, G. (2011). Improving the Delphi process: Lessons from social psychological research. Technological Forecasting and Social Change, 78(9), 1500-1513. https://doi.org/10.1016/j.techfore.2011.07.007
  • Nickerson, R. S. (1998). Confirmation Bias: A Ubiquitous Phenomenon in Many Guises. Review of General Psychology, 2(2), 175-220. https://doi.org/10.1037/1089-2680.2.2.175
  • Smith, J., Noble, H. (2014). Bias in research. Evidence-Based Nursing, 17(4), 100-101. https://doi.org/10.1136/eb-2014-101946
  • Smith, J., Noble, H. (2025). Understanding sources of bias in research. Evidence-Based Nursing.
  • Noble, H., Smith, J. (2015). Issues of validity and reliability in qualitative research. Evidence-Based Nursing, 18(2), 34-35.
  • Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537-567. https://doi.org/10.1146/annurev.psych.50.1.537
  • Bergen, N., Labonté, R. (2020). “Everything Is Perfect, and We Have No Problems”: Detecting and Limiting Social Desirability Bias in Qualitative Research. Qualitative Health Research, 30(5), 783-792. https://doi.org/10.1177/1049732319889354
  • Rowe, G., Wright, G. (1999). The Delphi technique as a forecasting tool: issues and analysis. International Journal of Forecasting, 15(4), 353-375. https://doi.org/10.1016/S0169-2070(99)00018-7
  • Hanea, A. M. et al. (2017). Investigate Discuss Estimate Aggregate for structured expert judgement. International Journal of Forecasting, 33(1), 267-279.
  • Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F., Wintle, B. C. (2018). A practical guide to structured expert elicitation using the IDEA protocol. Methods in Ecology and Evolution, 9, 169-180. https://doi.org/10.1111/2041-210X.12857
  • Colson, A. R., Cooke, R. M. (2018). Expert elicitation: using the classical model to validate experts’ judgments. Review of Environmental Economics and Policy, 12(1), 113-132. https://doi.org/10.1093/reep/rex022
  • Mellers, B. et al. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 25(5), 1106-1115. https://doi.org/10.1177/0956797614524255
  • Rethinking the Role of Teams and Training in Geopolitical Forecasting: The Effect of Uncontrolled Method Variance on Statistical Conclusions (2024). Psychological Science. https://doi.org/10.1177/09567976241266481
  • Mitchell, D. J., Russo, J. E., Pennington, N. (1989). Back to the future: Temporal perspective in the explanation of events. Journal of Behavioral Decision Making, 2(1), 25-38. https://doi.org/10.1002/bdm.3960020103
  • Klein, G. (2007). Performing a Project Premortem. Harvard Business Review, September 2007.
  • Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41(5), 545-547. https://doi.org/10.1188/14.ONF.545-547
  • Tetlock, P. E., Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Crown.