Scientific Consensus Gives Us Information Only If We Are Free to Doubt
There are so many credentialed people on the internet with sufficiently differing views that it sometimes seems as if we could find an expert somewhere to support almost any harebrained idea. So how does a non-expert figure out the truth?
Searching for a Valid Consensus
Most of us lack the time, training, and inclination to investigate most subjects sufficiently so we are often urged to adopt the consensus opinion. While an individual expert may have wild and crazy ideas, the consensus will most likely be an average informed view.
But it’s not that simple. Most of the time it is impossible for the public to determine the consensus opinion. What is usually labeled as consensus opinion is what media believe it to be. And the media are experts neither in the subject nor the sampling methodology needed.
But, even if we knew the consensus, a recent open access study put out by the AMISTAD lab at Harvey Mudd College in California points out an additional requirement for a valid consensus: The consensus must have been reached independently, and without coercion.
That seems intuitively right, of course. But it’s more than intuition. The researchers provided calculations, based on Bayes’ Theorem, to demonstrate the process. Dependence between experts (that is, when experts base their judgements on the judgement of another expert) or any sort of coercive treatment causes the relationship between consensus and truth to fall apart quickly.
This study starts off by showing that, given unfavorable observer conditions, an emerging consensus gives a strong signal in favor of the hypothesis. One of the issues, however, is that in many cases, even with strong evidence, no consensus will form. But when a consensus does form, it is very trustworthy.
However, as soon as dependence (whether through influence or coercion) is added into the model, even in small amounts, the ability of a consensus to serve as an indicator of truth drops considerably. A consensus can easily form around a false idea if there are dependencies between the participating individuals.
This study is in line with previous work on the subject, which shows that oftentimes observational judgements tend to average out towards truth — but only when the observations are independent of each other.
Listen to Your Baloney Detector
The results also mirror what professor Jay Richards has suggested about consensus. Using climate change as an example, Richards gives us a list of 12 things to think about to see if a consensus should be considered trustworthy. For example, he advises,
‘Science says’ is a weaselly claim. ‘Science,’ after all, is an abstract noun. It can’t speak. Whenever you see these phrases used to imply a consensus, it should trigger your baloney detector.
Jay Richards, “Politics Disguised as Science: When to Doubt a Scientific ‘Consensus’” at The Stream
Richards, like Allen et. al., suggests that the process by which the consensus formed is of critical importance. While not infallible, a consensus that is arrived at by independent testimony of independent experts should be considered good evidence. But when people inside or outside try to apply political muscle to the process at any point, the consensus fails to be meaningful.
Jonathan Bartlett is a Senior Fellow at the Walter Bradley Center for Natural & Artificial Intelligence. He is the author of several textbooks and edited volumes which have been used by universities as diverse as Princeton and DeVry.
For more breaking news about the interface of natural & artificial intelligence, visit MindMatters.AI.
Copyright 2021 Mind Matters.