At the end of August, Cliscep had an interesting post about a Google campaign against misinformation on social media based on (psychological) inoculation research. That post also pointed to some videos on the inoculation Science website. Looking around this website, I also found a link to the COVID-19 Vaccine Communication Handbook in the “Inoculation explained” tab. The handbook was a collaboration between several (social science) researchers among whom Stephan Lewandowsky and John Cook as two of the lead authors. That promises to be interesting, I wonder what tricks they came up with this time.
Going through the handbook, trust in scientists seems to be an important element in the handbook and this infobox at the bottom of page 4 immediately caught my eye:
Trust in scientists increases:
Surveys in several countries have shown trust in scientists to increase.
In Germany, the share of people who completely trust scientists
doubled between 2019 and November 2020, and around 70% of the
public trust scientists. In the U.K., 64% of respondents indicated in
April 2020 that the pandemic had made them more likely to listen to
scientists and researchers.
It caught my attention because, by using the present tense, the title seems to suggest that trust is on the increase (at least at the time the handbook came out). I can certainly agree that trust in scientists surged at the beginning of the pandemic, but as far as I know, that initial trust was already declining for quite a while by the time that the handbook was released (January 7, 2021). Therefor it didn’t make much sense to me that trust in scientists was still increasing at that time and it made me wonder what evidence the handbook provides to support this position.
Let’s first look at the three claims made in that infobox:
- The share of people who completely trust scientists doubled between 2019 and November 2020 (Germany)
- The share of people who trust scientists is around 70% (Germany)
- 64% of respondents indicated that the pandemic made them more likely to listen to scientists and researchers (UK)
The second and third claims have supporting links that go to results of polls conducted in April and May of 2020. That however was the beginning of the pandemic when trust surged, so that would not be evidence of a long-term increase.
The first claim is the most promising of the three. It shows that the level of trust was still high more than half a year into the pandemic, but unfortunately that is the only claim of the three for which no link to supporting data is provided…
There is another link placed below that infobox that goes to a wiki page on “trust in scientists” and this page has the interesting subtitle “the contextual and ephemeral nature of trust”. Its first paragraph starts with roughly the same information as the infobox, but there were some extra details (I highlighted them in green):
Trust in scientists has in general increased during the pandemic in several countries, despite claims and concerns about decreasing trust. For example, according to a survey administered via the Corona Spezial Science Barometer, the share of Germans who completely trust scientists doubled between 2019 and November mid-2020, and around 70% of the German public now trust scientists. Similarly, in the United Kingdom, the Open Knowledge Foundation found that 64% of poll respondents were more likely to listen to expert advice from qualified scientists and researchers.
That first sentence is rather puzzling, especially that “despite claims and concerns about decreasing trust” part. I understand that there were claims and concerns about declining trust, but it is not clear whether these claims and concerns were founded, meaning whether or not this decrease actually took place according to the authors. The “in general” part of the sentence and the “ephemeral nature of trust” in the subtitle seem to suggest that those claims and concerns were indeed founded, so my take from this sentence is that there was a decline of trust during the pandemic, but that the trust level in November 2020 was still higher than in 2019. Therefor an increase relative to 2019.
I think that is much more plausible than trust still on the increase, but it now raises the question of how much did trust in scientists decline during the pandemic.
Unfortunately, also here, no link to that November poll is provided. The two links that were provided in that paragraph were the same links that were also in the infobox.
Then follow the results of two more polls about confidence in medical scientists in the United States. Unsurprisingly, also these polls were conducted at the end of April (National Survey) and the beginning of May (Pew research), so are also from the beginning of the pandemic.
Basically, to support their claim that trust in scientists “in general” increased “during the pandemic”, they only provided links to polls conducted at the beginning of the pandemic when trust levels increased rapidly, but not to the data that would show where the trust level finally ended up in November…
Fine, if they don’t provide a link to the November poll, then I can find it on my own. Now I had some more details that I can use to find that poll. From the placement of the link to the German polls in that paragraph, I understand that these are both related to the “Corona Spezial Science Barometer”. However, I already visited the Corona special edition of the Science barometer (Wissenschaftbarometer) and that web page only has information about polls conducted in April 2020 and May 2020. That made me think that there probably also might be a November poll conducted by the Wissenschaftbarometer and it is this page that I need to find in order to figure out the evolution of trust in scientists during the pandemic. Luckily, I quickly found the November 2020 Wissenschaftbarometer press release with also links to their November poll brochure and the data.
This is the result of the question “How much do you trust science and research?” (yellow emphasis by the Wissenschaftbarometer):
The red bar is the share of people completely trusting the science and that share went from 9% in 2019 to 20% in November 2020. That is indeed a doubling relative to 2019. What however wasn’t mentioned in the handbook is that this share first quadrupled in April, then decreased to less than 3.5 times just one month later and finally ended up at just a doubling in November.
That is why I think their title “Trust in scientists increases” is rather misleading. By using the present tense, it might be wrongly perceived as trust in scientists still increasing (at the time of the release of the handbook), while the last two datapoints (April 2020 and November 2020) showed a steadily decreasing trend.
Now let’s look at the claim that around 70% of the public trust scientists. This number is the sum of those who trust the scientists completely (red) and those who rather trust them (orange). That was 36 + 37 = 73% in April and went down to 31 + 35 = 66% in May. That is indeed around 70% as they claimed.
However, in November trust in scientists was 20 + 40 = 60%. One needs somewhat more imagination to call that “around 70%”. Do I understand it correctly that the “around 70%” claim is only for the April and May polls? That is not clear from the infobox in the Handbook, it only becomes clear when one finds the November data oneself and then make the sums.
Concluding, I agree that the November data shows that the share of those who completely trust scientists doubled since 2019, but that is only part of the story and it is now not really a surprise to me that the authors of the handbook didn’t provide a link to supporting data for that claim. If they would have done that, then their readers would be able to easily check out the November data themselves and the authors would have some explaining to do why almost half of those who initially completely trusted scientists in April lost that complete trust by November…
Trust, yet verify 