Neutralizing misinformation through inoculation: the conclusion that could not be a conclusion

This is already the third post on the Neutralizing misinformation through inoculation paper of Cook, Lewandowsky and Ecker (2017). This post will focus on one paragraph in the general conclusions of this paper (my emphasis):

The ongoing focus on questioning the consensus, in concert with the gateway belief status of perceived consensus, underscores the importance of communicating the consensus [68,69]. However, positive consensus messaging is not sufficient, given recent findings that misinformation can undermine positive information about climate change [33,56]. As a complement to positive messages, inoculation interventions are an effective way to neutralize the influence of misinformation.

Although these are nice sounding conclusions, I have one problem with it: these could never ever be one of the conclusions that can be drawn from these two experiments described in the paper…

Experiment 1 looked at misinformation in the form of “false balance media coverage” and claimed that “the greatest effects were seen in perceived consensus” and were discussed on basis of that variable. As described in previous post, these were the findings (in ascending order of perceived consensus in a scale of 0 → 100):

,

Group Perceived consensus (mean)
Misinformation only 63.5
Control 68.9
Inoculation + Misinformation 70.0
Consensus + inoculation + Misinformation 83.9
Consensus + Misinformation 86.1

If we only look at that data, then I think we can conclude that:

  • The misinformation only message has a negative effect on the perceived consensus (which makes sense)
  • The consensus only message did just fine on its own (it showed the highest value of perceived consensus of all groups)
  • When a consensus message was combined with an inoculation message, it did a bit less well (but still high and the difference with a consensus only message could well be not significant)
  • The inoculation only message did remarkably worse than the consensus only message or the combined consensus/inoculation message (it was only marginally better than the control group – also here, that difference could well be not significant)

So, I am not really sure how they could reach the conclusion that a “positive consensus messaging is not sufficient” or “as a complement to positive messages, inoculation interventions are an effective way to neutralize the influence of misinformation” from the results of experiment 1. According to their own findings on false balance media coverage, the consensus message is more than sufficient and an inoculation message is found not to be necessary (or could even affect the consensus message negatively, depending whether the difference is significant or not).

I didn’t write about experiment 2 yet. In short, it investigated the misinformation in the form of “fake experts casting”. In this experiment the participants were split up into four groups: control, misinformation only, inoculation only and inoculation + misinformation. The results from the dependent variables were not found to be significant or only marginally significant. For example, if we want to compare with the results from experiment 1, the perceived consensus values looked like this (in ascending order of perceived consensus in a scale of 0 → 100):

Group Perceived consensus (mean)
Misinformation only 44.5
Inoculation only 50.4
Inoculation + Misinformation 51.6
Control 54.5

If we just compare with the previous experiment, it is clear that inoculation is hardly effective (the perceived consensus situated below the control group, somewhere in the middle of the misinformation group and the control group).

How did they then demonstrated the neutralization by inoculation in experiment 2? They managed to extract from the data that the misinformation had a polarizing effect on “climate attitudes” (people with low free-market support increased “climate acceptance”, while people with high free-market support decreased “climate acceptance”) and that inoculation apparently neutralized this effect. The inoculation + misinformation group showed “slightly less” polarization than the control group, demonstrating that the polarizing influence of misinformation was neutralized by the inoculation.

Whatever the case might be, it is clear that there was no group in which the participants received the consensus only message, nor was there a group that got a combined inoculation + consensus message, so no conclusions could be drawn about these two aspects on basis of the results of experiment 2 either.

These two statements are most probably the conclusions of the two referenced papers (numbered 33 and 56), but it could never be one of the conclusions of the “neutralizing misinformation through inoculation” paper. I would expect such statements in a “caveat” section or in the “discussion” section with the appropriate mention that this was not found in this paper, but they were put in the “conclusions” section of the paper, without even mentioning that this paper didn’t confirm the findings of those two referenced papers.

Concluding, unless I understood something completely wrong, the statements that a “positive consensus messaging is not sufficient” and “as a complement to positive messages, inoculation interventions are an effective way to neutralize the influence of misinformation” could never be one of the conclusions of this paper. After all, experiment 1 showed that the:

  1. consensus message on its own is more than enough (it performed best of all groups)
  2. combined inoculation + consensus message did not perform better than the consensus message alone (maybe even a tad less)

and experiment 2 didn’t research these two aspects at all.

Advertisements

2 thoughts on “Neutralizing misinformation through inoculation: the conclusion that could not be a conclusion

  1. tumbleweedstumbling

    Perhaps we are seeing a consequence of clever use of paywalls? Many people can not access scientific (and I use the term loosely) literature because of paywalls. Therefore, for many readers, especially in the general public, the abstract is all they actually see and read. You can put whatever you like in the abstract because it is only the abstract and the body of the article contains the “real” information. However, the abstract is the only part many people (especially journalists and the general public) ever read.

    Reply
    1. trustyetverify Post author

      This paper is open access. You can find it here:
      http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0175799
      You can read it or download it there, also (most of) the materials can be downloaded.

      But even if one read the complete paper, not all nuances will become immediately clear. I have read it numerous times by now, yet it was only after I went into detail what was done in both experiments, that I suddenly noticed that something wasn’t right with that paragraph in the conclusions section.

      I now realize that I only made a link to the paper in the first post of this series. It would be better if I made one in this post too. Done.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s