Cooking a consensus: the symmetrical asymmetry

symmetry according to Cook

Looking at the consensus statements was a really interesting experience. It all started with viewing a video in which the consensus was preached and that got me interested what exactly those scientists agreed upon.

It led me first to the Oreskes survey that concluded on a rather uncontroversial statement and, unsurprisingly, came with a 100% consensus. Then came the Doran & Zimmerman survey that also took two rather uncontroversial statements and took an absurd low sample to come to their 97% consensus. Last post was about the Anderegg survey in which a database of scientists was created, those were divided according to their stance towards climate matters and concluded that those who were “unconvinced” by the “evidence” published less. How surprising đŸ˜‰

This post will be about the latest and biggest review so far of scientific opinion on climate change: “Quantifying the consensus on anthropogenic global warming in the scientific literature” from Cook et al in 2013. I have seen this survey before and wrote some posts about it. But this time I will focus on what they claim the consensus is all about.

To recapitulate: John Cook and fellow bloggers of Skeptical Science searched in March 2012 the ISI Web of Science for papers published from 1991 until 2011 using topic searches for global warming or global climate change. They restricted to articles with an abstract of 1,000 words or less. The search was updated in May 2012. From each article the title and abstract were shown to the volunteers who classified it according to category and endorsement level.

These are the results as I found them yesterday in their searchable database:

  Endorsement level Number
1 Explicitly endorses and quantifies AGW as 50+%
(paper explicitly states that humans are causing most of global warming)
2 Explicitly endorses but not quantify or minimize
(paper explicitly states humans are causing global warming or refers to anthropogenic global warming/climate change as a given fact)
3 Implicitly endorses AGW without minimizing it
(paper implies humans are causing global warming. E.g., research assumes greenhouse gases cause warming without explicitly stating humans are the cause)
4 No position
(paper doesn’t address or mention issue of what’s causing global warming)
5 Implicitly minimizes/reject AGW
(paper implies humans have had a minimal impact on global warming without saying so explicitly. E.g., proposing a natural mechanism is the main cause of global warming)
6 Explicitly minimizes/rejects AGW but does not quantify
(paper explicitly minimizes or rejects that humans are causing global warming)
7 Explicitly minimizes/rejects AGW as less than 50%
(paper explicitly states that humans are causing less than half of global warming)
  Total 12,280

At first glance this seems a nice symmetrical distribution. Level 1 is the strongest position, level 2 a bit less and level 3 even less. Followed by those who show no opinion. Finished by level 5 which implies minimizing/reject AGW, followed by level 6 which explicitly minimize/reject AGW and finally the clear rejecting it. Therefor it seems logical to exclude level 4 (because they have no opinion), add level 1 to 3 together and compare that with level 5 to 7. This results in 97.1% abstracts that agree (or 98%in the current list from the database).

It seems really beautiful: if you only want to consider quantifications, you could compare level 1 and 7. If you only want explicit endorsements, you can compare 1 and 2 vs. 6 and 7. If you only want implicit endorsements you could compare 3 and 5.

That is the theory…

The problem surfaces when one tries to figure out what those levels stand for. One can’t hide the fact that the distribution of the levels is not exactly symmetrical as it appears to be at first glance. The levels don’t necessarily follow each other. More, some levels overlap each other and not in a symmetrical way.

Level 1

According to the definition it is: “Explicitly endorses and quantifies AGW as 50+%”. This means from 50% until 100%. No doubts whatsoever here.

Level 2 and 3

Are a bit more tricky. These are the guidelines for rating abstracts in level 2:

2. Explicit Endorsement of AGW without quantification
2.1. Mention of anthropogenic global warming or anthropogenic climate change as a given fact.
2.2. Mention of increased CO2 leading to higher temperatures without including anthropogenic or reference to human influence/activity relegates to implicit endorsement.

That seems surreal. Just mentioning global warming as a given fact or stating that increased CO2 concentration lead to higher temperatures will put the abstract in level 2!

Now the guidelines for level 3:

3. Implicit Endorsement of AGW
3.1 Mitigation papers that examine GHG emission reduction or carbon sequestration, linking it to climate change
3.2 Climate modeling papers that talks about emission scenarios and subsequent warming or other climate impacts from increased CO2 in the abstract implicitly endorse that GHGs cause warming
3.3 Paleoclimate papers that link CO2 to climate change
3.4 Papers about climate policy (specifically mitigation of GHG emissions) unless they restrict their focus to non-GHG issues like CFC emissions in which case neutral
3.5 Modeling of increased CO2 effect on regional temperature – not explicitly saying global warming but implying warming from CO2
3.6 Endorsement of IPCC findings is usually an implicit endorsement. (updated this so it is more than just reference to IPCC but actual endorsement of IPCC)

This means simply acknowledging that human activity will cause some warming is enough to put it in level 3!

So it seems that level 2 and 3 will cover all abstracts which accept human activity as the cause for any amount of global warming. From negligible to important.

Level 4

Is a league on its own. Of those abstracts no opinion could be found in the abstracts or title.

Level 5 and 6

An abstract will only be categorized in those two levels if it rejects or minimizes anthropogenic global warming. Let’s look at the definitions: “minimize” means agreeing with anthropogenic global warming, but not seeing it as the dominant influence. Logically this means from 1 until 49%. Reject is rather simple, it means 0%. Just by looking at the definition the anthropogenic component will fall in the 0 – 49% range.

Level 7

According to the definition it is: “Explicitly minimizes/rejects AGW as less than 50%”. Rejecting AGW means zero percent. Minimising means above zero and below 50 percent. No doubts whatsoever here. But this means that in this category there are as well those who reject AGW as those who agree that humans cause some warming.

In practice this means that the levels are top heavy, they are biased towards level 2 and 3. The clear consensus of 97% is dependent on this construct. This doesn’t necessarily proves bad faith, but the methodology is likely skewing the results towards the top.

What did they assessed exactly? Cook et al tried to quantify the degree to which abstracts agree with the level of human influence on climate. But the methodology is fundamentally flawed because of the overlapping levels which benefits the warming part of the equation. Therefor only level 1, 5, 6 and 7 make sense. Level 4 shows no opinion and 2 – 3 can be in the complete range, so basically meaningless in determining which side is dominant. Ironically this means that if we try to validate the claim that humans are the primary cause of climate change, then there are only 65 abstracts that really support a strong position on anthropogenic global warming (0.5%). Of levels 2 and 3 we can not be sure, due to the ambiguous nature of their guidelines for those two levels.

The only thing that can be proven from all this is that 97% (from paper) or 98% (from the database) of the scientists agree that anthropogenic CO2 gives at least some warming and that is not really controversial.

But that is not how it is brought in the mainstream media. The most famous reaction was the tweet from president Obama (or rather the organization that manages his twitter account):

twitter 20130516 ninety-seven-percent

This statement is problamatic:

  • That “97% of the scientists think global warming is man made” is the big question: only 65 out of 12.280 (0.53%) stated unequivocally that global warming was more than 50% to be blamed on humans. The vast majority stated that at least some of the warming was to blame on us. The methodology was as such that it could be less than 50% as well as more than 50%. So this conclusion wasn’t really supported by this paper.
  • The statement “climate change is real” is ambiguous. Skeptics and alarmist could agree with this statement, but they would have each a different definition of what they mean by “climate change is real”…
  • And finally, do the scientists agree that it is dangerous? Well, to be very short, this question was not even asked in the survey!

The contrast between what was found in the paper and what was brought forward in the media couldn’t be much bigger.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s