Leave a known bias in the data or correct the data for the bias: a false dilemma

The Karl et al paper left me with more questions than answers. They used low quality, scarce, spatially incomplete data and it were their assumptions that made the difference, yet they seem to give it such a high importance that, even when contradicted by high quality data with better spatial coverage, they seem to be sure that their conclusion is relevant?!?! Other articles didn’t seem to be bothered by it, they just focused on the conclusion and pitied the “deniers” who again had to take yet another blow. This was no different for an article at Dailykos with the fascinating title As climate denier heads explode over the loss of the “hiatus”, one simple question shuts them up about a response from Tom Peterson to an email of Antony Watts. It was amusing to read that the author of the Dailykos artice thinks that “deniers” “lost” the hiatus, while it is still clearly visible in all other datasets. Beyond the hyperbole there was some insight of a scientist who actually contributed to the papern, so I could see how one of the authors of the paper justifies coming to this conclusion with such data.

This is the part where he explains it:

So let me give you two examples from our paper. One of the new adjustments we are applying is extending the corrections to ship data, based on information derived from night marine air temperatures, up to the present (we had previously stopped in the 1940s). As we write in the article’s on-line supplement, “This correction cools the ship data a bit more in 1998-2000 than it does in the later years, which thereby adds to the warming trend. To evaluate the robustness of this correction, trends of the corrected and uncorrected ship data were compared to co-located buoy data without the offset added. As the buoy data did not include the offset the buoy data are independent of the ship data. The trend of uncorrected ship minus buoy data was -0.066°C dec-1 while the trend in corrected ship minus buoy data was -0.002°C dec-1. This close agreement in the trend of the corrected ship data indicates that these time dependent ship adjustments did indeed correct an artifact in ship data impacting the trend over this hiatus period.”

The second example I will pose as a question. We tested the difference between buoys and ships by comparing all the co-located ship and buoy data available in the entire world. The result was that buoy data averaged 0.12 degrees C colder than the ships. We also know that the number of buoys has dramatically increased over the last several decades. Adding more colder observations in recent years can’t help but add a cool bias to the raw data. What would you recommend we do about it? Leave a known bias in the data or correct the data for the bias? The resulting trend would be the same whether we added 0.12 C to all buoy data or subtracted 0.12 C from all ship data.

That second example was the question that the author of the Dailykos article alluded to (and is also the subtitle): “What would you recommend we do about it? Leave a known bias in the data or correct the data for the bias?”. At first glance, it sounds reasonable, but I think it is a false dilemma. It leaves us with the apparent choice of:

  1. leave the known bias into the equation and get a wrong result
  2. correct the bias and get a correct result.

Option one is an obvious no no. If one is sure there is a bias, there is nothing wrong with trying to adjust it (when the strength of the bias is known). So, option two seems the only real choice and following that the result doesn’t support the “pause”…

But is this the real choice we have? I think it is the wrong question altogether, knowing that the conclusion depended most on the adjustments of those sea surface temperatures.

Those two options can only be relevant to the question about the existence of the “pause” IF it is possible to reliably determine global sea surface temperatures from the measured temperatures from buckets, intake water or buoys. If that is true, those two options are perfectly valid options and option two would be the correct one.

If that is not true, then whether the bias is fixed or not has no relevance for determining whether the pause is real or not. And wasn’t that what they wanted to demonstrate with this paper?

What is it with their fascination for incomplete, scarce datasets? And why give those so much importance, even in the face of the (more reliable) measurements?

I can imagine journalists or activists are falling for this, but it is mind boggling that a scientist who actually worked on this paper presents such a weak argument…

4 thoughts on “Leave a known bias in the data or correct the data for the bias: a false dilemma

  1. manicbeancounter

    This comment intrigues me

    The resulting trend would be the same whether we added 0.12 C to all buoy data or subtracted 0.12 C from all ship data.

    If that was the only adjustment then it would not make any difference. So adjusting recent more accurate data upwards (what did happen) would not make any difference to correcting downwards for the bias in the less accurate data. But logically past data should have been adjusted downwards. This latter is also consistent with the adjustment policy for land temperature anomalies. That is assume that the current data is correct, and adjust the past. This despite the measurement bias due to UHI being that more recent data having a warming bias. Again that would not matter if this was the adjustment policy was always to adjust assuming the present data was correct. But if for land we assume that the present data is correct (despite it is the past data that has less bias), so illogically cool the past and for the oceans assume that past data is correct (despite knowing that recent data is more accurate) and illogically warming the present data, are we not applying contradictory adjustment policies, with the net effect of showing more recent warming than has actually occurred?


    1. trustyetverify Post author

      There were indeed more adjustments than just the 0.12 °C from the buoy/ship data and the adjustments of ocean data were both upward and downward. There was the adjustment derived from night marine air temperatures data that adjusted the ship temperatures before the “pause” downwards and the comparison ship/bouy data that adjusted temperature upwards in the “pause”, therefor straighting it out, in order to become an almost linear trend.

      I don’t know what it would change if the same adjustment policies would/could be applied for both land and ocean data, but I do know that their conclusion was due to a change in assumptions on basis of incomplete data and in contradiction with the current observations of other datasets. Not only surface datasets, but also satellite datasets (that assumed to be of higher quality).

      This makes it hard for me to accept their conclusion.


  2. poitsplace

    Yes, the unseen choice was in using the shitty dataset. This is a staple with the catastrophic AGW “scientists”…stitch together whatever dataset gives them the best warming trend and then treat it as if that’s where the data led them. Never mind that it ignores satellite data or at some point an entire trend in a proxy is based on just a couple trees.

    They are just a part of a green cult. Never mind the evidence…we know the warming has to be there…we just have to find it and then we’ll know we have the “right” answer because it will show that warming. It doesn’t matter if it’s these pseudoscientists or John Cook saying “the warming is totally there, it’s just covered by ENSO activity” and then conveniently ignoring the fact that AMO/PDO activity did the exact same thing during the period often used to calculate the anomalies (roughly 1950-1980)


    1. trustyetverify Post author

      This was so obvious because it contradicted every other dataset and it was really clear Karl and his group cherry picked the least reliable sources and went from there. In most cases it is much less obvious.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s