This is chapter nine from Donna LaFramboise's book
The Delinquent Teenager: "The Immense Edifice That Wasn't"
For an introduction explaining why I'm reviewing this piece of work, please click here.
{Courier font identifies LaFramboise's words
Laframboise, (2011-10-09). T D T W W M W T C E (Kindle Locations 195-201). Ivy Avenue Press. Kindle Edition. }
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
9 - The Immense Edifice That Wasn't Many people believe the IPCC goes to the trouble of verifying the research on which it bases its conclusions. An oft-repeated quote from President Barack Obama's science advisor, John Holdren, is a marvelous example of this. Holdren says the IPCC is the source of "the most important conclusions" about climate change, and that these conclusions rest on:
...an immense edifice of painstaking studies published in the world's leading peer-reviewed scientific journals. They have been vetted and documented in excruciating detail by the largest, longest, costliest, most international, most interdisciplinary, and most thorough formal review of a scientific topic ever conducted.
Donna, you fancy yourself an investigative journalist, yet your always condemning and insinuating incompetence along with dishonesty. What's up with that?
This chapter again you make slanderous charges, you drape your words with insinuations right and left. To try to create a cartoon and ignore all of this:
This document captures the writing teams for the IPCC Working Groups I, II, and III contributions to the IPCC Fifth Assessment Report (AR5). Names, roles, and countries of residence are provided, as extracted from the AR5 nominations database, except in the case of experts that work for international organizations or institutes. For the most up to date list for each Working Group, please consult the individual Working Group Website.
Go ahead investigate this link of IPCC authors, review the names that goes on for dozens of pages. It's easy enough to track down their academic/professional records. Show us who is incompetent. As for process, here's a link to the IPCC calendar:
Why does Donna disregard all of that? I myself believe it's because to her the IPCC is a political enemy to defeat.
Here's a similar quote, from climate modeler Richard Rood:
The scientists who write the IPCC reports use exquisite rigor...the result is a document which is based on the facts...which have been scrutinized to the highest level possible.
But as we have discovered, the IPCC takes research findings at face value. It doesn't double-check that the raw data actually shows what a researcher claims it does. It feels no need to look under the hood - and discourages its expert reviewers from doing so.
"IPCC takes research findings at face value" The IPCC is there to collect and inventory the scientific literature. Donna is demanding that IPCC approach peer reviewed studies as untrustworthy. Donna want every data point to be verified.
Come on can we think about that. This is crazy-making.
The notion is impossible.
Donna would demand that the IPCC hires hundreds of reviewers and add whole new layers of bureaucracy to the process.
... Holdren and Rood are therefore mistaken. The IPCC does not scrutinize the facts on which it relies. It performs no vetting whatsoever - never mind the sort that could be described as excruciatingly detailed.
No organizing charter ever mandated the IPCC with auditing every study it processes!
Donna has fabricated another straw-man.
When IPCC insiders were asked for their thoughts about quality assurance, their questionnaire answers confirmed this. Here are some of their verbatim remarks:
As far as I can tell, there is no data quality assurance associated with what the IPCC is doing… (p. 99) Since the IPCC is a review body, it does not do data assurance or quality control in a systematic fashion.(p. 52) Quality assurance and error identification is not existent… (p. 384) Data quality assurance, per se, is beyond the scope of the work of the IPCC… (p. 203)
- - -
7. What is your view of how IPCC handles data quality assurance and quality control and identification and rectification of errors, including those discovered after publication? Data quality assurance, per se, is beyond the scope of the work of the IPCC since its job is to assess the science.
That said, it does have the task of assessing the scientific methods that are used to develop a given data product so as to determine whether we should have confidence in a given estimate, of say, the global mean temperature anomaly. Thus it must inter-compare results that are obtained using different data sources and procedures, and must also assess the reliability of different data sources. But it cannot undertake data quality assurance itself.
- - -
Many of these individuals said the IPCC should not be held responsible for the accuracy of statements that appear in research papers it cites since "that is an issue for the journals concerned." In the words of someone else, "it is expected that a paper published in an important journal" has already received a quality assurance check.
What a refreshing piece of journalism from Donna, acknowledging the other side. And, it's a point worth keeping in mind.
Other IPCC insiders, however, recognize the shortcomings of this approach. There are thousands of journals out there, but no accreditation process to ensure their quality.
"No accreditation process," Donna willfully ignores that scientific quality assurance is achieved by the community of engaged scientists looking at and discussing these studies.
This is a public process (that is within the community of scientists who can follow the discussion) and when there are problems with studies they get weeded out by this process, which is how science has operated for hundreds of years.
What Donna's trying to do is shift the goal so that every paper is treated as false until re-proven accurate.
Also, let's keep in mind what she's demanding would require an phenomenal increase in cash, manpower and additional layer of cumbersome bureaucracy - something that no one is willing to do. Something that would create it's own problems.
How smart is it, therefore, to blindly assume that a published paper is an accurate paper? As one person observed, some research merely makes an interesting contribution to the 'intellectual conversation' (p. 332). That standard is surely far too low to justify an IPCC conclusion.
I think Donna is hoping we blindly buy into her paranoia.
Her interpretation of those words has more to do with the agenda she's peddling here than with what the respondent was actually saying. There's much more nuance going on there, you can decide:
7. What is your view of how IPCC handles data quality assurance and quality control and identification and rectification of errors, including those discovered after publication?Through the processes of successive refinement and abstraction, certain themes or ideas gain prominence and greater importance. The higher level of influence of ideas taken as leitmotivs in the later stages should require a higher level of scrutiny. Was this done in previous reviews. The level of confidence of a normal scientific paper put into the journal literature as part of the ongoing intellectual conversation is not necessarily written to the same level of rigour required to justify the weight of an IPCC major conclusion.
Reverse quality assurance to the level of the original articles should be carried out for major IPCC findings, by IPCC scientists other than those responsible for forward progression of the finding.
Here again, it seems like pie in the sky talk.
Who is going to finance this huge new layer of professional reviewers for every important paper the IPCC references?
How many more opportunities for mischief and crazy-making will those politicized positions open up?
Once we understand a few other relevant facts, Holdren's claimed edifice crumbles entirely. Academic journals make use of a quality-control mechanism called peer review. The general idea is that, when a paper is submitted to a journal in the hope that it will be published, it gets assigned to an employee of the journal called an editor. The editor sends copies of the paper to reviewers presumed to be knowledgeable about the topic under discussion.
Generally speaking, there are three reviewers whose identities remain anonymous even after the paper is published. Sometimes these reviewers are called referees. Although the reviewers look over the paper it is important to appreciate that, in many cases, only the most cursory of assessments takes place. In the words of one senior scientist:
A reviewer is normally not paid for his work. With the best will in the world, he is able to spend no more than a few hours examining any particular manuscript. He is able to do little more than see that the story being told is superficially coherent and makes no obvious errors of fact.
Oh boy, now Donna is introducing the conspiracy of laziness.
Reviewers are superficial and can't be trusted to fulfill their duties. Does she show us incidents, no.
All we are given is Donna's paranoia supported by what-ifs" and "it-coulds."
If the reviewers have concerns, they tell the editor about them - who then asks the paper's author for a response. Sometimes a paper will undergo a major re-write before the editor is satisfied it is fit for publication. But this does not mean its conclusions are correct. Far from it. Richard Horton, the editor of The Lancet medical journal, argues this point forcefully:
Peer review does not prove that a piece of research is true. The best it can do is say that, on the basis of a written account of what was done and some interrogation of the authors, the research seems on the face of it to be acceptable for publication…Experience shows, for example, that peer review is an extremely unreliable way to detect research misconduct.
A couple reviewers, of course, are a poor substitute for mass scrutiny. Sometimes reviewers are chosen poorly; other times they're lazy. ...
This is where Donna forget's to tell us about the community of competitive professionals who read these reports and can balance them agains their own knowledge. People who are ready to point out flaws, to correct and supersede others work. That's how science functions and it's not perfect but it has served society quite well.
Conflating peer review with scientific soundness impoverishes our appreciation of the scientific process. Peer review should be one criterion that people use in assessing the strength of any given piece of research – nothing more, nothing less.
What this all adds up to is that the only time research findings can be considered valid is if someone else, working entirely independently, follows the same procedures as those described in the paper and arrives at the same result.
This simply does not ring true!
There are plenty of ways to cross-validate someone's work without necessarily copying their original experiment.
Donna's suggestion would be forcing scientific knowledge to reproduction via cloning. Works great for what it's worth, but doesn't get you very far in the evolutionary race.
There used to be perfect clarity in the scientific community that unless a piece of research had passed that kind of test, it should be viewed with caution. Based on McIntyre's experience with the two unpublished papers discussed above, it appears the IPCC now regards research as reliable long before it has even appeared in print.
what "perfect clarity?" When was this golden era?
You know, this sounds like something you said because it sounds poetic and fits your plot. But, it doesn't have anything to do with the history of how science actually functions and moves forward.
Or will you show us what you base your claim on, with more that "could-bes" and "what-ifs"?
When one remembers that a great deal of climate research involves computer modeling (employing millions of lines of computer code), there's another reason for concern. As geophysics professor Jon Claerbout points out:
An article about computational science in a scientific publication isn't the scholarship itself, it's merely advertising...The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures.
Peer-reviewers don't get within a mile of climate modeling supercomputers and their software. Which means they have no realistic way of evaluating entire categories of research papers that are central to the IPCC's analysis. All a peer-reviewer can do is assess the advertising - the portions of the story the climate modeler chooses to discuss in his or her paper.
Here again Donna's logic only works if you are working from the premise that every scientist is to be distrusted and that nothing less than absolute perfection is acceptable.
As for her claim that scientists have no realistic ways to evaluate models... I don't buy that one either. Scientific models have been used extensively and successfully throughout modern scientific times.
Spend some time looking at some of the links I share below.
Phil Jones, one of the world's most prominent climatologists, has published in the most prestigious journals. When he testified before a UK parliamentary committee early in 2010, he was asked how often peer reviewers had sought to examine his raw data and computer codes. "They've never asked," he replied.
While we're on the subject of quality assurance, IPCC insiders who answered the questionnaire identified another weak link. A great deal of climate research involves huge collections of data - such as temperature records from thousands of locations stretching back scores of years. But the accuracy of these numbers has never been verified by independent personnel. As one IPCC insider observed, academic journals may consider unverified data good enough - but quality control mechanisms surely need to be in place before the IPCC relies on such data to make real-world decisions.
Here we are back in the land of erecting impossible standards.
Worse than that, Donna is suggesting our scientists are so unreliable that every measurement ever written down needs to be re-verified. This is crazy-making. Who's going to do the reverifying? Hobbyists who don't want to believe anything regarding man-made global warming to begin with?
Has Donna presented any examples, or incidents, that demonstrate a need for such extreme measures? I'm still waiting.
Let us return to that quote from presidential advisor John Holdren. He says the IPCC's conclusions are the result of the most thorough formal review of a scientific topic ever conducted. How can this be the case when the IPCC hasn't bothered to verify the temperature data on which so much of climate science rests?
Would an auditor approve a company's financial statements before confirming the accuracy of the underlying numbers?
Bring on the melodrama, assume the worst from scientists and demand new impossible standards. That's not the stuff of a investigative journalist, that's propaganda.
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Below are a number of links to sources that talk about climate models; how they work; their strengths; limitations; usefulness. Some are quite basic and other's provide higher level details along with links to further investigation:
ANNEX 2 VERIFICATION A2.1 INTRODUCTION
Verification processes are, in the present context, intended to help establish an inventory’s reliability. These processes may be applied at either national or global levels of aggregation and may provide alternative information on annual emissions and trends. The results of verification processes may:
- (i) Provide inputs to improve inventories;
- (ii) Build confidence in emissions estimates and trends;
- (iii) Help to improve scientific understanding related to emissions inventories.
Verification processes may also enhance international cooperation in improving inventory estimates.
There are different approaches to verification. One approach is to evaluate emissions estimates and trends, for example, as part of the United Nations Framework Convention on Climate Change (UNFCCC) review of emissions inventories. Another approach entails an evaluation of aggregate inventories on a global or regional basis, with the objective of providing further scientific insight.
A number of options or tools for verification are discussed in this Annex. Their application, as well as the types of information needed, will vary according to the role and intention of the verification process. . .
The Physics of Climate Modeling
By Gavin A. Schmidt, January 2007
National Academy of Sciences ~ Climate Modeling 101
ABSTRACT: "Testing Climate Models: An Approach"
Richard Goody, James Anderson, and Gerald North
The scientific merit of decadal climate projections can only be established by means of comparisons with observa- tions. Testing of models that are used to predict climate change is of such importance that no single approach will pro- vide the necessary basis to analyze systematic errors and to withstand critical analysis.
Appropriate observing systems must be relevant, global, precise, and calibratable against absolute standards. This paper describes two systems that satisfy these criteria: spectrometers that can measure thermal brightness temperatures with an absolute accuracy of 0.1 K and a spectral resolution of 1 cm−1, and radio occultation measurements of refractiv- ity using satellites of the GPS positioning system, which give data of similar accuracy.
Comparison between observations and model predictions requires an array of carefully posed tests. There are at least two ways in which either of these data systems can be used to provide strict, objective tests of climate models. The first looks for the emergence from the natural variability of a predicted climate “fingerprint” in data taken on different occasions. The second involves the use of high-order statistics to test those interactions that drive the climate system toward a steady state. A correct representation of these interactions is essential for a credible climate model.
A set of climate model tests is presented based upon these observational and theoretical ideas. It is an approach that emphasizes accuracy, exposes systematic errors, and is focused and of low cost. It offers a realistic hope for resolving some of the contentious arguments about global change.
How reliable are climate models?
Predicting changes: Testing climate model accuracy
Here's an interesting video, it's a decade old, so please remember there have been many advances, but still for a general orientation and for the hearing out of a scientist's mouth - it's nice:
40/40 Vision: Richard Somerville - Can Climate Models Be Trusted?
UCTV - Date: 5/2/2001 - 59 minutes
As we enter a new millennium, we have committed our children and grandchildren to a world of rising sea level, melting ice caps, and disrupted weather patterns. Are the computer models that tell this story reliable? How strong is the evidence for the scenarios they produce? How should policy makers and the public react to climate model forecasts? Join renowned meteorologist Richard Somerville as he presents a comprehensive overview of the climate change issue, ranging from the newest research results to the ongoing diplomatic and political debates sparked by climate science. (#5378)
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
