A walk down memory lane in honor of the Paris Climate Conference, (well and also for my pal AL with his hate-on for Dr. Mann and rejection of climate proxy work in general as described in the comments thread at Prof Ivar Giaever's pathetic YouTube PseudoScience talk), I'm going to reprint a few posts I'd originally written for my Citizenschallenge Blog a couple years back since the information they hold is as valid and relevant today as it was then.
_________________________________________
Here is an interesting letter that Dr. Mann wrote in response to a series of questions he received from the ranking members of the House Committee on Energy and Commerce, Oversight and Investigations committee.
The sad thing is that in hindsight it seems as though both US Representative Whitfield and US Representative Barton didn't read a single word of this letter and instead continued steamrolling their personal agenda of attacking the science and ignoring the evidence.
I reprint it complete,
though I have taken the liberty of adding paragraph breaks and highlights.
===================================================
July 15, 2005
From:
Michael E. Mann, Ph.D.
Associate Professor and
Director of Earth System Science Center Department of Meteorology
The Pennsylvania State University
To:
Joe Barton, Chairman
House Committee on Energy and Commerce
Ed Whitfield, Chairman
Subcommittee on Oversight and Investigations
2125 Rayburn House Office Building Washington, D.C. 20515
Dear Chairman Barton and Chairman Whitfield:
From:
Michael E. Mann, Ph.D.
Associate Professor and
Director of Earth System Science Center Department of Meteorology
The Pennsylvania State University
To:
Joe Barton, Chairman
House Committee on Energy and Commerce
Ed Whitfield, Chairman
Subcommittee on Oversight and Investigations
2125 Rayburn House Office Building Washington, D.C. 20515
Dear Chairman Barton and Chairman Whitfield:
This letter responds to your letter of June 23, 2005, which seeks information on
issues relating to my research on the historical record of temperatures and climate
change.1 Your letter lays out a number of “concerns” about the research my colleagues
and I have conducted about global warming. Your letter also inquires about the role I
played in the preparation of the United Nation’s Intergovernmental Panel on Climate
Change Third Assessment Report (the so-called “TAR”).
I will address each of your questions in turn. Before doing so, however, let me state that my research findings, which support the conclusion that the earth’s surface is warming, and that recent warming is due in large part to human influences, are consistent with the overwhelming scientific consensus on climate change. My research has been subject to intensive peer-review. Other scientists have replicated all facets of my research and have found it accurate and reliable. The specific conclusion published by my colleagues and me that late 20th century Northern Hemisphere warmth is anomalous in the context of at least the past millennium is common to many studies.
Based on multiple supporting studies, the TAR came to a similar conclusion. The TAR did not rely solely on the work of my colleagues and me in reaching this conclusion. Recent work since the TAR has provided further support for this conclusion, which is now common to more than a dozen independent studies published in the peer-reviewed scientific literature. (I have provided for reference a comprehensive review by Jones and Mann in the journal “Reviews of Geophysics” of the American Geophysical Union (AGU).) The criticisms your letter cites have been soundly rejected by the scientific community.
I will address each of your questions in turn. Before doing so, however, let me state that my research findings, which support the conclusion that the earth’s surface is warming, and that recent warming is due in large part to human influences, are consistent with the overwhelming scientific consensus on climate change. My research has been subject to intensive peer-review. Other scientists have replicated all facets of my research and have found it accurate and reliable. The specific conclusion published by my colleagues and me that late 20th century Northern Hemisphere warmth is anomalous in the context of at least the past millennium is common to many studies.
Based on multiple supporting studies, the TAR came to a similar conclusion. The TAR did not rely solely on the work of my colleagues and me in reaching this conclusion. Recent work since the TAR has provided further support for this conclusion, which is now common to more than a dozen independent studies published in the peer-reviewed scientific literature. (I have provided for reference a comprehensive review by Jones and Mann in the journal “Reviews of Geophysics” of the American Geophysical Union (AGU).) The criticisms your letter cites have been soundly rejected by the scientific community.
The most serious contention in your letter — namely, that my work has not been subject to replication because I have failed to make available the underlying research data — is incorrect. Your letter notes that the National Research Council’s “gold standard” for scientific research is the ability of other scientists to replicate first-generation research, and I fully agree.
My colleagues and I follow the National Research Council’s guidance with regards to the disclosure of research data, and all of our data and methodologies have been fully disclosed and are available to anyone with a computer and an internet connection.
As a result of our willingness to share our research with others, an independent team of scientists has used the research data my colleagues and I have made public to replicate our research and confirm the reliability of our findings. See Wahl, E.R., Ammann, C.M., Robustness of the Mann, Bradley, Hughes Reconstruction of Surface Temperatures: Examination of Criticisms Based on the Nature and Processing of Proxy Climate Evidence, Climate Change (2005) (forthcoming) and associated website:
http://www.cgd.ucar.edu/ccr/ammann/millennium/refs/Wahl_ClimChange2007.pdf.
Let me now turn to your specific questions, which ask that I provide the following information:
Question 1: Your letter first asks that I furnish the Committee my curriculum vitae, along with a “list of all studies relating to climate change research for which you were an author or co-author and the source of funding for those studies.”
Answer: This material is attached.
Q2: Your letter next asks that I “[l]ist all financial support” I have received to
support my research.
A: See attachment.
Q3: Your letter requests that I provide, for all “work involving federal grants or funding support under which you were a recipient of funding or a principal investigator,” “all agreements relating to those underlying grants or funding, including, but not limited to, any provisions, adjustments, or exceptions made in the agreement relating to the sharing of research results.”
A: These requests are not directed to the appropriate person. The committee should contact the University of Massachusetts and University of Virginia offices of grant administration for these materials. With respect to the UMass NSF research funds (which supported the 1998 Nature article), it should furthermore be noted that I was not the Principal Investigator for this NSF project, and I am not, nor have I ever, been in possession of any official paperwork related to this grant.
Q4: Your next question asks for “the location of all data archives relating to each published study for which” I was “an author or co-author” and whether such data would be sufficient to permit other researchers to replicate the work.
page 2
A: The data, descriptions of methods, and results related to my research — more
than sufficient to permit other researchers to replicate the research — have been
extensively archived (in many cases, in several archives) on public websites, and data
links within the websites. The website addresses appear in the margin.2
Q5: This question begins by stating that, “[a]ccording to The Wall Street Journal, you have declined to release the exact computer code you used to generate your results.” The question then poses a series of questions: “(A) Is that correct? (B) What policy on (page 3) sharing research and methods do you follow? (C) What is the source of that policy? (D) Provide this exact computer code used to generate your results.”
Q5: This question begins by stating that, “[a]ccording to The Wall Street Journal, you have declined to release the exact computer code you used to generate your results.” The question then poses a series of questions: “(A) Is that correct? (B) What policy on (page 3) sharing research and methods do you follow? (C) What is the source of that policy? (D) Provide this exact computer code used to generate your results.”
The question presumes that in order to replicate scientific research, a second researcher has to have access to exactly the same computer program (or “code”) as the initial researcher. This premise is false.
The key to replicability is unfettered access to all of the underlying data and methodologies used by the first researcher. My data and methodological information, and that of my colleagues, are available to anyone who wants them.3 As noted above, other scientists have reproduced our results based on publicly available information.
It also bears emphasis that my computer program is a private piece of intellectual property, as the National Science Foundation and its lawyers recognized. The National Science Foundation — the government agency that establishes policy in this area — has confirmed that my colleagues and I have met every requirement of transparency and openness in our research. My research is all based on data sets regarding the Earth’s (page 4) climate that are freely and widely available to all researchers. Whether I make available my computer programs is irrelevant to whether our results can be reproduced. And whether I make my computer programs publicly available or not is a decision that is mine alone to make.
Since other scientists have used the methods we described and the data we archived to replicate our results, the issue of whether my computer program is available has no bearing whatsoever on the veracity of our results. The question you posed — whether I have fully satisfied established scientific standards for data-sharing — has been fully considered by the National Science Foundation. As your letter notes, two Canadian researchers, Steve McIntyre and Ross McKitrick, contacted NSF to inquire whether I had complied with National Science Foundation requirements.
The National Science Foundation twice informed them that I have, in fact, complied with all applicable transparency and openness standards and that, under long-standing Foundation policy, the computer codes referred to by The Wall Street Journal are considered the intellectual property of researchers and are not subject to disclosure.4
(page 5)
With this background in mind, let me now respond to your specific inquiries:
A (Q5A): I have made available all of the research data that I am required to under United States policy as set by the National Science Foundation. In accordance with the rules promulgated by the Foundation and supported by the Foundation’s General Counsel, I maintain the right to decline to release any computer codes, which are my intellectual property.
A (Q5B): The policy regarding sharing research and methods I and my colleagues follow is to disclose any information that might be useful to other researchers, including the data, description of methodology, and so forth, that would enable a competent scientist to replicate our work. The proof here, of course, is that other scientists have in fact succeeded in replicating our work. And, as noted above, our policies are fully in keeping with those established by the National Science Foundation.
A(Q5C): The source of these policies is the National Science Foundation.
A(Q5D): My computer program is a piece of private, intellectual property, as the National Science Foundation and its lawyers recognize. It is a bedrock principle of American law that the government may not take private property “without [a] public use,” and “without just compensation.”
That notwithstanding, the program used to generate the original Mann et al. 1998 temperature reconstructions is posted at this website: ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/
(see “METHODS” subdirectory)
Q6: The Committee next asks that, “[r]egarding study data and related information that is not publicly archived, what requests have you and your co-authors received for data relating to climate change studies, what was your response, and why?”
A: I can of course only speak for myself, but I do not believe that there is any “study data” used in my published work that is not publicly archived. Having said that, I do respond diligently to any requests from scientific colleagues for data or methodological details relating to our research.
A (Q5A): I have made available all of the research data that I am required to under United States policy as set by the National Science Foundation. In accordance with the rules promulgated by the Foundation and supported by the Foundation’s General Counsel, I maintain the right to decline to release any computer codes, which are my intellectual property.
A (Q5B): The policy regarding sharing research and methods I and my colleagues follow is to disclose any information that might be useful to other researchers, including the data, description of methodology, and so forth, that would enable a competent scientist to replicate our work. The proof here, of course, is that other scientists have in fact succeeded in replicating our work. And, as noted above, our policies are fully in keeping with those established by the National Science Foundation.
A(Q5C): The source of these policies is the National Science Foundation.
A(Q5D): My computer program is a piece of private, intellectual property, as the National Science Foundation and its lawyers recognize. It is a bedrock principle of American law that the government may not take private property “without [a] public use,” and “without just compensation.”
That notwithstanding, the program used to generate the original Mann et al. 1998 temperature reconstructions is posted at this website: ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/
(see “METHODS” subdirectory)
Q6: The Committee next asks that, “[r]egarding study data and related information that is not publicly archived, what requests have you and your co-authors received for data relating to climate change studies, what was your response, and why?”
A: I can of course only speak for myself, but I do not believe that there is any “study data” used in my published work that is not publicly archived. Having said that, I do respond diligently to any requests from scientific colleagues for data or methodological details relating to our research.
(page 6)
Q7: This question poses a number of questions based on an article published by
McIntyre and McKitrick in Energy & Environment. The question states that these
authors “report a number of errors and omissions in Mann et al. 1998 and how these may
affect the underlying conclusions of the work.” The question goes on to list a number of
topics that I should address in a “narrative explanation.”
A: I want to begin by emphasizing that nothing in McIntyre and McKitrick’s article undermines the conclusion of my research. My colleagues and I stand foursquare behind our work. So does the scientific community.
The various claims of McIntyre and McKitrick — including the ones repeated in your question — have been exhaustively examined by two different groups of climate researchers who have found their objections to be unfounded.5 See also National Center for Atmospheric Research, Media Advisory: The Hockey Stick Controversy New Analysis Reproduces Graph of Late 20th Century Temperature Rise (May 11, 2005) (available at: http://www.ucar.edu/news/releases/2005/ammann.shtml).
Moreover, it is my understanding that several other groups of climate researchers have examined McIntyre and McKitrick’s criticisms and also have found their criticisms lacking in merit. On the other hand, I know of no independent scientific group that has found any of McIntyre and McKitrick’s claims to be valid.
Nor is that surprising. Energy & Environment is not a peer reviewed scientific journal;
{also see: Adapting to the New Media Climate for Climate Change
http://www.yaleclimatemediaforum.org/2012/01/ee-covering-climate-change-in-the-age-of-digital-media/}
... it is a journal primarily devoted to policy rather than science that appears to engage in, at most, haphazard review of its articles. And neither McIntyre nor McKitrick is a trained climate scientist.
According to the biographical data on their websites, Mr. McIntyre is a mining industry executive with no formal training in any discipline related to climate research and Mr. McKitrick is an economist with no scientific training, hardly credentials that lend force to their academic arguments. See http://www.rossmckitrick.com/uploads/4/8/0/8/4808045/rm.cv.pdf and http://www.uoguelph.ca/~rmckitri/research/stevebio.doc.
Adding to the problem, the editor of Energy & Environment, Ms. Sonja Boehmer- Christiansen, has candidly acknowledged that the publication has a clear editorial bias.
In the September 5, 2003 issue of the Chronicle of Higher Education, Ms. Boehmer- Christiansen is quoted as describing the editorial policy of Energy & Environment in this way: “I’m following my political agenda – a bit, anyway. *** But isn’t that the right of (page 7) an editor?”
As to “peer review,” Ms. Boehmer-Christiansen has acknowledged in an email to Dr. Tim Osborn of the Climatic Research Unit at the University of East Anglia (U.K.), that in her rush to get the McIntyre and McKitrick piece into print for political reasons Energy & Environment dispensed with what scientists consider peer review (“I was rushing you to get this paper out for policy impact reasons, e.g. publication well before COP9”).
As Ms. Boehmer-Christiansen added, the “paper was amended until the very last moment. There was a trade off in favour of policy.” McIntyre and McKitrick’s work has been discredited by ample peer-reviewed, scientific work.
A: I want to begin by emphasizing that nothing in McIntyre and McKitrick’s article undermines the conclusion of my research. My colleagues and I stand foursquare behind our work. So does the scientific community.
The various claims of McIntyre and McKitrick — including the ones repeated in your question — have been exhaustively examined by two different groups of climate researchers who have found their objections to be unfounded.5 See also National Center for Atmospheric Research, Media Advisory: The Hockey Stick Controversy New Analysis Reproduces Graph of Late 20th Century Temperature Rise (May 11, 2005) (available at: http://www.ucar.edu/news/releases/2005/ammann.shtml).
Moreover, it is my understanding that several other groups of climate researchers have examined McIntyre and McKitrick’s criticisms and also have found their criticisms lacking in merit. On the other hand, I know of no independent scientific group that has found any of McIntyre and McKitrick’s claims to be valid.
Nor is that surprising. Energy & Environment is not a peer reviewed scientific journal;
{also see: Adapting to the New Media Climate for Climate Change
E&E: Covering Climate Change in the Age of Digital Media
January 31, 2012http://www.yaleclimatemediaforum.org/2012/01/ee-covering-climate-change-in-the-age-of-digital-media/}
... it is a journal primarily devoted to policy rather than science that appears to engage in, at most, haphazard review of its articles. And neither McIntyre nor McKitrick is a trained climate scientist.
According to the biographical data on their websites, Mr. McIntyre is a mining industry executive with no formal training in any discipline related to climate research and Mr. McKitrick is an economist with no scientific training, hardly credentials that lend force to their academic arguments. See http://www.rossmckitrick.com/uploads/4/8/0/8/4808045/rm.cv.pdf and http://www.uoguelph.ca/~rmckitri/research/stevebio.doc.
Adding to the problem, the editor of Energy & Environment, Ms. Sonja Boehmer- Christiansen, has candidly acknowledged that the publication has a clear editorial bias.
In the September 5, 2003 issue of the Chronicle of Higher Education, Ms. Boehmer- Christiansen is quoted as describing the editorial policy of Energy & Environment in this way: “I’m following my political agenda – a bit, anyway. *** But isn’t that the right of (page 7) an editor?”
As to “peer review,” Ms. Boehmer-Christiansen has acknowledged in an email to Dr. Tim Osborn of the Climatic Research Unit at the University of East Anglia (U.K.), that in her rush to get the McIntyre and McKitrick piece into print for political reasons Energy & Environment dispensed with what scientists consider peer review (“I was rushing you to get this paper out for policy impact reasons, e.g. publication well before COP9”).
As Ms. Boehmer-Christiansen added, the “paper was amended until the very last moment. There was a trade off in favour of policy.” McIntyre and McKitrick’s work has been discredited by ample peer-reviewed, scientific work.
Nonetheless, let me try to respond to the Committee’s specific questions.
A(7A,7B): The Committee inquires about the sensitivity of the results of the Mann et al. 1998 study to the inclusion or omission of certain North American tree-ring data (“the bristlecone pine series” and “archived Gaspe tree ring data” referred to in the Committee’s letter). For a complete scientific response, you should consult the article my co-authors and I published back in 1999 addressing precisely these issues: Mann, M.E., Bradley, R.S., and Hughes, M.K., Northern Hemisphere Temperatures During the Past Millennium: Inferences, Uncertainties, and Limitations, Geophysical Research Letters, 26, 759-62 (1999).
The issues raised by the Committee involve a 100 year sub-interval of our reconstruction from AD 1400-1500. As my co-authors and I explained in our 1999 article cited above, given the proxy data available at that time, certain key tree-ring data (including the series mentioned above) were essential, if the reconstructed temperature record during early centuries were to have any climatologic “skill” (that is, any validity or meaningfulness). These conclusions were of course reached through analyses in which these key datasets were excluded, and the results tested for statistical validity.
Our conclusions have been confirmed by Wahl and Ammann (see above). These researchers have demonstrated that the reconstructions produced by McIntyre and McKitrick result from ignoring these key data, and fail the accepted, basic tests for statistical validity. Moreover, Wahl and Ammann demonstrate that the climatologically improbable results obtained by McIntyre and McKitrick, which
would suggest that the Northern Hemisphere was unusually warm during the 15th century (the middle of the so-called “Little Ice Age”), are statistically meaningless, and an artifact of both their exclusion of key proxy data (as discussed above) and the use of a flawed implementation of the Mann et al. 1998 method. See http://www.cgd.ucar.edu/ccr/ammann/millennium/CODES_MBH.html) (chart at the bottom of the page).
Since 1999 new proxy data have become available and new methodologies developed for using them. Studies using these data and methodologies have confirmed the primary conclusion of our work (e.g. Mann et al. 1998 and Mann et al. 1999) that the most recent decades were likely the warmest of the past 1,000 years for the Northern Hemisphere on the average. The most recent such study (published in Nature) in fact extends this conclusion to at least the past 2,000 years. Moberg, A., D.M. Sonechkin, K. Holmgren, N.M. Datsenko, and W. Karlen, Highly Variable Northern Hemisphere (page 8)
A(7A,7B): The Committee inquires about the sensitivity of the results of the Mann et al. 1998 study to the inclusion or omission of certain North American tree-ring data (“the bristlecone pine series” and “archived Gaspe tree ring data” referred to in the Committee’s letter). For a complete scientific response, you should consult the article my co-authors and I published back in 1999 addressing precisely these issues: Mann, M.E., Bradley, R.S., and Hughes, M.K., Northern Hemisphere Temperatures During the Past Millennium: Inferences, Uncertainties, and Limitations, Geophysical Research Letters, 26, 759-62 (1999).
The issues raised by the Committee involve a 100 year sub-interval of our reconstruction from AD 1400-1500. As my co-authors and I explained in our 1999 article cited above, given the proxy data available at that time, certain key tree-ring data (including the series mentioned above) were essential, if the reconstructed temperature record during early centuries were to have any climatologic “skill” (that is, any validity or meaningfulness). These conclusions were of course reached through analyses in which these key datasets were excluded, and the results tested for statistical validity.
Our conclusions have been confirmed by Wahl and Ammann (see above). These researchers have demonstrated that the reconstructions produced by McIntyre and McKitrick result from ignoring these key data, and fail the accepted, basic tests for statistical validity. Moreover, Wahl and Ammann demonstrate that the climatologically improbable results obtained by McIntyre and McKitrick, which
would suggest that the Northern Hemisphere was unusually warm during the 15th century (the middle of the so-called “Little Ice Age”), are statistically meaningless, and an artifact of both their exclusion of key proxy data (as discussed above) and the use of a flawed implementation of the Mann et al. 1998 method. See http://www.cgd.ucar.edu/ccr/ammann/millennium/CODES_MBH.html) (chart at the bottom of the page).
Since 1999 new proxy data have become available and new methodologies developed for using them. Studies using these data and methodologies have confirmed the primary conclusion of our work (e.g. Mann et al. 1998 and Mann et al. 1999) that the most recent decades were likely the warmest of the past 1,000 years for the Northern Hemisphere on the average. The most recent such study (published in Nature) in fact extends this conclusion to at least the past 2,000 years. Moberg, A., D.M. Sonechkin, K. Holmgren, N.M. Datsenko, and W. Karlen, Highly Variable Northern Hemisphere (page 8)
Temperatures Reconstructed from Low- and High-resolution Proxy Data, Nature, 433,
613-617 (2005).
A(7C): The Committee inquires about the calculation of the R2 statistic for temperature reconstruction, especially for the 15th Century proxy calculations. In order to answer this question it is important to clarify that I assume that what is meant by the “R2” statistic is the squared Pearson dot-moment correlation, or r2 (i.e., the square of the simple linear correlation coefficient between two time series) over the 1856-1901 “verification” interval for our reconstruction. My colleagues and I did not rely on this statistic in our assessments of “skill” (i.e., the reliability of a statistical model, based on the ability of a statistical model to match data not used in constructing the model) because, in our view, and in the view of other reputable scientists in the field, it is not an adequate measure of “skill.” The statistic used by Mann et al. 1998, the reduction of error, or “RE” statistic, is generally favored by scientists in the field. See, e.g., Luterbacher, J.D., et al., European Seasonal and Annual Temperature Variability, Trends and Extremes Since 1500, Science 303, 1499-1503 (2004).
RE is the preferred measure of statistical skill because it takes into account not only whether a reconstruction is “correlated” with the actual test data, but also whether it can closely reproduce the mean and standard deviation of the test data. If a reconstruction cannot do that, it cannot be considered statistically valid (i.e., useful or meaningful). The linear correlation coefficient (r) is not a sufficient diagnostic of skill, precisely because it cannot measure the ability of a reconstruction to capture changes that occur in either the standard deviation or mean of the series outside the calibration interval. This is well known. See Wilks, D.S., STATISTICAL METHODS IN ATMOSPHERIC SCIENCE, chap. 7 (Academic Press 1995); Cook, et al., Spatial Regression Methods in Dendroclimatology: A Review and Comparison of Two Techniques, International Journal of Climatology, 14, 379-402 (1994).
The highest possible attainable value of r2 (i.e., r2 = 1) may result even from a reconstruction that has no statistical skill at all. See, e.g., Rutherford, et al., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate (2005) (in press, to appear in July issue)(available at: http://www.geo.umass.edu/faculty/bradley/rutherford2005.pdf).
For all of these reasons, we, and other researchers in our field, employ RE and not r2 as the primary measure of reconstructive skill.
As noted above, in contrast to the work of Mann et al. 1998, the results of the McIntyre and McKitrick analyses fail verification tests using the accepted metric RE. This is a key finding of the Wahl and Ammann study cited above. This means that the reconstructions McIntyre and McKitrick produced are statistically inferior to the simplest possible statistical reconstruction: one that simply assigns the mean over the calibration period to all previous reconstructed values. It is for these reasons that Wahl and Ammann have concluded that McIntyre and McKitrick’s results are “without statistical and climatological merit.”
A(7C): The Committee inquires about the calculation of the R2 statistic for temperature reconstruction, especially for the 15th Century proxy calculations. In order to answer this question it is important to clarify that I assume that what is meant by the “R2” statistic is the squared Pearson dot-moment correlation, or r2 (i.e., the square of the simple linear correlation coefficient between two time series) over the 1856-1901 “verification” interval for our reconstruction. My colleagues and I did not rely on this statistic in our assessments of “skill” (i.e., the reliability of a statistical model, based on the ability of a statistical model to match data not used in constructing the model) because, in our view, and in the view of other reputable scientists in the field, it is not an adequate measure of “skill.” The statistic used by Mann et al. 1998, the reduction of error, or “RE” statistic, is generally favored by scientists in the field. See, e.g., Luterbacher, J.D., et al., European Seasonal and Annual Temperature Variability, Trends and Extremes Since 1500, Science 303, 1499-1503 (2004).
RE is the preferred measure of statistical skill because it takes into account not only whether a reconstruction is “correlated” with the actual test data, but also whether it can closely reproduce the mean and standard deviation of the test data. If a reconstruction cannot do that, it cannot be considered statistically valid (i.e., useful or meaningful). The linear correlation coefficient (r) is not a sufficient diagnostic of skill, precisely because it cannot measure the ability of a reconstruction to capture changes that occur in either the standard deviation or mean of the series outside the calibration interval. This is well known. See Wilks, D.S., STATISTICAL METHODS IN ATMOSPHERIC SCIENCE, chap. 7 (Academic Press 1995); Cook, et al., Spatial Regression Methods in Dendroclimatology: A Review and Comparison of Two Techniques, International Journal of Climatology, 14, 379-402 (1994).
The highest possible attainable value of r2 (i.e., r2 = 1) may result even from a reconstruction that has no statistical skill at all. See, e.g., Rutherford, et al., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate (2005) (in press, to appear in July issue)(available at: http://www.geo.umass.edu/faculty/bradley/rutherford2005.pdf).
For all of these reasons, we, and other researchers in our field, employ RE and not r2 as the primary measure of reconstructive skill.
As noted above, in contrast to the work of Mann et al. 1998, the results of the McIntyre and McKitrick analyses fail verification tests using the accepted metric RE. This is a key finding of the Wahl and Ammann study cited above. This means that the reconstructions McIntyre and McKitrick produced are statistically inferior to the simplest possible statistical reconstruction: one that simply assigns the mean over the calibration period to all previous reconstructed values. It is for these reasons that Wahl and Ammann have concluded that McIntyre and McKitrick’s results are “without statistical and climatological merit.”
(page 9)
A(7D): The Committee asks “[w]hat validation statistics did you calculate for the
reconstruction prior to 1820, and what were the results?” Our validation statistics were
described in detail in a table provided in the supplementary information on Nature’s
website accompanying our original nature article, Mann, M.E., Bradley, R.S., Hughes,
M.K., Global-Scale Temperature Patterns and Climate Forcing Over the Past Six
Centuries, Nature, 392, 779-787 (1998). These statistics remain on Nature’s website (see
http://www.nature.com/nature/journal/v392/n6678/suppinfo/392779a0.html) and on our
own website. See ftp:holocene.evsc.virginia.edu/pub/Mannetal98.
A(7E): The Committee asks how I “choose particular proxies and proxy series.” Again, this information is furnished in detail in both our original 1998 article in Nature, and expanded upon in a follow-up article published in 2000. See Mann et al., Global Temperature Patterns in Past Centuries: An Interactive Presentation, Earth Interactions 4-4, 1-29 (2000), specifically this link therein: http://www.ncdc.noaa.gov/paleo/ei/ei_cover.html.
As our 1998 study and the additional information mentioned above make clear, we made use of all long-term, annually-resolved proxy indicators available to us in the public domain or through colleagues at the time the research was initiated (1996-1997) that met requirements for suitable length, age model reliability, and in the case of tree ring series, replication, inter-correlation and metadata as described above.
Q8: This question asks me to “[e]xplain in detail” my work “for and on behalf of the Intergovernmental Panel on Climate Change,” including my “role in the Third Assessment Report” (referred to as “TAR”), and a host of information as to how TAR was prepared and how the authors of TAR verified the soundness of the data that formed the basis for the conclusions set forth in TAR.
A: As is set forth on my curriculum vitae, I was one of ten lead authors of chapter 2 of TAR, and I served as a contributing author for chapters 7, 8, and 12 of the report. Given the breadth of the project, there were two layers of editorial review that oversaw the work of the lead authors for each chapter, so the chapter reflected a consensus scientific view, not merely the views of any single author. The TAR had 672 scientist reviewers. In the United States, anyone who wanted to review the drafts was allowed access to them to provide a review. I am not myself familiar with any scientific document that has been more comprehensively reviewed than the TAR.
Information concerning the “dates of key meetings,” the steps taken by “reviewers, and lead authors to ensure the data . . . were sound and accurate,” and the “identity of people who wrote and reviewed” portions of TAR should be obtained directly from the Intergovernmental Panel on Climate Change (“IPCC”). As I am sure you can appreciate, I am not an agent of the IPCC and I am not empowered to speak on IPCC’s behalf on these matters. Nor have I been authorized by the IPCC to make public information that the IPCC itself has not chosen to make publicly available. If the Committee is interested in pursuing these matters, I would urge that the Committee (page 10) contact Sir John Houghton, the head of the Working Group, at the Hadley Centre in England.
A(7E): The Committee asks how I “choose particular proxies and proxy series.” Again, this information is furnished in detail in both our original 1998 article in Nature, and expanded upon in a follow-up article published in 2000. See Mann et al., Global Temperature Patterns in Past Centuries: An Interactive Presentation, Earth Interactions 4-4, 1-29 (2000), specifically this link therein: http://www.ncdc.noaa.gov/paleo/ei/ei_cover.html.
As our 1998 study and the additional information mentioned above make clear, we made use of all long-term, annually-resolved proxy indicators available to us in the public domain or through colleagues at the time the research was initiated (1996-1997) that met requirements for suitable length, age model reliability, and in the case of tree ring series, replication, inter-correlation and metadata as described above.
Q8: This question asks me to “[e]xplain in detail” my work “for and on behalf of the Intergovernmental Panel on Climate Change,” including my “role in the Third Assessment Report” (referred to as “TAR”), and a host of information as to how TAR was prepared and how the authors of TAR verified the soundness of the data that formed the basis for the conclusions set forth in TAR.
A: As is set forth on my curriculum vitae, I was one of ten lead authors of chapter 2 of TAR, and I served as a contributing author for chapters 7, 8, and 12 of the report. Given the breadth of the project, there were two layers of editorial review that oversaw the work of the lead authors for each chapter, so the chapter reflected a consensus scientific view, not merely the views of any single author. The TAR had 672 scientist reviewers. In the United States, anyone who wanted to review the drafts was allowed access to them to provide a review. I am not myself familiar with any scientific document that has been more comprehensively reviewed than the TAR.
Information concerning the “dates of key meetings,” the steps taken by “reviewers, and lead authors to ensure the data . . . were sound and accurate,” and the “identity of people who wrote and reviewed” portions of TAR should be obtained directly from the Intergovernmental Panel on Climate Change (“IPCC”). As I am sure you can appreciate, I am not an agent of the IPCC and I am not empowered to speak on IPCC’s behalf on these matters. Nor have I been authorized by the IPCC to make public information that the IPCC itself has not chosen to make publicly available. If the Committee is interested in pursuing these matters, I would urge that the Committee (page 10) contact Sir John Houghton, the head of the Working Group, at the Hadley Centre in England.
For the Committee’s convenience, I have sent along with this letter copies of key
scientific articles referred to in this letter. Please let me know if you have questions.
Respectfully submitted,
Michael E. Mann, Ph.D.
Associate Professor
Respectfully submitted,
Michael E. Mann, Ph.D.
Associate Professor
~ ~ ~ ~ ~ ~ ~
Footnotes
_________________________________
1) This response is submitted without waiving any objection I might have to the Committee’s jurisdiction over the subject matter of this inquiry.
2) http://fox.rwu.edu/~rutherfo/supplements/jclim2003a/ http://www.ngdc.noaa.gov/paleo/pubs/jones2004/jones2004.html http://www.ngdc.noaa.gov/paleo/pubs/mann2003b/mann2003b.html http://www.ngdc.noaa.gov/paleo/pubs/mann2003/mann2003.html http://www.ngdc.noaa.gov/paleo/ei/ei_cover.html http://www.ncdc.noaa.gov/paleo/ei/ei_datarev.html http://www.ncdc.noaa.gov/paleo/ei/ei_reconsa.html http://www.ncdc.noaa.gov/paleo/ei/ei_reconsb.html http://www.ncdc.noaa.gov/paleo/ei/ei_reconsc.html http://www.ncdc.noaa.gov/cgi-bin/paleo/mannplot2.pl http://www.ncdc.noaa.gov/paleo/ei/data_supp.html http://www.ncdc.noaa.gov/paleo/ei/stats-supp-annual.html http://www.ncdc.noaa.gov/paleo/ei/stats-supp-cold.html http://www.ncdc.noaa.gov/paleo/ei/stats-supp-warm.html http://www.ncdc.noaa.gov/paleo/ei/ei_nodendro.html ftp://ftp.ngdc.noaa.gov/paleo/paleolimnology/newengland/glacial_lake_hitchcoc k/
ftp://eclogite.geo.umass.edu/pub/mann/MANNETAL98/nino3.dat ftp://eclogite.geo.umass.edu/pub/mann/ONLINE- PREPRINTS/Millennium/DATA/RECONS/ ftp://holocene.evsc.virginia.edu/pub/sdr/temp/nature/MANNETAL98/ http://www.ngdc.noaa.gov/paleo/pubs/mann1998/frames.htm ftp://eclogite.geo.umass.edu/pub/mann/MANNETAL98/ ftp://holocene.evsc.virginia.edu/pub/MBH98/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/FIGUREDATA/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/INSTRUMENTAL/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/METHODS/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/PROXY/ ftp://holocene.evsc.virginia.edu/pub/mann/Filter/lowpass.m ftp://holocene.evsc.virginia.edu/pub/mann/Filter/lowpassmin.m http://www.atmos.ucla.edu/tcd/ssa/ http://holocene.evsc.virginia.edu/Mann/tools/MTM-RED http://holocene.evsc.virginia.edu/Mann/tools/MTM-COHERE http://holocene.evsc.virginia.edu/Mann/tools/CMPLXDEMOD http://holocene.evsc.virginia.edu/Mann/tools/MTM-SVD
3) All of the proxy data (tree-rings, coral, ice cores, and historical documents) used in Mann et al. (1998) has been available since May 2000 on this public website: ftp://holocene.evsc.virginia.edu/pub/MBH98.
{Incidentally see: http://www.ncdc.noaa.gov/paleo/recons.html
http://www.ipcc.ch/publications_and_data/ar4/wg1/en/ch6s6-6.html}
The methodology used by my colleagues and me is described in detail in the initial publication, and further expanded upon in July 2004 on Nature’s supplementary website (http://www.nature.com/nature/journal/v430/n6995/suppinfo/nature02478.html) and on our own website, ftp://holocene.evsc.virginia.edu/pub/MANNETAL98. Moreover, independently-derived source codes for implementing our algorithm, and all required input data, have been posted on the website of the National Center for Atmospheric Research. See http://www.ucar.edu/ccr/ammann/millennium/CODES_MBH.html.
For these reasons, charges that our work is not subject to replication are unfounded. The initial description of the work was sufficient to permit researchers to independently produce the key algorithms. See, e.g., Zorita, E., F. Gonzalez- Rouco, and S. Legutke, Testing the Mann et al.(1998) approach to paleoclimate reconstructions in the context of a 1000-yr control simulation with the ECHO-G Coupled Climate Model, J. Climate, 16, 1378-1390 (2003); Von Storch, H., E. Zorita, J.M. Jones, Y. Dimitriev, F. Gonzalez-Rouco, F., and S.F.B. Tett, Reconstructing Past Climate from Noisy Data, Science, 306, 679-682 (2004).
Not only have we replicated our results with a different methodology (Rutherford, S., Mann, M.E., Osborn, T.J., Bradley, R.S., Briffa, K.R., Hughes, M.K., Jones, P.D., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate (2005) (to appear in July issue), but an independent group has replicated our original methods and results (See Wahl, E.R. and Ammann, C.M., Robustness of the Mann, Bradley, Hughes Reconstruction of Surface Temperatures: Examination of Criticisms Based on the Nature and Processing of Proxy Climate Evidence, Climatic Change (2005) (forthcoming)).
4) For the sake of completeness, let me quote in its entirety the email message sent by Dr. David J. Verardo, Director, Paleoclimate Program, Division of Atmospheric Sciences, National Science Foundation to Mr. Steve McIntyre (copied to me), on December 17, 2003, in response to a previous email that McIntyre had sent to Dr. Verardo (copied to me):
Even more recently, the National Science Foundation confirmed its view that my computer codes are my intellectual property. A recent issue of the Chronicle of Higher Education states: “According to David Stonner, of the Congressional-affairs office at the National Science Foundation, Mr. McIntyre contacted the foundation last year to ask for Mr. Mann's computer code. Mr. Stonner said the agency had told Mr. McIntyre that the code was the intellectual property of Mr. Mann . . ..” Richard Monastersky, Congressman Demands Complete Records on Climate Research by 3 Scientists Who Support of Global Warming, Chronicle of Higher Education (July 1, 2005), available at: http://chronicle.com/article/Congressman-Demands-Complete/121447/.
5) See, e.g., Rutherford, S., Mann, M.E., Osborn, T.J., Bradley, R.S., Briffa, K.R., Hughes, M.K., Jones, P.D., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate (2005) (in press, to appear in July issue); Wahl, E.R. and Ammann, C.M., Robustness of the Mann, Bradley, Hughes Reconstruction of Surface Temperatures: Examination of Criticisms Based on the Nature and Processing of Proxy Climate Evidence, Climatic Change (2005) (forthcoming).
Footnotes
_________________________________
1) This response is submitted without waiving any objection I might have to the Committee’s jurisdiction over the subject matter of this inquiry.
2) http://fox.rwu.edu/~rutherfo/supplements/jclim2003a/ http://www.ngdc.noaa.gov/paleo/pubs/jones2004/jones2004.html http://www.ngdc.noaa.gov/paleo/pubs/mann2003b/mann2003b.html http://www.ngdc.noaa.gov/paleo/pubs/mann2003/mann2003.html http://www.ngdc.noaa.gov/paleo/ei/ei_cover.html http://www.ncdc.noaa.gov/paleo/ei/ei_datarev.html http://www.ncdc.noaa.gov/paleo/ei/ei_reconsa.html http://www.ncdc.noaa.gov/paleo/ei/ei_reconsb.html http://www.ncdc.noaa.gov/paleo/ei/ei_reconsc.html http://www.ncdc.noaa.gov/cgi-bin/paleo/mannplot2.pl http://www.ncdc.noaa.gov/paleo/ei/data_supp.html http://www.ncdc.noaa.gov/paleo/ei/stats-supp-annual.html http://www.ncdc.noaa.gov/paleo/ei/stats-supp-cold.html http://www.ncdc.noaa.gov/paleo/ei/stats-supp-warm.html http://www.ncdc.noaa.gov/paleo/ei/ei_nodendro.html ftp://ftp.ngdc.noaa.gov/paleo/paleolimnology/newengland/glacial_lake_hitchcoc k/
ftp://eclogite.geo.umass.edu/pub/mann/MANNETAL98/nino3.dat ftp://eclogite.geo.umass.edu/pub/mann/ONLINE- PREPRINTS/Millennium/DATA/RECONS/ ftp://holocene.evsc.virginia.edu/pub/sdr/temp/nature/MANNETAL98/ http://www.ngdc.noaa.gov/paleo/pubs/mann1998/frames.htm ftp://eclogite.geo.umass.edu/pub/mann/MANNETAL98/ ftp://holocene.evsc.virginia.edu/pub/MBH98/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/FIGUREDATA/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/INSTRUMENTAL/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/METHODS/ ftp://holocene.evsc.virginia.edu/pub/MANNETAL98/PROXY/ ftp://holocene.evsc.virginia.edu/pub/mann/Filter/lowpass.m ftp://holocene.evsc.virginia.edu/pub/mann/Filter/lowpassmin.m http://www.atmos.ucla.edu/tcd/ssa/ http://holocene.evsc.virginia.edu/Mann/tools/MTM-RED http://holocene.evsc.virginia.edu/Mann/tools/MTM-COHERE http://holocene.evsc.virginia.edu/Mann/tools/CMPLXDEMOD http://holocene.evsc.virginia.edu/Mann/tools/MTM-SVD
3) All of the proxy data (tree-rings, coral, ice cores, and historical documents) used in Mann et al. (1998) has been available since May 2000 on this public website: ftp://holocene.evsc.virginia.edu/pub/MBH98.
{Incidentally see: http://www.ncdc.noaa.gov/paleo/recons.html
http://www.ipcc.ch/publications_and_data/ar4/wg1/en/ch6s6-6.html}
The methodology used by my colleagues and me is described in detail in the initial publication, and further expanded upon in July 2004 on Nature’s supplementary website (http://www.nature.com/nature/journal/v430/n6995/suppinfo/nature02478.html) and on our own website, ftp://holocene.evsc.virginia.edu/pub/MANNETAL98. Moreover, independently-derived source codes for implementing our algorithm, and all required input data, have been posted on the website of the National Center for Atmospheric Research. See http://www.ucar.edu/ccr/ammann/millennium/CODES_MBH.html.
For these reasons, charges that our work is not subject to replication are unfounded. The initial description of the work was sufficient to permit researchers to independently produce the key algorithms. See, e.g., Zorita, E., F. Gonzalez- Rouco, and S. Legutke, Testing the Mann et al.(1998) approach to paleoclimate reconstructions in the context of a 1000-yr control simulation with the ECHO-G Coupled Climate Model, J. Climate, 16, 1378-1390 (2003); Von Storch, H., E. Zorita, J.M. Jones, Y. Dimitriev, F. Gonzalez-Rouco, F., and S.F.B. Tett, Reconstructing Past Climate from Noisy Data, Science, 306, 679-682 (2004).
Not only have we replicated our results with a different methodology (Rutherford, S., Mann, M.E., Osborn, T.J., Bradley, R.S., Briffa, K.R., Hughes, M.K., Jones, P.D., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate (2005) (to appear in July issue), but an independent group has replicated our original methods and results (See Wahl, E.R. and Ammann, C.M., Robustness of the Mann, Bradley, Hughes Reconstruction of Surface Temperatures: Examination of Criticisms Based on the Nature and Processing of Proxy Climate Evidence, Climatic Change (2005) (forthcoming)).
4) For the sake of completeness, let me quote in its entirety the email message sent by Dr. David J. Verardo, Director, Paleoclimate Program, Division of Atmospheric Sciences, National Science Foundation to Mr. Steve McIntyre (copied to me), on December 17, 2003, in response to a previous email that McIntyre had sent to Dr. Verardo (copied to me):
Dear Mr. McIntyre,
I apologize if my last electronic message was not clear but let me clarify the US NSF's view in this current message. Dr. Mann and his other US colleagues are under no obligation to provide you with any additional data beyond the extensive data sets they have already made available. He is not required to provide you with computer programs, codes, etc. His research is published in the peer-reviewed literature which has passed muster with the editors of those journals and other scientists who have reviewed his manuscripts.
You are free to your analysis of climate data and he is free to his. The passing of time and evolving new knowledge about Earth's climate will eventually tell the full story of changing climate. I would expect that you would respect the views of the US NSF on the issue of data access and intellectual property for US investigators as articulated by me to you in my last message under the advisement of the US NSF's Office of General Counsel.
Respectfully,
David J. Verardo Director,
Paleoclimate Program Division of Atmospheric Sciences National Science Foundation
Even more recently, the National Science Foundation confirmed its view that my computer codes are my intellectual property. A recent issue of the Chronicle of Higher Education states: “According to David Stonner, of the Congressional-affairs office at the National Science Foundation, Mr. McIntyre contacted the foundation last year to ask for Mr. Mann's computer code. Mr. Stonner said the agency had told Mr. McIntyre that the code was the intellectual property of Mr. Mann . . ..” Richard Monastersky, Congressman Demands Complete Records on Climate Research by 3 Scientists Who Support of Global Warming, Chronicle of Higher Education (July 1, 2005), available at: http://chronicle.com/article/Congressman-Demands-Complete/121447/.
5) See, e.g., Rutherford, S., Mann, M.E., Osborn, T.J., Bradley, R.S., Briffa, K.R., Hughes, M.K., Jones, P.D., Proxy-based Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, Journal of Climate (2005) (in press, to appear in July issue); Wahl, E.R. and Ammann, C.M., Robustness of the Mann, Bradley, Hughes Reconstruction of Surface Temperatures: Examination of Criticisms Based on the Nature and Processing of Proxy Climate Evidence, Climatic Change (2005) (forthcoming).
No comments:
Post a Comment