As you know, there is much discussion about whether or not a “strategy memo” leaked from the Heartland Institute is a fake. We are told by a trustworthy source that this policy memo was leaked to him, and that he then tricked the Heartland Institute to supply him with additional documents, which he then used to verify the “strategy memo” based on cross reference of factual information. Only after the apparent veracity of the memo was determined did that individual, Peter Gleick, release all of the documents to the public.<\/p>\n
Subsequently, a number of untrustworthy sources, such as Heartland related people and the usual gaggle of Science Haters, have insisted that the original strategy memo is a fake. One set of evidence used to suggest this is that the memo was different from the other documents in several ways: It was a photocopy or a fax with different formatting, etc. This of course is evidence of nothing. There is nothing that requires that all of the documents associated with a particular institution, or even a particular event such as a board meeting at an institution, be created, formatted, and distributed with the same look, feel, and technology. It it obvious to me that if this is the case of Heartland getting caught red handed, they might then be grasping at straws.<\/p>\n
However, we can use science to address this question further, and this is exactly what Shawn Otto has done. In a piece posted moments ago (here<\/a> and soon to be at Huffington Post) Shawn carries out an analysis using a standard and widely respected software system to compare a sample of Gleick’s writing, some samples from Heartland, and the “strategy memo.” In this analysis, the memo is entered as an unknown, and the software shows the difference between that unknown document and the known document. Read Shawn’s analysis to see the details; the conclusion is that the strategy memo was more likely written in house at Heartland than by Peter Gleick. Using the strategy memo as it exists:<\/strong><\/p>\n Canonicizers: none Analyzed by Nearest Neighbor Driver with metric Canberra Distance Using the strategy memo with the sentence “Principals and teachers are heavily biased toward the alarmist Canonicizers: none Analyzed by Nearest Neighbor Driver with metric Canberra Distance The numbers represent distance. The heartland related documents are more like the memo than Gleick’s writing. This result is similar to Shawn Otto’s, but using different source documents.<\/p>\n Had the situation been reversed, there would be a reasonable argument to make that Peter had written the memo in question. As it is, we see that the memo was not very likely written by Peter. This test fails to support the idea that the document is a fake. It does not conclusively prove that it is not a fake, for various reasons, but there is no a priori reason to claim it to be a fake, as far as I can tell. I will continue to assume that the strategy memo is a Heartland document. It is factually aligned with Heartland material. It is the style of Heartland documents, textually. Denial of it’s legitimacy by Heartland is expected and not meaningful.<\/p>\n Methods and Materials<\/strong><\/p>\n See Otto 2012<\/a> for a description of methods.<\/p>\n Documents used for comparison:<\/p>\n As you know, there is much discussion about whether or not a “strategy memo” leaked from the Heartland Institute is a fake. We are told by a trustworthy source that this policy memo was leaked to him, and that he then tricked the Heartland Institute to supply him with additional documents, which he then used … Continue reading Is the Heartland "Strategy Memo" a Fake? Let's try using science!<\/span>
\n
\nIn order to verify Shawn’s results, and to take this a small step further, I decided to use the same software on a slightly different set of documents. I use three documents from Heartland by authors Pullmann, Lehmann, (these were not used by Shawn) and the Bast document used in Shawn’s analysis. For Gleick, I use a paper he published a while back, which was not used by Shawn. These are compared to the memo in question, with the following results:<\/p>\n
\nAnalyzed by Nearest Neighbor Driver with metric Canberra Distance
\nusing Word 2Grams as events
\n1. HeartlandPullman 928.793701231854
\n2. HeartlandLehmann 1037.8138003775507
\n3. Bast 1124.7548224327647
\n4. Gleick 4310.191823786998<\/p>\n
\nusing Sentence Length as events
\n1. HeartlandLehmann 25.02741702741703
\n2. HeartlandPullman 26.36184285751981
\n3. Bast 27.209481200489734
\n4. Gleick 42.91967923917308<\/p>\n
\n perspective.” removed as per the suggestion of Steven Mosher<\/a>:<\/strong><\/p>\n
\nAnalyzed by Nearest Neighbor Driver with metric Canberra Distance
\nusing Word 2Grams as events
\n1. HeartlandPullman 921.2521372517942
\n2. HeartlandLehmann 1028.7531365975046
\n3. Bast 1115.6711036869247
\n4. Gleick 4302.822829339724<\/p>\n
\nusing Sentence Length as events
\n1. HeartlandLehmann 24.956989247311828
\n2. HeartlandPullman 26.307196401799096
\n3. Bast 26.952400231347603
\n4. Gleick 43.03531757495257<\/p>\n\n