{"id":9106,"date":"2010-11-17T14:20:13","date_gmt":"2010-11-17T14:20:13","guid":{"rendered":"http:\/\/scienceblogs.com\/gregladen\/2010\/11\/17\/an-excellent-example-of-either\/"},"modified":"2010-11-17T14:20:13","modified_gmt":"2010-11-17T14:20:13","slug":"an-excellent-example-of-either","status":"publish","type":"post","link":"https:\/\/gregladen.com\/blog\/2010\/11\/17\/an-excellent-example-of-either\/","title":{"rendered":"An excellent example of either crappy science reporting or crappy science …"},"content":{"rendered":"

Let’s have a look and see if we can decide.
\n
\n\"ResearchBlogging.org\"<\/a><\/span>Sciencedaily.com has this piece<\/a> on a paper published in the Journal of Medical Ethics<\/em> in which the claim is made that “US Scientists Significantly More Likely to Publish Fake Research.” The problem is that the statistics given don’t show that.<\/p>\n

The study is said to look at papers withdrawn according to PubMed between 2000 and 2010. Which means 2001 through 2009 inclusively (though I’m guessing that is not what was meant). Here’s the data:<\/p>\n

Papers retracted:<\/em> 788<\/strong>
\n… because of error:<\/em> 545<\/strong>
\n… “attributed” to fraud <\/em>(no indication in the write up what this attribution is attributed to): the rest of the papers. (We are left to do our own math, possibly because the number is so tiny that it might fail to impress us?): 243<\/strong><\/p>\n

So that’s 243 papers over 9, 10 or 11 years.<\/p>\n

Now, here’s the tricky part:<\/p>\n

The highest number of retracted papers (260) were written by US first authors.<\/p>\n

Interesting. We were<\/em> talking about fraudulent papers, but now we are citing numbers that include papers with errors. We are also told that “one in three” of these are fraudulent, again, we need to do the math ourselves. The answer is 86.66.<\/p>\n

Now, at this point, I should mention that there are between about 3200 and 4000 days in the period “between 2000 and 2011” and there are probably more than 100 scientific papers published every day. So the rate of production of US based alleged fraudulent papers is minuscule.<\/p>\n

But that is not the real problem with this report. This is the problem: “The highest number of retracted papers were written by US first authors (260), accounting for a third of the total. One in three of these was attributed to fraud” … “The UK, India, Japan, and China each had more than 40 papers withdrawn during the decade. Asian nations, including South Korea, accounted for 30% of retractions. Of these, one in four was attributed to fraud.”<\/p>\n

Well, that’s pretty meaningless unless we want to get the calculator out again, and even then, useful only if we want to assume that the rate of “fraud” against “withdrawn” is identical for all cases, but here we have a hint that it varies by as much as about 30%. But who cares about that … you see the problem, right? We are told (in the article’s title) that the RATE<\/em> of fraud in the US is higher than anywhere else, but we are then told in the body of the piece that the NUMBER<\/em> of fraudulent cases is highest in the US.<\/p>\n

Not the same thing at all.<\/p>\n

But, wait, there’s more. Or should I say, less.<\/p>\n

One would think that the rate of fraud being alarmingly higher (or even moderately but statistically significantly higher) in the US would be important enough that it would be mentioned in the abstract<\/a> of the original published paper. Let’s see if it is:<\/p>\n

Abstract<\/strong><\/p>\n

Background<\/strong> Papers retracted for fraud (data fabrication or data falsification) may represent a deliberate effort to deceive, a motivation fundamentally different from papers retracted for error. It is hypothesised that fraudulent authors target journals with a high impact factor (IF), have other fraudulent publications, diffuse responsibility across many co-authors, delay retracting fraudulent papers and publish from countries with a weak research infrastructure.<\/p>\n

Methods<\/strong> All 788 English language1<\/sup> research papers retracted from the PubMed database between 2000 and 2010 were evaluated. Data pertinent to each retracted paper were abstracted from the paper and the reasons for retraction were derived from the retraction notice and dichotomised as fraud or error. Data for each retracted article were entered in an Excel1<\/sup> spreadsheet for analysis.<\/p>\n

Results<\/strong> Journal IF3<\/sup> was higher for fraudulent papers (p<0.001). Roughly 53% of fraudulent papers were written by a first author who had written other retracted papers ('repeat offender'), whereas only 18% of erroneous papers were written by a repeat offender (\u00cf?=88.40; p<0.0001). Fraudulent papers had more authors (p<0.001) and were retracted more slowly than erroneous papers (p<0.005). Surprisingly, there was significantly more fraud than error among retracted papers from the USA (\u00cf?2=8.71; p<0.05) compared with the rest of the world.\n Conclusions<\/strong> This study reports evidence consistent with the ‘deliberate fraud’ hypothesis. The results suggest that papers retracted because of data fabrication or falsification represent a calculated effort to deceive. It is inferred that such behaviour is neither na\u00c3\u00afve, feckless nor inadvertent.<\/p><\/blockquote>\n<\/blockquote>\n

\n

A few notes:
\n1<\/sup>Note that since only English Language papers are used, there will be a slight reduction of papers from non-US sources in the sample.<\/p>\n

2<\/sup>Funny how an article looking into fraud and retracted papers would use a non-OpenSource, and thus unverifiable, mathematics tool in their research. Is this a case of product placement or merely misplaced understanding of the concept of replicabiltiy and transparency?<\/p>\n

3<\/sup>IF = Impact Factor<\/p>\n

\n

The headline from Sciencedaily is not even mentioned.<\/p>\n

Now, please revisit this statement from the abstract: “Surprisingly, there was significantly more fraud than error among retracted papers from the USA (\u00cf?2=8.71; p<0.05) compared with the rest of the world.”<\/p>\n

Surprising? Why? Because of the original hypothesis that fraud would more likely come from places with “weak research infrastructure.” I’m not sure why they thought that but the evidence contradicted it, so they are surprised. Does the quoted sentence say that “US Scientists Significantly More Likely to Publish Fake Research, Study Find”? No. It says that fraud has a higher rate relative to error. Perhaps US scientists have the same or even lower rates of fraud, but screw up their data more often.<\/p>\n

Conclusion: Interesting paper mauled by crappy science reporting. I expect to see a retraction!<\/p>\n

Steen, R. (2010). Retractions in the scientific literature: do authors deliberately commit research fraud? Journal of Medical Ethics<\/span> DOI: 10.1136\/jme.2010.038125<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Let’s have a look and see if we can decide.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[5020],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p5fhV1-2mS","jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/posts\/9106"}],"collection":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/comments?post=9106"}],"version-history":[{"count":0,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/posts\/9106\/revisions"}],"wp:attachment":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/media?parent=9106"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/categories?post=9106"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/tags?post=9106"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}