Research Misconduct, Retraction, and Cleansing the Medical Literature: Lessons from the Poehlman Case FREE

In September 2003, the Annals editor, Dr. Sox, received a troubling letter from the provost of the University of Vermont in Burlington. The university had investigated alleged research misconduct by a former faculty member, Eric Poehlman, PhD, and determined that he had published fraudulent research in 3 journals. In its letter, the university referred only to the Annals of Internal Medicine article, a 1995 article on energy expenditure after menopause (1). After consulting the Uniform Requirements for Manuscripts of the International Committee of Medical Journal Editors, which mandates retraction in such circumstances (2), Dr. Sox published a brief notice of retraction in the 21 October 2003 issue of Annals(3). Soon after, Dr. Poehlman's lawyer asked Annals to retract the retraction until the federal Office of Research Integrity (ORI) had completed its ongoing investigation. After consulting with American College of Physicians legal counsel and top management, Dr. Sox called the ORI. The ORI properly said nothing about the case itself but did say that the letter from the provost at the University of Vermont was sufficient grounds for immediate retraction. In March 2005, the ORI announced its findings: Poehlman had published fabricated research in 10 articles, each in a different journal (4). Furthermore, he had included fraudulent findings in National Institutes of Health grant applications, which is a federal criminal offense. In the end, Poehlman agreed to acknowledge responsibility for fabricating data (5 - 6) and pleaded guilty to a single charge of reporting false data in a funded grant application. He awaits sentencing.

Scientific fraud can divert the search for truth, and it pollutes the record of that search, that is, the scientific literature. Once someone identifies an article as fraudulent, the scientific community has 2 duties: 1) to warn scientists to ignore the article and 2) to prevent further pollution by scientists who inadvertently cite the article. The Poehlman case illustrates the problems associated with carrying out these 2 duties. According to the ORI in 2003, the other 2 journals that the University of Vermont notified had not retracted or corrected Poehlman's articles. Furthermore, authors continued to cite Poehlman's Annals article after the 2003 notice of retraction.

The United States is 1 of only a few countries with a governmental system for evaluating allegations of scientific fraud. Widely publicized cases of scientific misconduct in research sponsored by the National Institutes of Health led the U.S. Congress to create the Office of Scientific Integrity in 1989 (later renamed the Office of Research Integrity). The ORI promulgated regulations governing the handling of allegations of scientific misconduct in research supported by the Department of Health and Human Services (7). The ORI has since developed a body of case law and fine-tuned its regulations (8). It receives 30 to 40 new cases per year, a sharp reminder that dishonesty in scientific research is a substantial ongoing problem.

Although the ORI's jurisdiction is scientific misconduct in research supported by the Department of Health and Human Services, its influence extends informally to other research. Federal regulations mandate that, as a condition of receiving federal funds, research institutions must investigate all allegations of misconduct in research supported by Department of Health and Human Services funds and must report their findings to the ORI. Because institutions usually use the same administrative procedures for handling research misconduct regardless of the source of support, the mandate affecting federally funded research becomes the institutional standard for defining, investigating, and adjudicating all research misconduct.

Science is international, but enforcement of ethical conduct of research is local. Few countries have anything like the ORI. What does an editor do when someone alleges misconduct, the responsible author's institution refuses to investigate, and no national standard exists (9 - 12)? Editors do not have the legal standing, expertise, time, or money to go into foreign institutions, secure evidence, and spend months or years uncovering misconduct, adjudicating, hearing appeals, and sanctioning offenders. Yet, if editors refuse to publish work from institutions in countries lacking standard investigative procedures, which include, for example, the United Kingdom (9) and India, they would limit the flow of good science, reduce the value of their journals, and be grossly unfair to honest authors. International organizations of editors are trying to correct the situation. The Committee on Publication Ethics, originated by British medical editors, has an important advisory function, but because it lacks legal standing and the power to punish or exonerate, it will ultimately be ineffective against recalcitrant authors. Fear of legal action is impeding an investigation in Japan, another country with no established system for handling allegations of research misconduct (13). Clearly, the solutions must be local, and editors are working through the Committee on Publication Ethics to encourage their governments to set up systems like the ORI, something we believe that a country simply must do if it expects the world to take its science seriously. This task will seem endless. However, in the meantime, if local systems for investigating allegations of scientific misconduct fail, an editor can withdraw a journal's support for a suspected fraudulent article (14 - 15) or publish an expression of concern (2,9,12,16 - 19).

The ORI has neither the mandate nor the resources to lead the task of correcting a scientific literature polluted by fraudulent research. This responsibility lies with the community of scientists. When an ORI investigation ends with a finding of misconduct, the work is just beginning. Following the investigation, the community must identify all of a fraudulent author's articles, publish retractions, and rid the literature of references to the fraudulent articles. Our purpose here is to define the responsibilities of various parties in carrying out these 3 tasks.

The first step is to identify every tainted article, which is very difficult with an uncooperative author like Poehlman. The University of Vermont report cites many instances that “display Dr. Poehlman's contempt not just for the truth, but for [the investigating] Panel, the University, and his profession” (20). However, to settle the criminal charges from the ORI investigation, Poehlman agreed to identify the fabrications in his publications and grant applications. In the end, he agreed “to retract or correct ten scientific articles due to research misconduct” (21 - 22).

Identifying every fraudulent article starts with a negotiated agreement, as in the Poehlman case, but seldom ends there. Experience strongly suggests that authors who have committed scientific fraud hardly ever acknowledge every fraudulent article. This observation leads to the following principle: Treat every article as suspect until proven otherwise. In April 2005, Poehlman had 204 publications listed in PubMed. If only 10 have undergone careful scrutiny by the ORI, how sure are we about the other 194? We aren't sure, which is why someone must check each article.

Conceptually, the scientific community bears responsibility for cleaning its own house, but administratively, someone must take charge when the occasion arises. The aftermath of several scientific fraud cases has established a compelling precedent for the accused author's institution to lead the investigation to identify every fraudulent article (23). The case of Robert Slutsky at the University of California, San Diego, is particularly salient. Slutsky had published 137 papers with 93 different coauthors when someone noticed anomalies in a few of his publications. The university's response was exemplary. It formed a committee to check Slutsky's work. The committee contacted Slutsky's coauthors and held them responsible for defending the integrity of every published paper. The committee classified 77 articles as “valid,” 48 as “questionable,” and 12 as “fraudulent.” Slutsky's lawyer asked journals to retract 15 articles. The committee asked for retraction of 2 additional articles (23).

We think that the University of California, San Diego, has set the bar at the proper level. Peer institutions should judge each other by this necessarily very high standard. Although meeting this standard will involve thousands of hours of investigation by faculty working with those most immediately responsible and affected, the coauthors, we think that institutions will rise to the occasion because their reputations and the integrity of the scientific record are at stake (24).

The University of Vermont is also meeting this standard. On 16 June 2005, Dr. Rennie spoke with Dr. Russell Tracy, who heads their investigative effort. The usual approach to investigating alleged scientific fraud is to check for inconsistencies between the published data and the raw data. Poehlman covered his tracks by presenting his co-investigators with already analyzed “results” rather than raw data. When the University of Vermont investigators looked for the raw data, they often couldn't find any. They sorted Poehlman's publications into those done at the University of Vermont and those done elsewhere. They assumed that Poehlman's coauthors had done no wrong and sought their help to divide the publications according to whether Poehlman had access to the raw data from the study. This task produced 3 groups of publications: those certainly tainted; those for which the coauthors could accept full responsibility because they were confident that Poehlman could not have altered the data; and those about which they were uncertain. The investigators' work continues, now with the help of coauthors currently at other research institutions, such as the University of Maryland in College Park, where Poehlman used to work and where several of his then co-investigators still work. In a telephone conversation with Dr. Rennie on 6 July 2005, Dr. Joseph Giffels, assistant vice-president for academic affairs at the University of Maryland, said that the university was adopting the University of Vermont investigators' approach. Eventually, the 2 institutions expect to publish a list of tainted articles and to provide it to the journals involved.

The investigators at the University of Vermont and the University of Maryland invented their own approach to checking the literature; they were not aware of the University of California, San Diego's, procedures for the Slutsky case. We hope that our article will alert university officials to the powerful precedent set by these 3 institutions and encourage them to check every article published by an author who has published fake evidence. We suggest a fail-safe bureaucratic approach to transmitting the lessons learned so far: When the ORI officially notifies an institution of the outcome of its investigation, its letter should tell the institution what is expected and describe the processes used at the University of California, San Diego; the University of Maryland; and the University of Vermont.

After identifying all tainted articles, the next step is to retract each one. Publicity reduces citation of fraudulent work (25 - 27). If an authoritative source informs a journal that an official investigation has found that the journal has published an article containing fraudulent evidence, the journal must retract the article. If offered legal advice that an author will sue the journal if it publishes a retraction, the editor must hold a steady course. We know of no applicable case.

Editors should consider several points as they prepare to retract. First, they should reserve “retraction” for scientific misconduct. When authors withdraw an article because of an error, the word “correction” is appropriate (2) (see Glossary). To underscore the crucial distinction between retraction and correction, the notice should briefly describe the rationale for taking action (28 - 29). Second, editors should begin by asking the authors to retract. However, a recalcitrant author is not an excuse for inaction. After receiving notice from an institution or the ORI, the editor must retract (2). Third, the editor should ask each author to sign the retraction. If some will not sign, the editor should so note. If all refuse, the editor must write a statement to accompany the retraction or publish one from an official from the guilty scientist's institution (30 - 31). Fourth, the editor must insist that authors of a retraction notice accept full responsibility. Authors may balk at using words like “falsification,” “fabrication,” and “research misconduct” and may propose such wording as “the data were not reproducible” or “there was a systematic error.” Given a finding of misconduct, this sophistry is clearly deceptive. The notice must give the real reasons and name the guilty parties. Ideally, it should quote the actual language used by the investigating institution. Fifth, the journal should display the notice prominently, label it as a retraction, and identify it in the table of contents. The notice should contain the full reference to the retracted article (2).

Journal editors play a pivotal role in repairing the damage from a tainted article. Sadly, some fail to retract articles that they know are fraudulent (23,32). Evidence on this point is limited, but we know that only Annals retracted Poehlman's tainted articles shortly after receiving a notice from the University of Vermont. Furthermore, only 5 of the 10 articles identified in March 2005 as being definitely fraudulent (4) had been retracted as of November 2005. Editors who fail to retract undo the hard work of investigating panels and negate the courage of the whistleblower (29).

Some journal editors may not know what to do. The ORI has produced a helpful booklet to guide journal editors (33). The booklet describes the responsibility of editors to retract articles containing fraudulent data, referring to the Uniform Requirements for Manuscripts (2). The Council of Science Editors and the World Association of Medical Editors are also resources for editors seeking advice.

The National Library of Medicine could encourage reluctant editors. It currently publishes a list of retracted articles (go to pubmed.gov, find “Special Queries” in the left-hand toolbar, and scroll down to “Retracted Publication”). It could make this list even more useful by adding to it any other article that an official investigation has found to be tainted, and it could attach a cautionary note to the tainted article's PubMed citation. When the journal publishes a retraction, the Library could replace the cautionary note with a notice of retraction.

The system for informing readers about a fraudulent article is faulty. The National Library of Medicine publicizes a retraction by placing its PubMed citation within the PubMed citation of the retracted article. This necessary action is not sufficient, as illustrated by Poehlman's Annals article, which has received 16 citations since the 2003 Annals retraction notice. Presumably, none of the citing authors knew about the retraction.

What can journals do to avoid publishing a citation of a retracted article? In principle, the solution is straightforward: Journals should require authors to attest that they have checked their manuscript's reference list against the National Library of Medicine master list of retracted articles. The Library would make the task much easier if it created a Web-based program that would check a manuscript's reference list against the master list of retracted articles and send a report to the manuscript author. Journals could require corresponding authors to attest that they had checked their submitted manuscripts' references against the master list. At the 2006 meeting of the International Committee of Medical Journal Editors, the member journals changed the Uniform Requirements for Manuscripts to specify authors' responsibility to check their manuscripts for references to retracted articles, citing PubMed as the authoritative source of information for retractions (2).

Journal editors have other opportunities to prevent citations of retracted articles. They could link retraction notices to retracted articles on their journal Web sites, and they could facilitate access to the full text of retracted articles and notices of retraction. Currently, an interested reader can access the full text of a retracted article and its corresponding notice of retraction only through the journal's Web site (or by reading the print journal), and many journals charge a fee for Web access, even 1 year after publication. Readers should not have to pay to see the self-correction mechanisms of science at work. We recommend that journals provide free Web access to retracted fraudulent articles and the accompanying notices of retraction, starting the day the journals publish the retractions.

Authors who cite an article that proves to be fraudulent should correct the record, lest readers unwittingly propagate its falsehoods. One hundred eighty-six of Poehlman's articles are in the Science Citation Index database (Thomson Science, Philadelphia, Pennsylvania); collectively, they have 3007 citations. Someone could use the Science Citation Index database to identify each author who cited fraudulent articles by Poehlman. Contacting the authors of 3007 articles is a formidable undertaking. Who will take responsibility?

Since the ORI announced its findings in the Poehlman case, several other cases (13,16,18 - 19,34) have come to light. Scientific misconduct is endemic, so the scientific community must improve its response. The Poehlman case teaches us 5 lessons about the dissemination of defective research. First, the scientific community must assume that every article written by an author who has committed scientific fraud is unreliable until someone close to the work has explained the specific reasons why he or she can vouch for its integrity. Second, the guilty scientist's coauthors bear primary responsibility for publicly validating or retracting their joint publications. Third, journal readers cannot necessarily count on journal editors to retract tainted articles. Fourth, in the interests of transparency, editors should use the word “retraction” only in cases of fraud. Finally, even when a journal has published a retraction, authors continue to cite a fraudulent article, often for years. We can expect progress toward solving these problems if the parties involved recognize their responsibilities and act. The (Table) identifies the main parties (research institutions, editors, and citing authors) and delineates each one's responsibilities in investigating claims of fraudulence, correcting the scientific literature, and preventing misconduct.

This discussion has been about repairing damage. However, preventing damage would save careers from ruin and save the time spent investigating allegations and assessing the integrity of articles that become suspect by association with a fraudulent author. Everyone has a responsibility to promote a culture in which research misconduct does not happen.

Retraction: What a journal issues when an investigation has shown that an article contains faked data or has been plagiarized. It tells the reader to ignore that article.

Expression of concern: What a journal issues when the editor is concerned that an article contains faked data or has been plagiarized but an investigation has either not begun or has begun but has not reaches a conclusion about that article.

Correction: What a journal issues to correct a mistake by substituting correct information or by asking the reader to disregard specified parts of an article (for example, a reference to a retracted article).