The full content of Annals is available to subscribers

Subscribe/Learn More  >
Academia and the Profession |

Performance during Internal Medicine Residency Training and Subsequent Disciplinary Action by State Licensing Boards

Maxine A. Papadakis, MD; Gerald K. Arnold, PhD; Linda L. Blank; Eric S. Holmboe, MD; and Rebecca S. Lipner, PhD
[+] Article, Author, and Disclosure Information

From the University of California at San Francisco, San Francisco, California; American Board of Internal Medicine, Philadelphia, Pennsylvania; and the Culliton Group, Washington, DC.

Note: Dr. Lipner from the ABIM had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Grant Support: From the ABIM Foundation.

Acknowledgment: The authors thank F. Daniel Duffy, MD; Michael G. Shlipak, MD; and Robert M. Wachter, MD, for their contributions to the analysis and interpretation of the findings.

Potential Financial Conflicts of Interest: Employment: G.K. Arnold, E.S. Holmboe, R.S. Lipner (American Board of Internal Medicine). Honoraria: M.A. Papadakis (American Board of Internal Medicine).

Requests for Single Reprints: Maxine A. Papadakis, MD, University of California at San Francisco, 533 Parnassus Avenue, S-245, San Francisco, CA 94143; e-mail, papadakm@medsch.ucsf.edu.

Current Author Addresses: Dr. Papadakis: University of California at San Francisco, 533 Parnassus Avenue, S-245, San Francisco, CA 94143.

Drs. Arnold, Holmboe, and Lipner: American Board of Internal Medicine, 510 Walnut Street, Suite 1700, Philadelphia, PA 19106.

Ms. Blank: c/o Maxine A. Papadakis, MD, University of California at San Francisco, 533 Parnassus Avenue, S-245, San Francisco, CA 94143.

Ann Intern Med. 2008;148(11):869-876. doi:10.7326/0003-4819-148-11-200806030-00009
Text Size: A A A

Background: Physicians who are disciplined by state licensing boards are more likely to have demonstrated unprofessional behavior in medical school. Information is limited on whether similar performance measures taken during residency can predict performance as practicing physicians.

Objective: To determine whether performance measures during residency predict the likelihood of future disciplinary actions against practicing internists.

Design: Retrospective cohort study.

Setting: State licensing board disciplinary actions against physicians from 1990 to 2006.

Participants: 66 171 physicians who entered internal medicine residency training in the United States from 1990 to 2000 and became diplomates.

Measurements: Predictor variables included components of the Residents' Annual Evaluation Summary ratings and American Board of Internal Medicine (ABIM) certification examination scores.

Results: 2 performance measures independently predicted disciplinary action. A low professionalism rating on the Residents' Annual Evaluation Summary predicted increased risk for disciplinary action (hazard ratio, 1.7 [95% CI, 1.3 to 2.2]), and high performance on the ABIM certification examination predicted decreased risk for disciplinary action (hazard ratio, 0.7 [CI, 0.60 to 0.70] for American or Canadian medical school graduates and 0.9 [CI, 0.80 to 1.0] for international medical school graduates). Progressively better professionalism ratings and ABIM certification examination scores were associated with less risk for subsequent disciplinary actions; the risk ranged from 4.0% for the lowest professionalism rating to 0.5% for the highest and from 2.5% for the lowest examination scores to 0.0% for the highest.

Limitations: The study was retrospective. Some diplomates may have practiced outside of the United States. Nondiplomates were excluded.

Conclusion: Poor performance on behavioral and cognitive measures during residency are associated with greater risk for state licensing board actions against practicing physicians at every point on a performance continuum. These findings support the Accreditation Council for Graduate Medical Education standards for professionalism and cognitive performance and the development of best practices to remediate these deficiencies.


Grahic Jump Location
Figure 1.
Incidence of disciplinary actions, by program director rating.

Incidence of disciplinary actions per 10 000 person-years over 16 years for 66 171 internal medicine diplomates, based on professionalism rating by the program director after the first year of residency. Ratings range from 1 (lowest) to 9 (highest). Error bars indicate 95% CIs.

Grahic Jump Location
Grahic Jump Location
Figure 2.
Incidence of disciplinary actions, by certification on intial examination attempt.

Incidence of disciplinary actions per 10 000 person-years over 16 years for 66 171 internal medicine (IM) diplomates, based on performance on their initial attempt at the IM certification examination Error bars indicate 95% CIs.

Grahic Jump Location




Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).


Submit a Comment/Letter
Addressing unprofessional behavior in residents
Posted on June 6, 2008
Denise M. Dupras
Mayo Clinic
Conflict of Interest: None Declared

To the Editor: I read with great interest the article (1) by Papadakis and colleagues on the relationship between performance during internal medicine training and subsequent disciplinary action by state licensing boards. I was particularly interested in the relationship between poor ratings on professionalism by program directors and subsequent disciplinary action.

The decision of a program director to give a particular rating on any dimension of the ABIM Resident's Evaluation Summary is clearly influenced by many sources of data. With regards to professionalism, this should include input from patients, peers, allied health and other members of the health care team; however, at the time this study was conducted I suspect little of that information was available. This raises the issue of what led program directors to give low ratings in professionalism.

This leads to the question of how professionalism should be determined and by whom. Hickson and colleagues at Vanderbilt describe a disruptive behavior pyramid and a process for addressing unprofessional behaviors (2). As suggested by Papadakis and colleagues (1) this framework could serve as a model for the entire medical education continuum. The Vanderbilt system relies heavily on patient and family complaints; however, peers and allied health colleagues spend more time with residents in training than patients and faculty. Peers may be in the best position to make observations about behaviors and provide meaningful input and feedback about perceived unprofessional actions.

The introduction of a peer evaluation system has allowed us to identify and address concerns about professionalism very early in the training cycle. As described at Vanderbilt, we have a multi-level, stepwise review process. Minor concerns brought forth by peer, nursing or other allied health are reviewed at a weekly operations meeting attended by the program director (PD), chief medical residents (CMRs), and associate program directors (APD) including the competency committee chair. Typically the CMRs convene an informal meeting with the resident where the issues are discussed and feedback is provided. If a resident does not respond to feedback, the resident, a CMR, and one or more APDs meet to discuss the concerns. The next level of intervention is a review at a monthly competency committee meeting. The committee reviews the issues and makes a recommendation to the PD regarding the nature of further disciplinary action, which may lead to probation. Like, Vanderbilt, we have found that this multi-level intervention is an effective process to deal with disruptive behaviors.

References 1. Papdakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS. Performance during Internal Medicine residency training and subsequent disciplinary action by State licensing boards. Ann Intern Med. 2008;148:869-876. 2. Hickson GB, Pichert JW, Webb LE, Gabbe SG. A complementary approach to promoting professionalism: identifying, measuring and addressing unprofessional behaviors. Acad Med. 2007;82:1040-1048

Conflict of Interest:

None declared

Misleading to say that maintenance of certification has been imposed by medical profession
Posted on June 9, 2008
Edward J. Volpintesta
No Affiliation
Conflict of Interest: None Declared

The authors make a statement that requires clarification. They say that "the medical profession has imposed requirements for maintenance of certification".

But it is the American Board of Medical Specialties (ABMS) and not the medical profession that is promoting maintenance of certification in its current form. Indeed many physicians feel that the ABMS has assumed unrightful and unregulated authority over their professional lives. And that CME in general has become a profitable cottage industry that depends more on doctors' fears of not being good enough until they are recertified than it does on improving their professionalism.

Perhaps the rush to "performance assessment" is an over-reaction to the Institute of Medicine's (IOM) 1999 report, To Err is Human. The questionable conclusions to that report sent a shock wave of fear to the public, reporting that 46,000 to 98,000 preventable hospital deaths occurred yearly. Never mind that in 2001 a follow-up study appearing in JAMA lowered the estimate to 5,000 to 15,000 deaths. It never received any publicity.

Initial certification has value as a final exam after finishing residency. But maintenance of certification is an entirely different matter. The ABMS fails to understand that doctors tailor their practices over time and their cognitive and procedural skills change accordingly. Nowhere is this more obvious than in general internal medicine and family medicine.

In addition, maintenance of certification (recertification) must eliminate the pass/fail approach. Ideally, recertification should be a process that doctors look forward to as an enriching experience not as a threatening one that can harm their reputations and livelihoods. It should be an opportunity to assess the knowledge they need and use and improve areas of weakness.

But for many just the opposite occurs. Doctors, fearing that they might be dropped from HMO panels engorge themselves on knowledge they do not need. They take board review courses that promise to improve their scores. I saw one that guaranteed a passing grade. In this context, recertification does not serve doctors at all. It exploits them.

The ABMS is a formidable presence that needs to be better regulated by doctors in practice so that it serves, and doesn't harm them. Its self- proclaimed role as final arbiter of doctors' competence and as custodian of the public's safety is an assumption that doctors have accepted placidly. But,increasingly doctors are criticizing the boards in the medical literature.

Doctors' lives are over-regulated and over-drudged as it is, with an endless number of demands on their time and energy; often depriving them of peace of mind and tranquility. Worse, they can be a source of distraction that interferes with diagnosis and treatment. They do not need more.

Medicine is being practiced in an ever-worsening hostile environment. Health insurers' regulations and the imminent threat of frivolous malpractice suits have greatly reduced the satisfaction of a career in medicine. The last thing doctors need is to be sold out by the institutions that are supposed to help them.

Apparently, under the Canadian system, certification is maintained by accumulating a certain number of approved CME credits appropriate to one' scope of practice. There are no recertication exams.

This practical approach eliminates the doctors' force-feeding themselves with knowledge just to pass a test; and the waste of time and energy that accompanies it. What's more it allows doctors to tailor their education to their needs.

The ABMS should take a serious look at the Canadian process of maintenance of certification.

Conflict of Interest:

None declared

Should recertfication distinguish between academic and practicing generalists?
Posted on June 10, 2008
Edward J. Volpintesta
No Affiliation
Conflict of Interest: None Declared

To state that "the public has the right to expect that practicing physicians will participate in "performance assessment" in a study funded by the ABIM Foundation seems self-serving, especially when one considers that the exams are costly and that additional certification in hypertension, diabetes, and sleep disorders, to name a few are also being planned.

Most studies correlating board recertification and performance have limitations; and solid conclusions cannot be drawn. Besides the costs and consumption of time, recertification exams do not take into account that doctors' skills and practices are different: demographics, proximity to academic medical centers, the number and variety of specialists available, socioeconomic factors, and the expectations of the patients in the community all play a role in shaping a doctors' practices. There are other factors as well.

Clearly, physicians' competence can only be evaluated in the context of their individual practices. The ABMS doesn't seem to understand this, as well-intentioned as they may be. Perhaps they need to distinguish more clearly distinguish between academic generalists and practcing generalists.

Conflict of Interest:

None declared

Disciplinary studies are not the domain of the ABMS
Posted on June 10, 2008
Edward J. Volpintesta
No Affiliation
Conflict of Interest: None Declared

Attempting to correlate cognitive scores with disciplinary actions could have dangerous ramifications. It is little consolation that this information is only available to investigators in the ABIM study. If it were to fall into the wrong hands, it is possible that a physician undergoing a frivolous malpractice suit could have this information used against him.

How? A plaintiff's attorney, having gained this information could imply that the likelihood of a defendant physician being negligent is high because he scored poorly on a board exam. As paranoiac as it sounds, this is something to think about.

Physicians are undergoing hyper-scrutiny in almost everything they do. Pretty soon they will hesitate to treat a patient for fear of being second-guessed. In fact this may be called the "Age of Physician Hyper- Scrutiny". Hyperscrutiny can diminish physicians' confidence and job satisfaction because they are led to believe that they never be good enough. What's worrisome is that behind many of these studies and regulations are our colleagues in academia who are supposed to be helping physicians in practice. Have they become so disconnected from the reality of their non-academic medicine? Even more troublesome is how little criticism they receive.

The authors mention that "licensing actions taken by state medical boards mostly detect egregious behavior ". My answer is what business is that of the ABMS? It not its role to act as policeman for the medical profession. It is neither a licensing nor a disciplinary body and has not been chartered by the government to act as one. Leave disciplining to the medical boards. And spend more time on helping doctors to remain competent as they struggle to maintain their integrity and skills while dealing with the never-ending sea of administrative and legal intrusions that make it almost impossible to be a good and compassionate physician.

Conflict of Interest:

None declared

Can Non-Cognitive Traits During Residency Predict Future Disciplinary Action?
Posted on June 11, 2008
Arun Sivanandam
Vattikuti Urology Institute, Henry Ford Hospital, Detroit, MI
Conflict of Interest: None Declared

Re: Papadakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS. Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Ann Int Med. 2008;148:869- 76.

To The Editor: We read with interest associational analysis between performance during Internal Medicine residency training and disciplinary action by State Licensing Boards (SLB) reported by Maxine et al. We observe a major mismatch between the conglomerate assessment of cognitive, non-cognitive and educational-behavioral traits together with surrogate endpoints of certification evaluation and punitive action by SLBs through a bureaucratic complaint-driven machine. The objective of residency is to gain cognitive and educational skills. Crucially, residency is not an appropriate study period of overall personality traits since non-cognitive die is cast by age 16 with family upbringing as a major driver. Logically, cognitive, non-cognitive, and psychomotor-behavioral traits should be studied with endpoints specific to each trait because they may not follow a similar longitudinal growth trends.

This study missed department and program director as significant variables. We strongly suspect a correlation between poorly rated programs and the residents' SLB disciplinary actions. From personal experience, there is no back-end mechanism to ensure accurate concurrent recording of ACGME competency assessments. Coupled with personality differences among program directors, this is a slippery slope for subjectivity of assessment by program directors in residents' evaluation process. As such, adjustment for program-director variation in competency assessments should have been performed through tests of extrabinomial variation.

We are surprised by author's decision to exclude physicians who received disciplinary actions during residency from analysis. Receipt of a disciplinary action could alter behavioral status and act as a confounder in risk analyses. Moreover, we fail to understand how two percent of diplomates with <3 years residency training could be included in analysis without adequate adjustment.

Figures 1 and 2 have wide fluctuations in confidence intervals for low professionalism and low ABIM z-scores respectively that do not indicate significance between categories of either rating. While hazard ratios were presented, confidence interval for adjusted hazard ratio of Canadian medical school graduates indicates non-significance (Table 3). Additionally, a lack of trend analysis reports for both associational studies casts doubts about the predictive validity of these findings.

In addition to the endpoint parameter of disciplinary action by SLBs, inclusion of other markers of poor physicianship is justified, such as success in private practice, feedback from patients and hospitals, and established physician rating systems. In absence of direct assessment of patient care, extreme caution needs to be exercised in studying non-cognitive traits as a variable.

Arun Sivanandam, MS Mahendra Bhandari, MS, MCh(Uro), FAMS, DSc(Hon) Vattikuti Urology Institute Henry Ford Hospital, Detroit MI

Conflict of Interest:

None declared

What determines professionalism?
Posted on December 1, 2008
Ahmad Kaako
UTCOM Internal Medicine Resident
Conflict of Interest: None Declared

To the Editor: What is professionalism in our medical practice? Is the measurement of non"“professional behavior in residency programs really reliable? What are the factors that influence the Program director or faculty member to determine that a resident has lapses in his/her professionalism? Is an (ABIM) in-training exam score a good indicator of the medical knowledge? All these questions should be taken in consideration when talking about the association between non-professional behavior, poor exam scores and disciplinary actions.

One definition of professionalism per EPSTEIN AND HUNDERT (1) is: "Professional competence is the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served."

Different physicians may have different values, cultures and reactions to same stimulator within the acceptable range of normal. So the same act could be professional to one physician and non-professional to another. The measurement of professionalism is variable from one program to another according to each individual system. For instance; some rely more on the communication skills with peers and nurses, others rely more on the interpersonal communication between the program director and the resident which, in this case, my be not very reliable.

It is worthy to indicate that our program has developed its own policy to address professionalism issues. An accountability committee has been formed to provide an early feedback to residents about any consistent unprofessional behavior. A scoring system has been developed to help rate this unprofessional behavior and the ways for remediation. It is not clear whether this policy would decrease unprofessional behaviors, but it would definitely help residents to identify deficiencies and lapses they have to work on, to avoid major consequences such as non-promotion.


(1) Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287:226-35. [PMID: 11779266]

Conflict of Interest:

None declared

Submit a Comment/Letter

Summary for Patients

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.


Buy Now for $32.00

to gain full access to the content and tools.

Want to Subscribe?

Learn more about subscription options

Related Articles
Journal Club
Topic Collections
PubMed Articles
Forgot your password?
Enter your username and email address. We'll send you a reminder to the email address on record.