ABIM examines the impact of adding open-book to a secure assessment

Philadelphia, PA, August 16, 2017 – More than 800 doctors participated in a research study to help the American Board of Internal Medicine (ABIM) determine the feasibility and impact of incorporating electronic resources used at the point of care into Maintenance of Certification (MOC) exams.

ABIM conducted the study based on physician requests to make the secure MOC exam "open-book" to reflect how doctors look up information during daily practice to diagnose and treat patients. The study results were published this week in Annals of Internal Medicine.

"Practicing physicians have been directly involved in assessment innovations at ABIM that help better reflect how they use their current knowledge supplemented by electronic resources to make important care decisions every day," said Richard J. Baron, MD, president and CEO of ABIM. "Our open-book study invited doctors to explore a new exam format and help determine whether it could still support a rigorous process robust enough for public accountability."

ABIM announced plans to introduce open-book MOC assessment options beginning in 2018, including new shorter Knowledge Check-Ins physicians can take at home and the traditional 10-year MOC exam taken at a testing center. The study will help inform some aspects of how ABIM will implement open-book assessments.

About the study

In spring 2016, ABIM launched its Research Study on Open-Book Assessments, which used a simulated exam to test four different scenarios: a closed-book exam, a closed-book exam with additional completion time, an open-book exam and an open-book exam with additional completion time.

ABIM invited doctors who took the Internal Medicine MOC exam or passed the Internal Medicine Certification exam between 2012 and 2015 to participate in the study. About 825 physicians volunteered to participate and were randomly assigned to one of the four scenarios. Those in the open-book scenarios had access to UpToDate®, an online resource widely used by internists and internal medicine subspecialists.

The authors found:

Post-study survey results indicated that having access to resources did reduce physicians' anxiety about the exam.

The open-book option enhanced the exam's capacity to differentiate between high and low performances and would likely have minimal impact on the pass rate for the exam.

Physicians in open-book conditions used more time, but time constraints did not adversely affect exam performance and did not change the specific skill or factor that is measured by the exam.

One limitation to the study was that it only tested the use of one external resource. Researchers noted the need to better understand the variety of medical resources doctors use and how increasing the number of resources would impact the exam's efficacy.

"The open-book exam did not become solely a measure of physicians' information retrieval skills; it remained an important measure of physicians' current clinical knowledge, " said Rebecca S. Lipner, PhD, lead author and ABIM Senior Vice President of Assessment and Research. "Even when physicians had access to an online resource, their knowledge still mattered and may be correlated with better patient care."

Additional Information

View the research article in Annals of Internal Medicine.

Review more research from ABIM and other organizations about board certification, MOC and physician assessment.

Learn more about the ABIM MOC program, which includes independent assessment and continuous learning activities that doctors can complete to stay current with medical knowledge.

###

ABIM Board Certified Doctors Make a Difference
Internists and subspecialists who earn and maintain board certification from the American Board of Internal Medicine (ABIM) differentiate themselves every day through their specialized knowledge and commitment to continual learning in service of their patients. Established as an independent nonprofit more than 80 years ago, ABIM continues to be driven by doctors who want to achieve higher standards for better care in a rapidly changing world. Visit ABIM's blog to learn more and follow ABIM on Facebook and Twitter. ABIM is a member of the American Board of Medical Specialties.