Medical EducationOriginal Article

Meaningful use of COMSAE Phase 1 in preparation for COMLEX-USA Level 1

Xiaolin Wang, PhD; Hotaka Maeda, PhD; Brandon Craig, PhD; Tsung-Hsun Tsai, PhD; Jeanne M. Sandella, DO; and Marie Fleury, DO, MBA
Notes and Affiliations
Notes and Affiliations

Received: July 22, 2020

Accepted: February 4, 2021

Published: April 8, 2021

  • Xiaolin Wang, PhD, 

    National Board of Osteopathic Medical Examiners, Corporate Offices and National Center for Clinical Skills Testing, Chicago, IL, USA

  • Hotaka Maeda, PhD, 

    National Board of Osteopathic Medical Examiners, Corporate Offices and National Center for Clinical Skills Testing, Chicago, IL, USA

  • Brandon Craig, PhD, 

    National Board of Osteopathic Medical Examiners, Corporate Offices and National Center for Clinical Skills Testing, Chicago, IL, USA

  • Tsung-Hsun Tsai, PhD, 

    National Board of Osteopathic Medical Examiners, Corporate Offices and National Center for Clinical Skills Testing, Chicago, IL, USA

  • Jeanne M. Sandella, DO, 

    National Board of Osteopathic Medical Examiners, Corporate Offices and National Center for Clinical Skills Testing, Chicago, IL, USA

  • Marie Fleury, DO, MBA, 

    National Board of Osteopathic Medical Examiners, Corporate Offices and National Center for Clinical Skills Testing, Chicago, IL, USA

Abstract

Context: The Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) is a three level national standardized licensure examination for the practice of osteopathic medicine. The Comprehensive Medical Self Assessment Examination (COMSAE) is a three phase self assessment tool designed to gauge the base knowledge and ability of candidates preparing for COMLEX-USA.

Objectives: To investigate how COMSAE Phase 1 (Phase 1) was used by candidates and how completing Phase 1 impacted their performance on the COMLEX-USA Level 1 (Level 1) examination.

Methods: Using data from the 2018–2019 administration of Level 1 and Phase 1 examinations, we counted the frequency of the unique Phase 1 forms taken by the candidates and calculated the correlation between the candidates’ first attempt Phase 1 scores and the number of Phase 1 forms taken. We then calculated the correlation between the Level 1 scores and the Phase 1 scores. Next, we applied a multilevel regression model to examine the candidates’ score improvement on the multiple Phase 1 forms taken. Finally, we investigated the effect of practicing through Phase 1 on the candidates’ Level 1 performance using logistic regression models.

Results: The majority of candidates took one (2,414; 33.9%) to two (2,196; 30.8%) timed Phase 1 forms prior to the Level 1 examination. There was a significant negative correlation (r=−0.48, t(6,505)=−44.05, p<0.001) between the candidates’ first attempt Phase 1 scores and the number of Phase 1 forms taken. There was a strong and positive correlation (r=0.66 to 0.74, p<0.001) between Phase 1 and Level 1 scores. With other variables controlled, on average, candidates’ Phase 1 scores increased 23.2 points on one attempt from the previous attempt. Having the most recent Phase 1 score controlled, a greater number of Phase 1 forms taken was associated with an improvement on the Level 1 performance.

Conclusions: The significant correlation between Phase 1 and Level 1 performance provided validity evidence for Phase 1. Moreover, our results suggested that candidates, especially those with lower performance on their initial Phase 1 attempt, might improve their Level 1 performance by taking multiple Phase 1 forms to monitor their academic improvement and gauge their readiness for Level 1.

Read Full Article