Medical EducationORIGINAL ARTICLE
Abstract
Context: As Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 has changed to Pass/Fail scoring, residency programs that required minimum Level 1 scores for applicant consideration may choose to focus on COMLEX-USA Level 2-Cognitive Evaluation (Level 2-CE) target scores for applicant selection. Therefore, finding ways to predict passing and high Level 2-CE performance based on students’ past performance and to guide their study accordingly is essential for helping students succeed in and beyond medical school.
Objectives: The purpose of this retrospective study is to evaluate the predictive value of major performance measures from pre-admission to clerkship years on Level 2-CE. Then, based on the predictive value of those measures, the objective is to establish a predictive model and optimal cutoff scores with strong predictors to advise students on their preparation of Level 2-CE.
Methods: School-based performance measures for 948 first-time takers of the Level 2-CE Testing Cycles of 2019/20 to 2023/24 were analyzed. Correlational and multiple regression analyses were utilized to establish a predictive model utilizing: (1) preadmission and preclerkship performance (Medical College Admission Test [MCAT], undergraduate science grade point average [GPA], and preclerkship examination average); (2) national examination performance including the new COMLEX-USA Level 1 pass/fail-only status, individual and average clinical subject Comprehensive Osteopathic Medical Achievement Test (COMAT) scores, and the less studied Comprehensive Osteopathic Medical Self-Assessment Examination (COMSAE) Phase 2; and (3) clinical evaluation scores by preceptors. Then, receiver operating characteristic (ROC) curves were utilized to identify the optimal cutoff scores on the average clinical subject COMATs and COMSAE Phase 2 for student advising.
Results: A predictive model of COMLEX Level 2-CE was established with average clinical subject COMAT scores, first-time COMSAE Phase 2, preclerkship examination mean, and COMLEX Level 1 Pass/Fail status as the significant predictors. This model explained 73.9 % of the variance in Level 2-CE performance. Optimal cutoffs of the average clinical subject COMAT scores and first-time COMSAE Phase 2 performance were identified for passing Level 2-CE (COMSAE=447, average COMAT score=94.4) as well as having a high performance of Level 2-CE (650 & 700, respectively).
Conclusions: This study not only added evidence in support of previous studies on the bivariate associations between Level 2-CE and individual major performance measures from the preadmission to clerkship years, but also explored the use of the less-studied COMSAE Phase 2 in predicting Level 2-CE outcomes and provided a better predictive model utilizing the combination of individual performance measures. Most importantly, the current study also demonstrated a way to set optimal cutoff scores on strong predictors during the clerkship years to help the school guide students to better success on COMLEX Level 2-CE and long-term residency goals.