|
 |
|
|||||||
|
|||||||||
MEDICAL EDUCATION |
Relation Between Variables of Preadmission, Medical School Performance, and COMLEX–USA Levels 1 and 2 Performance
Address correspondence to Donna Dixon, PhD, New York College of Osteopathic Medicine of New York Institute of Technology, PO Box 8000, Old Westbury, NY 11568-8000.E-mail: ddixon{at}nyit.edu.
The purpose of this study was to investigate the relation between preadmission academic variables, osteopathic medical school performance in the first 2 years, and performance on the Comprehensive Osteopathic Medical Licensing Examination (COMLEX–USA) Levels 1 and 2. The study group comprised 174 students in the class of 2001 of the New York College of Osteopathic Medicine of the New York Institute of Technology, Old Westbury. Preadmission academic variables were the Medical College Admission Test (MCAT) subscores and undergraduate grade point averages (UGPAs).
Physical sciences (physical MCAT) and biological sciences MCAT (biological MCAT) subscores were significantly correlated with COMLEX–USA Level 1 performance, and verbal reasoning, physical, and biological MCAT subscores were correlated with Level 2 performance. COMLEX–USA Level 1 performance was correlated with the year 1 grade point average (GPA) (0.78) and the year 2 GPA (0.83). COMLEX–USA Level 2 performance was correlated with the year 1 GPA (0.64) and the year 2 GPA (0.68). Strong correlations existed between all year 1 and most year 2 course grades and COMLEX–USA Level 1 scores. School-specific regression models that were highly predictive of school performance and COMLEX-USA Level 1 performance were developed. COMLEX–USA Level 1 predictive models used preadmission variables combined with the year 1 and year 2 course grades. The year 2 courses' model had a higher predictive value for COMLEX–USA Level 1 performance (R2 = 0.81) than the year 1 courses' model (R2 = 0.77). Significant predictors of COMLEX–USA Level 1 performance in the combined year 1 and 2 courses' model were the pharmacology II, neuropathology, and pulmonary pathology grades, and the verbal and physical MCAT subscores (R2 = 0.820).
Medical schools have traditionally used Medical College Admission Test (MCAT) scores and undergraduate grade point averages (UGPAs) in the selection of candidates for admission. The validity of using preadmission academic variables to predict student performance has been studied. Most studies have focused on the relation of these variables to medical licensing examination scores as performance end-points. Only a few studies have examined the relationship between preadmission variables, osteopathic medical school performance, and Comprehensive Osteopathic Medical Licensing Examination–USA (COMLEX–USA) performance, and the results were not consistent. Medical College Admission Test scores were reported to be correlated with COMLEX–USA Level 1 and Level 2 scores,1,2 whereas another study found no significant correlations between preadmission data and COMLEX–USA Level 1 scores.3 The relation of preadmission variables to osteopathic medical school performance and COMLEX–USA scores has not been sufficiently investigated at other schools. There has also been a lack of predictive validity studies of the relation of pre-medical and medical school performance to COMLEX–USA Levels 1 and 2.
Researchers in the current study previously reported significant correlations between MCAT subscores and year 1 and year 2 GPAs and COMLEX–USA Level 1 performance at the New York College of Osteopathic Medicine of the New York Institute of Technology (NYCOM), Old Westbury.4 The purpose of the present study was to further examine the relation between preadmission variables and student academic performance, including year 1 and 2 GPAs, individual course grades, and COMLEX–USA Level 1 performance at NYCOM.
 |
Research Review |
|---|
 Top Research Review MethodsResultsCorrelation Between GPAs and...CommentReferences |
|---|
Allopathic Medical Education
Preadmission academic variables were correlated with performance on the United States Medical Licensing Examination (USMLE) Step 1.5-7 Elam and Johnson5 found that the biological sciences MCAT (biological MCAT) subscore was the only significant preadmission predictor of USMLE Step 1 performance. A study using data from 14 allopathic medical schools reported that total MCAT scores and undergraduate GPAs were predictors of USMLE Step 1 performance.6 Silver and Hodgson7 found that total MCAT scores and science UGPAs were related to performance on the National Board of Medical Examiners Part I (NBME I), a forerunner of the present USMLE Step 1.
Osteopathic Medical Education
Baker et al3
examined the relation between COMLEX–USA Level 1 scores, preadmission
variables, and medical school performance in detail, and they found no
correlations between preadmission variables and COMLEX–USA Level 1
performance. Cope et
al8 used
preadmission variables in predictive models and reported that the biological
MCAT and cumulative UGPA were predictive of COMLEX–USA Level 1 scores.
Moderate correlations were found between COMLEX–USA Levels 1 and 2 and
total MCAT
scores.1,2
COMLEX–USA Level 1 performance was reported to be strongly correlated with the year 1 and year 2 GPAs, as was Level 2 performance.1,3,8,9 A multisite study using data from 16 osteopathic medical schools found strong correlations between GPAs and COMLEX Level 1 scores within schools.10
|
Methods |
|---|
|
TopResearch ReviewMethodsResultsCorrelation Between GPAs and...CommentReferences |
|---|
The class of 2001 study group comprised 174 students (93 men and 81 women). These students completed the first 2 years of the standard NYCOM curriculum in 1999 and took the COMLEX–USA Level 1 examination in 1999 and the Level 2 examination in 2000. The database for analysis included preadmission academic variables, medical school course grades in years 1 and 2, and COMLEX–USA Level 1 and 2 scores. Individual course grades were obtained from institutional databases, and total COMLEX–USA scores were those reported by the National Board of Osteopathic Medical Examiners to the institution. Individual course grades used were those for written examinations. Grade point averages were calculated for the first year (year 1 GPA), the second year (year 2 GPA), and years 1 and 2 (2-year cumulative GPA).
Preadmission data were obtained from the American Association of Colleges of Osteopathic Medicine Application Service. The preadmission academic variables used in the analysis were MCAT subscores and UGPAs. Medical College Admission Test subscores used were verbal reasoning (verbal MCAT), physical sciences (physical MCAT), and biological MCAT. Pearson's correlation coefficients were calculated with Bonferroni adjustments. Researchers performed multiple linear regression analysis by entering all independent variables in a single step and constructed regression models to identify an optimal combination of predictors of COMLEX–USA Level 1 performance. All statistical analyses were calculated with Statistical Program for the Social Sciences (SPSS) statistical software for Windows, version 10.0.
|
Results |
|---|
|
TopResearch ReviewMethodsResultsCorrelation Between GPAs and...CommentReferences |
|---|
Descriptive Statistics for the Study Group
Preadmission data for the study group were mean MCAT subscores (verbal, 7.8; physical, 8.2; biological, 8.5) and mean UGPAs (science, 3.33; nonscience, 3.45; cumulative, 3.38). The study group had a mean COMLEX–USA Level 1 score of 514.87 (SD 73.90) and a mean COMLEX–USA Level 2 score of 525.87 (SD 74.12).
Relationship Between Preadmission Variables and Year GPAs
The correlations of preadmission academic variables with year 1 and 2 GPAs
are shown in Table 1.
The physical MCAT subscore, biological MCAT subscore, and the science and
nonscience UGPA were significantly correlated with the year 1 GPA. The
biological MCAT subscore, science UGPA, and nonscience UGPA were correlated
with the year 2 GPA. The five preadmission variables were used in a regression
model to predict the year GPAs. The physical MCAT subscore, biological MCAT
subscore, and science UGPA were significant predictor variables for the year 1
GPA (R2 = 0.284), and the biological MCAT subscore was a
predictor for the year 2 GPA (R2 = 0.157).
|
Relationship Between Preadmission Variables and COMLEX–USA Performance
Physical and biological MCAT subscores were correlated with COMLEX Level 1
and the verbal, physical, and biological MCAT subscores were correlated with
COMLEX Level 2 (Table
1). The science and nonscience UGPAs were not correlated.
Scientists used the five preadmission variables in a regression model
(Table 2) and found
that the physical and biological MCAT subscores were modest predictors of
COMLEX Level 1 performance (R2 = 0.232). The verbal MCAT
subscore was a weak predictor of COMLEX Level 2 performance (R2
= 0.168).
|
|
Correlation Between GPAs and COMLEX–USA Level 1 and Level 2 Performance |
|---|
|
TopResearch ReviewMethodsResultsCorrelation Between GPAs and...CommentReferences |
|---|
The correlations between medical school GPAs and COMLEX–USA scores are shown in Table 3. COMLEX–USA Level 1 scores were strongly correlated with both the year 1 and 2 GPAs. The correlations between year 1 and 2 GPAs and COMLEX–USA Level 2 scores were lower than for Level 1 scores. COMLEX–USA Level 1 performance and COMLEX–USA Level 2 performance were correlated (0.723). The year 1 and year 2 GPAs were predictors of COMLEX–USA Level 1 scores in a regression model (R2 = 0.702). The addition of preadmission variables to the model with the year 1 and year 2 GPAs model (Table 2) resulted in a slightly higher R2 value (0.774).
|
Relationship Between Preadmission Variables, Year 1 Courses, and COMLEX–USA Level 1 Performance
First-year basic science courses included anatomy, biochemistry, general
pathology, genetics, histology, microbiology, neuroscience, osteopathic
principles I, pharmacology, and physiology. Most of the first-year courses
were moderately correlated with only the physical and biological MCAT
subscores (P<.05). Strong correlations existed between all year 1
course grades and COMLEX–USA Level 1 scores
(Table 4).
|
Scientists used all year 1 course grades as variables in a regression analysis to predict COMLEX–USA Level 1 performance (Table 2; R2 = 0.652). Physiology and pharmacology grades were predictors of COMLEX–USA Level 1 performance. The addition of preadmission variables to the year 1 courses' model resulted in a higher R2 value (0.663).
Relationship Between Preadmission Variables, Year 2 Courses, and COMLEX–USA Level 1 Performance
Year 2 courses included the following pathology courses: cardiovascular,
gastrointestinal, musculoskeletal, obstetric and gynecologic, pulmonary,
renal, and neuropathology. The systems courses were cardiovascular,
gastrointestinal, nervous, obstetrics-gynecology, renal, respiratory, and
rheumatologic systems. Other specialty courses in year 2 included dermatology,
endocrinology, family practice, hematology, immunology, pharmacology II,
osteopathic principles II, pediatrics, psychiatry, surgery, and toxicology.
When correlations between preadmission variables and individual year 2 grades
were studied, the only significant correlations were between the biological
MCAT subscore and cardiovascular pathology and between the verbal and physical
MCAT subscores and pediatrics (P<.05).
Table 5 shows the correlations between individual year 2 course grades and COMLEX–USA Level 1 scores. All courses, except musculoskeletal pathology, family practice, and psychiatry, were significantly correlated with Level 1 (P<.05). All year 2 course grades were used in a regression model to predict COMLEX–USA Level 1 performance. Pharmacology II and pediatrics grades were found to be predictors of COMLEX–USA Level 1 performance (R2 = 0.730). When preadmission variables were added to the model, the predictive variables were pharmacology II, neuropathology, pulmonary pathology, and the physical and biological MCAT subscores. The R2 for this model was 0.807.
|
When all year 1 and 2 course grades were used in a predictive model for COMLEX–USA Level 1, physiology, pharmacology II, and neuropathology were significant variables (R2 = 0.771). When preadmission variables were added to the model, the significant predictors were pharmacology II, neuropathology, pulmonary pathology, and the verbal and physical MCAT subscores (R2 = 0.820) (Table 2).
|
Comment |
|---|
|
TopResearch ReviewMethodsResultsCorrelation Between GPAs and...CommentReferences |
|---|
This study has demonstrated significant correlations between academic preadmission variables, osteopathic medical school performance in the first 2 years, and COMLEX–USA Level 1 and 2 scores. The physical and biological MCAT subscores were correlated with both year 1 and 2 osteopathic medical school GPAs. Regression models identified the physical and biological MCAT subscores as predictor variables for COMLEX–USA Level 1 and the verbal MCAT subscore as a predictor for COMLEX–USA Level 2. Most previous studies have used composite MCAT scores (sum of the individual subscores) as a variable. Researchers reporting on the predictive validity of MCAT scores noted that the use of MCAT composite scores limits its use as a predictor and recommended that future studies investigate the differential validities of subscores.11 The use of MCAT subscores in the present study was more informative. While the preadmission variables alone were not highly predictive of either COMLEX–USA Level 1 or 2 performance, together with other performance measures preadmission variables improved the total predictive validity.
Only the physical and biological MCAT subscores were moderately correlated with first-year grades. Individual course grades in the first 2 years were strongly correlated to COMLEX–USA Level 1 scores with the exception of a few year 2 courses. Although many of the correlations between course grades and COMLEX–USA Level 1 are similar to some previously reported,3 differences among school courses make data comparisons between individual schools difficult to interpret.
Individual year 1 and 2 course grades with preadmission variables were used in predictive models of COMLEX–USA Level 1 scores. In all the models tested, the addition of preadmission variables raised the predictive value. The year 2 courses' model had a higher predictive validity than the year 1 model. The combined model, using all year 1 and 2 course grades and preadmission variables, had a higher predictive validity than the cumulative year 1 and 2 GPAs model. The predictive value of these models is higher than previously reported Level 1 models.8 The higher values found for COMLEX–USA Level 1 predictive models at NYCOM may be due to greater class heterogeneity and larger class size. More detailed studies of data from other NYCOM classes are needed to further confirm the predictive value of the COMLEX–USA Level 1 predictive models. A factor that might improve the predictive value of the preadmission variables would be the use of an undergraduate school selectivity index.
This study has identified many academic performance variables that predict both year 1 and 2 GPAs and COMLEX–USA Level 1 performance. These findings may be useful in predicting future performance of NYCOM students on Level 1 examinations. Further research is needed to determine the relationships between these variables, performance in the third and fourth years, and performance on COMLEX–USA Levels 2 and 3.
|
Acknowledgment |
|---|
The author thanks Arnold L. Nagler, PhD, (NYCOM) for his support and Larry R. Stepp, PhD (also of NYCOM), for his helpful comments on this manuscript.
From New York College of Osteopathic Medicine of New York Institute of Technology in Old Westbury.
|
References |
|---|
|
TopResearch ReviewMethodsResultsCorrelation Between GPAs and...CommentReferences |
|---|
1. Meoli FG, Wallace WS, Kaiser-Smith J, Shen L. Relationship of osteopathic medical licensure examinations with undergraduate admission measures and predictive value of identifying future performance in osteopathic principles and practice/osteopathic manipulative medicine courses and rotations. J Am Osteopath Assoc.2002; 102:615 -620.
2. Evans P, Goodson LB, Schoffman SI. Relationship between academic
achievement and student performance on the Comprehensive Osteopathic Medical
Licensing Examination–USA Level 2. J Am Osteopath
Assoc. 2003;103:331
-336.
3. Baker HH, Cope MK, Fisk R, Gorby JN, Foster RW. Relationship of
preadmission variables and first- and second-year course performance to
performance on the National Board of Osteopathic Medical Examiners'
COMLEX–USA Level 1 Examination. J Am Osteopath
Assoc. 2000;100:153
-161.
4. Dixon D. Correlation of preadmission variables with medical school performance and level 1 COMLEX–USA scores [abstract]. J Am Osteopath Assoc. 2002;102:510 .
5. Elam CL, Johnson MM. NBME Part 1 versus USMLE Step1: predicting scores based on preadmission and medical school performances. Acad Med. 1994;69:155 .
6. Wiley A, Koenig JA. The validity of the Medical College Admission Test for predicting performance in the first two years of medical school. Acad Med.1996; 71(10 Suppl):S83 -S85.[Medline]
7. Silver B, Hodgson CS. Evaluating GPAs and MCAT scores as predictors of NBME I and clerkship performances based on students' data from one undergraduate institution. Acad Med.1997; 72:394 -396.[Medline]
8. Cope MK, Baker HH, Fisk R, Gorby JN, Foster RW. Prediction of
student performance on the Comprehensive Osteopathic Medical Licensing
Examination Level 1 based on admission data and course performance.
J Am Osteopath Assoc.2001; 101:84
-90.
9. Hartman SE, Bates BP, Sprafka SA. Correlation of scores for the
Comprehensive Osteopathic Medical Licensing Examination with osteopathic
medical school grades. J Am Osteopath Assoc.2001; 101:347
-349.
10. Baker HH, Foster RW, Bates BP, Cope MK, McWilliams TE, Musser A, et
al. Relationship between academic achievement and COMLEX–USA Level 1
performance: a multisite study. J Am Osteopath Assoc.2000; 100:238
-242.
11. Koenig JA, Sireci SG, Wiley A. Evaluating the predictive validity of MCAT scores across diverse applicant groups. Acad Med. 1998;73:1095 -1106.[Medline]
This article has been cited by other articles:
|
|
 |
|
  D. J. Sefcik Attention Applicants: Please Submit Emotional Intelligence Scores J Am Osteopath Assoc, September 1, 2005; 105(9): 401 - 402. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
