Asano-Gonnella Center for Research in Medical Education & Health Care

The Jefferson Longitudinal Study of Medical Education is the most comprehensive, extensive and uninterrupted tracking system of its kind maintained in a single medical center. It was implemented in 1970 with the intention to track all medical students atThomas Jefferson University throughout their medical education and professional careers. Data were collected retrospectively for those who matriculated between 1964 and 1970 and prospectively from 1971 to date. The 50th anniversary of the JLS was celebrated in 2015.

The database includes demographics, personal qualities, performance measures in medical college, board scores, responses to matriculation and graduation questionnaires, program directors' ratings of clinical competence in residency and career follow-up surveys. The Longitudinal Study is routinely updated for all graduates using data from local sources as well as the Association of American Medical Colleges (AAMC), American Medical Association (AMA) and American Board of Medical Specialties (ABMS). Follow-up data also includes career outcomes from the AMA’s national data file, faculty appointments from the AAMC, board certification from the ABMS and periodic alumni surveys. Important statistics retrieved from the Longitudinal Study database on demographics, performance on licensing examinations, geographic location, specialty, evaluations of clinical competence in residency and perceptions of students and graduates on medical education at Thomas Jefferson University are routinely prepared for the Sidney Kimmel Medical College’s annual report.

The Longitudinal Study serves as a unique tool for institutional research and academic management of the outcomes assessment requirements of accrediting bodies, including the Liaison Committee on Medical Education (LCME) for the medical college, the Accreditation Council for Graduate Medical Education (ACGME) for residency programs and the Middle States Commission on Higher Education for the University. The most recent LCME accreditation review observed that:

“The Longitudinal Study continues to represent a unique information resource that serves as an invaluable tool for evaluation of educational program effectiveness. The Longitudinal Study enables the Center to provide the medical College with vital information about the intermediate and long-term outcomes of the curriculum.”   

As part of the JLS, we developed 10 psychometrically sound instruments (PDF) for the assessment of educational and patient outcomes and monitoring professional development of physicians-in-training and in-practice.

More than 200 studies using information from the Longitudinal Study have been published in peer-reviewed journals. Sample publications include:

THE IMPACT OF EARLY SPECIALIZATION ON THE CLINICAL COMPETENCE OF RESIDENTS

Joseph S. Gonnella, and J. Jon Veloski

The notion of specialization by addition rather than subtraction was addressed in this study. Presented are the results of a study of the relationship between first year postgraduate training and performance on a medical licensing examination of general clinical competence. The scores of 1,514 Jefferson graduates achieved on Part III of the National Board Examinations (NBE) were analyzed. Residents in family medicine, internal medicine, and flexible programs scored higher on Part III of the NBE than did those in surgery, pediatrics, obstetrics/gynecology, psychiatry and pathology. This finding was true even after a correction was made for their performance on Part II of the NBE and suggests that knowledge and skills in the broad aspects of medicine are not adequately emphasized in the first year of some residency training programs. Although the differences in performance among the various training programs could be due to other factors, it is strongly suggested that changes are needed in some programs to strengthen the general capabilities of residents.

The New England Journal of Medicine. 1982;306:275-277.

This study was replicated 15 years later with similar results:

Joseph S.Gonnella, Mohammadreza Hojat, James B.  M., Erdmann, and J. Jon Veloski. Advances in Health Sciences Education, 1, 125-139.

AN EMPIRICAL STUDY OF THE PREDICTIVE VALIDITY OF NUMBER GRADES IN MEDICAL SCHOOL USING 3 DECADES OF LONGITUDINAL DATA: IMPLICATIONS FOR A GRADING SYSTEM

Joseph S. Gonnella, James B. Erdmann, and Mohammadreza Hojat

Context: It is important to establish the predictive validity of medical school grades.  The strength of predictive validity and the ability to identify at-risk students in medical schools depends upon assessment systems such as number grades, pass/fair (P/F) or honors/pass/fail (H/P/F) systems.

Objective: This study was designed to examine the predictive validity of number grades in medical school, and to determine whether any important information is lost in a shift from number to P/F and H/P/F grading systems.

Subjects: The participants in this prospective, longitudinal study were 6656 medical students who studied at Jefferson Medical College over 3 decades.  They were grouped into 10 deciles based on their number grades in Year 1 of medical school.

Methods: Participants were compared on academic accomplishments in Years 2 and 3 of medical school, medical school class rank, delayed graduation and attrition, performance on medical licensing examinations and clinical competence ratings in the first postgraduate year.

Results: Results supported the short- and long term predictive validity of the number grades.  Ratings of clinical competence beyond medical school were predicted by number grades in medical school.  We demonstrated that small differences in number grades are statistically meaningful, and that important information for identifying students in need of remedial education is lost when students who narrowly meet faculty’s expectations are included with the rest of the class in a broad ‘pass’ category.

Conclusions: The findings refute the argument that knowledge of sciences basic to medicine is not critical to subsequent performance in medical school and beyond if an appropriate evaluation system is used.  Furthermore, the results of this study raise questions about abandoning number grades in favor of a pass/fail system.  Consideration of these findings in policy decisions regarding assessment system of medical students is recommended.

Medical Education. 2004;38:425-434.

THE FATE OF MEDICAL STUDENTS WITH DIFFERENT LEVELS OF KNOWLEDGE: ARE THE BASIC MEDICAL SCIENCES RELEVANT TO PHYSICIAN COMPETENCE?

Mohammadreza Hojat, Joseph S. Gonnella, James B. Erdmann, and J. Jon Veloski

Purpose: This study was designed to test the hypothesis that an early gap in knowledge of sciences basic to medicine could have a sustained negative effect throughout medical school and beyond.

Methods: A longitudinal prospective study of 4,437 students who entered Jefferson Medical College between 1972 and 1991 was conducted in which the students were divided into three groups.  Group 1 consisted of 392 who failed at least one of the three basic sciences courses in the first year of medical school. Group II was comprised of 398 who did not fail but had low first-year grade point averages; and 3,647 of the remaining sample were included in Group III. The groups were compared on retention and dismissal rates, medical school assessment measures, scores on medical licensing examinations, ratings of clinical competency in residency, board certification rates, and faculty appointments.

Results: Significant differences were observed among the three groups confirming the hypothesis that students’ level knowledge in sciences basic to medicine early in medical school could predict later performance during medical school and beyond. Implications for early diagnosis of academic deficiencies, for better preparation of medical students, and for the assessment of clinical competency are discussed.

Advances in Health Sciences Education. 1997;1:179-196.

THE DEVIL IS IN THE THIRD YEAR: A LONGITUDINAL STUDY OF EROSION OF EMPATHY IN MEDICAL SCHOOL

Mohammadreza Hojat, Michael J. Vergare, Kaye Maxwell, George Brainard, Steven K. Herrine, Gerald A. Isenberg, Jon Veloski, and Joseph S. Gonnella

Purpose: This longitudinal study was designed to examine changes in medical students’ empathy during medical school and to determine when the most significant changes occur.

Method: Four hundred fifty-six students who entered Jefferson Medical College in 2002 (n=227) and 2004 (n=229) completed the Jefferson Scale of Physician Empathy at five different times: at entry into medical school on orientation day and subsequently at the end of each academic year. Statistical analyses were performed for the entire cohort, as well as for the “matched” cohort (participants who identified themselves at all five test administrations) and the “unmatched” cohort (participants who did not identify themselves in all five test administrations).

Results: Statistical analyses showed that empathy scores did not change significantly during the first two years of medical school. However, a significant decline in empathy scores was observed at the end of rest of the sample (unmatched cohort, n=335). Patterns of decline in empathy scores were similar for men and women and across specialties.

Conclusions: It is concluded that a significant decline in empathy occurs during the third year of medical school. It is ironic that the erosion of empathy occurs during a time when the curriculum is shifting toward patient-care activities; this is when empathy is most essential. Implications for retaining and enhancing empathy are discussed.

Acad Med. 2009; 84:1182–1191.

This 20-item instrument was developed by Mohammadreza Hojat, PhD and his colleagues at the Asano-Gonnella Center for Research in Medical Education & Health Care to measure empathy in physicians and practicing health professionals (HP-version), medical students (S-version), and health professions students other than medical students (HPS-version).

The JSE has enjoyed broad national and international attention, has been translated into 57 languages and is being used by researchers and educators worldwide. A long list of 528 publications (PDF) by national and international researchers in which the JSE has been used (including 41 from our own research team) is an indication of broad attention of health professions researchers to this widely researched instrument.

A sample psychometric study:

Hojat, M. Gonnella, J.S., Nasca, T.J., Mangione, S., Vergare, M., & Magee, M. (2002). Physician empathy: Definition, measurement, and relationship to gender and specialty. American Journal of Psychiatry, 159, 1563-1569.

More information about the JSE

With funding from the National Board of Medical Examiners’ Edward J. Stemmler, MD Medical Education Research Fund, Center re­searchers investigated the predictors and out­comes of physicians' lifelong learning based on a national survey of 3,195 Thomas Jefferson University graduates between 1975 and 2000. Major findings sum­marized, in “Measurement and correlates of physician lifelong learning,” were published Academic Medicine. A new version of the scale was developed by a group of medical education re­searchers at the Virginia Commonwealth University Medical College led by Drs. Paul Mazmanian and Angela Wetzel who worked with the Center to adapt the Jefferson Scale of Physician Lifelong Learning for administration to medical students. The study was presented at the Annual Meeting of the Association of American Medical Colleges and published in Academic Medicine. 

Two book chapters on physician lifelong learning were also written. One appeared in the Handbook of Lifelong Learning Development (Edited by Margaret Caltone, Nova Science Publishers, 2010, Chapter2, pp. 37-78), and the other was included in a monograph published in 2012 by NOVA Science Publishers entitled “Continuing Professional Development and Lifelong Learning: Issues, Impacts, and Outcomes” edited by Drs. Greg Neimeyer and Jennifer Taylor.