On Twitter a while ago, a medical student asked me how surgical program directors select new residents. Then a discussion arose among some academic surgeons on the same topic. Someone suggested that medical school grades were the best way to tell whether an applicant would be a successful resident.
The fact is that we aren’t really sure what the best way to choose residents is.
First, here’s what we really do.
A 2011 paper from the Journal of Surgical Education reported on a survey of general surgery program directors, associate program directors, and department chairs, with 262 (65%) responding.
USMLE Step 1, used by 37 percent of programs, was the most common applicant screening criterion with USMLE Step 2 second at 24% and graduation from an LCME-accredited US med school third at 15 percent. The least important criteria were previous research experience and publications.
Final selection criteria were assessed using a Likert scale. The number one factor was the interview followed by the USMLE Step 1 score, letters of recommendation, and USMLE Step 2 score. The least important factor by far was whether an applicant had done a preliminary year. Research, publications, and a previous rotation at the institution also ranked near the bottom. Class ranking, the dean’s letter, and surprisingly, Alpha Omega Alpha status were in the middle.
Responses consisted of 49 percent from university programs, 38 percent from university affiliated hospital programs, and 13 percent from independent community hospital programs. The average number of applicants per program was 571.
The problem is that proof of the value of the above methods of selection is lacking. A paper from Academic Medicine in 2011 reviewed nine studies of USMLE scores and resident performance and found no correlation between those scores and the acquisition of clinical skills by students residents or fellows.
A meta-analysis of 80 studies and over 41,000 participants from the journal Medical Education in 2013 found that the USMLE Step 1 scores and medical school grades were associated with better resident performance. However, if you eliminate studies showing that better USMLE scores led to better scores on in-training exams and passing licensing tests, only two studies found that USMLE scores correlated well with subjective ratings of residents.
What about grades?
