The Challenges of Resident Candidate Assessment and Selection

Presented by Mark Jordan, MD, FACS and Justin La, MD on January 7, 2021 at 12:00 pm

The original intention of the USMLE was to serve as the primary assessment tool for state medical board physician licensing. Gradually, USMLE (in particular reliance on Step 1 scores) has been adopted as the primary screening and selection tool for the transition of candidates from UME to GME by residency directors and selection committees. Consensus has been developing that the current UME-GME transition reliance on USMLE Step 1 is inherently flawed since the results of a high stakes exam designed to qualify physicians for state licensing is not relevant in either holistically evaluating residency candidates or in providing an equitable means fairly distributing residency positions. ¬†This culminated in the Invitational Conference on USMLE Scoring (InCUS) conference in March 2019, where it became clear that USMLE Step 1 was no longer serving the stakeholders in what has become a flawed UME-GME transition system. Although the was general consensus of InCUS was that changes were needed, USMLE alone would not be the only component requiring a “fix”. However, several suggestions included implementing a “pass/fail” , composite, or categorical USMLE scoring system, as well as minimizing racial demographic difference affecting USMLE performance, among others.

From the program director’s perspective, it is unclear that ¬†changes in USMLE scoring alone would provide significant additional guidance in residency selection. ¬†Equitable selection of ¬†residency applicants has become major challenge, in view of the absence of hard data to reliably predict residency performance. In addition to USMLE part 1, our traditional tools have included letters of reference, medical school grades, election to AOA, ¬†Deans letter, and clinical course evaluations. Unfortunately, notwithstanding the MSPE and Dean’s letter, ¬†critical factors such as professionalism, accountability, social responsibility, team performance, peer interactions, and clinical skills cannot adequately be assessed. ¬† For many candidates, the most important determinants of future resident performance are observed during clinical rotations in the chosen subspecialty, either at the home school or as away rotations. Unfortunately, most students are faced with the challenge of a limited number of clinical elective rotations, and hence exposure to potential residency programs. As a consequence, most residency directors overly rely on the USMLE part 1 as a surrogate of clinical performance for students that have not rotated with them. During the COVID pandemic, the absence of in person away rotations has made clinical assessment of resident candidates even more challenging for programs.

Potential solutions could include incorporating a standardized residency assessment tool (RAT) utilized by all medical schools, that may include milestone Рtype assessments that correlate to the six core competencies, as well as evaluation of skills relevant  to the type of residency applied to ( e.g. spatial coordination, technical ability for surgical specialties). Other tools such as the Jefferson Scale of Empathy and Crowd sourcing of clinical skills to assess potential candidates are being piloted in some programs.

This webinar will review the current challenges of resident candidate assessment and selection. The pandemic coming on the heels of the planned conversion of USMLE conversion to Pass/Fail has magnified the importance of developing alternative and viable candidate assessment tools and will be a major factor informing our discussion.

Seminar Downloads
Seminar Archive

Presenter Bios

Dr. Mark Jordan is an alumnus of the University of Toronto Medical School (1977) and the Gallie training program in Surgery and Urology (1983). He was elected Junior AOA in 1976. Following his residency in Toronto, he did fellowships at the Cleveland Clinic (1983-1984) and University of Minnesota (1984-1986) before returning to the University of Toronto as a Faculty member in 1987. In 1989 he was recruited by his former fellowship director Dr. Richard L. Simmons to join the faculty at the University of Pittsburgh Medical School where he served for 14 years (1989-2003), advancing to the position of Professor of Surgery with Tenure, and Director of the Urologic Renal Transplant Division at the University of Pittsburgh Medical Center and the Veterans Administration Medical Center in Pittsburgh. After 14 years at the University of Pittsburgh, Dr. Jordan was recruited to assume the position of Harris L. Willits Chair and Program Director of the Urology Division in the Department of Surgery at Rutgers-New Jersey Medical School in July 2003. At that time he was also appointed Professor of Surgery (Tenured) at New Jersey Medical School and Chief of Urology at University Hospital in Newark, NJ.

Dr. Jordan has published more than 200 journal articles and written 12 chapters in medical textbooks. Dr. Jordan‚Äôs clinical expertise includes complex renal and reno-vascular surgery, renal transplantation, urologic oncology, and robotic surgery. Dr. Jordan served as President of the Society of Urologic Chairs and Program Directors (SUCPD, currently SAU), having previously served as SUCPD Secretary-Treasurer from 2011-2012 and on the Board of Directors from 2007-2011. He also served as a Councilor for the New Jersey Chapter of the American College of Surgeons. He has previously served on the Examination Committee for the American Board of Urology, as Councilor for the Association of Academic Surgery, on the Audit Committee for the American Urologic Association, and on the Executive Committee of the Northeast Section of the American Urologic Association, and is Past President of the Urologic Society for Transplantation and Renal Surgery. In 2011 and 2012, Dr. Jordan was nominated for the Golden Apple Teaching Award by the Rutgers-New Jersey Medical School Student Council. He has given many International and National Lectures. Dr. Jordan has been named a Top Doctor by New York Magazine in 2012, Top New York Doctor for Cancer in 2013, by Castle Connolly as Top Doctor in New York in 2012 and 2013, Top Doctor in New York/Jersey in 2013, Best Doctors in U.S. News and World Report in 2012 and 2013, Best Doctors in Orange County (2017) and ‚ÄúBest Doctors in America‚ÄĚ in consecutive years from 1994-present.

In December 2013, Dr. Jordan joined the Department of Urology at University of California, Irvine as Professor and Residency Program Director and Chief of Urology at the Long Beach VA Medical System. He currently serves as the SAU representative to the AAMC, on the AUA Education Council, on the VA National Surgery Board, and on the Program Committee for the Council for Faculty and Academic Societies (CFAS).

Justin La