This month, the IAMSE Publications Committee review is taken from the article titled The Predictive Power of short Answer Questions in Undergraduate Medical Education Progress Difficulty, published in Medical Science Educator (December 4, 2024). The study was conducted by Kenya Bracken and colleagues.
This research investigated whether formative short answer questions (SAQs), used as Concept Application Exercises (CAEs) at the Michael G. DeGroote School of Medicine, could serve as both effective learning tools and early indicators of academic risk. SAQs are intended to promote critical thinking and the application of knowledge in response to clinical scenarios. Although these assessments are not utilized for grading or making formal progress decisions, they were analyzed for their capacity to predict which learners may encounter challenges later in the curriculum.
The study analyzed data from over 10,000 CAEs across four student cohorts. The authors found that lower average scores on CAEs were significantly associated with future academic difficulties. Specifically, each one-point drop in average CAE score increased the odds of referral to the school’s Student Progress Committee by 37%. Interestingly, a single low-scoring CAE also flagged increased risk, further supporting the utility of SAQs as early warning signals.
The authors used logistic regression modeling and triangulated their findings by also examining performance on internal progress tests (PPI) and national licensing exams (MCCQE), both of which showed consistent patterns.
The study advances the conversation around how programmatic assessment can be leveraged not only to evaluate knowledge, but also to intervene early – especially in seeking to implement proactive, data-formed support systems for struggling students in medical curricula. SAQs, often viewed as lower-stakes formative tools, may offer valuable insights into students’ learning trajectories.
It is essential to read the full article for medical educators interested in enhancing early identification and intervention strategies. Future research could explore how these insights might shape timely academic support and improve long-term learner outcomes.
Komal Marwaha, M.D., PhD
Assistant Professor of Physiology
Texas Tech University Health Sciences Center, El Paso, Paul L. Foster School of Medicine