Feedback is a key feature of any system that promotes learning.  This is true whether we are talking about an individual student’s efforts to learn new knowledge or skills, or an organization’s efforts to improve its process or product. It is this context that evaluation was discussed as it applies to medical education.  Fundamentally, evaluation is the systematic collection of information for decision-making. It is a key component of a process of action, reflection and planning. Evaluation questions and strategies can range from a focus on learner’s experiences and abilities to larger organizational concerns characterized by questions about the curriculum, students, faculty, institutional processes or organizational mission. Regardless of the focus of a specific evaluation effort, the purpose of an evaluation is quality improvement.   

The first part of the presentation focused on student assessment and its relationship to determining competency.  Learners vary in their level of competency from novice to expert; the challenge is choosing assessment strategies appropriate for the level of competency. Assessment strategies vary in the extent to which they are objective or subjective and quantitative or qualitative, thus each requires specific implementation considerations to assure reliability, validity, efficiency and acceptability. Since each assessment strategy has strengths and weakness, a system of assessment that uses multiple strategies will provide the most accurate reflection of learner competency.  In basic science education, the multiple choice question (MCQ) is the most frequently used method of student assessment, most likely because of their objective quantitative format as well as their familiarity to both learners and faculty.  However, since patients don’t present with five choices during a medical encounter, MCQs have their limitations too.  A number of questions were provided to help educators think through decisions about which student assessment methods to choose.

The second part of the presentation focused on program evaluation.  Essentially, while the process of designing a program evaluation is similar to designing a student assessment, there are differences in terms of scale as well as the types of questions that frame the data gathering.  The program evaluation model by Kirkpatrick was used, indicating that evaluations can focus on participant reactions, learning, behavior change or real world impact. Again, evaluation strategies were discussed in terms of the various levels of Kirkpatrick’s model, with idea that each has strengths and weaknesses and that multiple measures provide the more data for decision-making purposes.  When deciding on an evaluation strategy, the question of resources, stakeholders, mission and values need to be considered.  

The discussion following the presentation focused on different methods of collecting information and their appropriateness to different needs or situations.  In addition, there was discussion of the strategies for disseminating evaluation information to faculty, as a means of involving faculty in on-going planning and decision-making.

View Slides from this Presentation