TESTING BASIC SCIENCE KNOWLEDGE WITH ESSAY QUESTIONS: DO THEY MEASURE UP?
Klara K. Papp1, Amy Wilson-Delfosse1, Terry Wolpaw1, Dan Wolpaw1, Siu Yan Scott1, John Mieyal1, 1Case Western Reserve University, Cleveland, Ohio 44106-4929 U.S.A.
Purpose
The Western Reserve2 (WR2) Curriculum was introduced in July 2006. A multi-dimensional approach to student assessment was created to measure early mastery of core medical competencies based on conceptual reasoning in a learner-centered, self-directed and reflective educational environment. This study asks: what evidence is there to justify using primarily essay-type questions to assess student knowledge of basic science curricula in medical education?
Methods
Students in the Class of 2010 (n=138) participated in this study. At the end of each 12-week multidisciplinary integrated section, they complete a 5-item essay question examination each with 3-5 subparts in a test secure environment. These were graded and used to identify whether or not students met expectations. Multiple choice questions assessing the same content were also administered, though not factored into grading decisions.
Results
For each section, on average, 7 faculty spent about 70-72 hours per section grading these examinations (ranging from 6-12 hours per faculty). Students achieved exceedingly well on the assessments; Section 1 [mean(stderror)] = 91.4(0.4); Section 2=80.3(0.8); Section 3=87.2(0.7). Correlations between essay questions scores and the corresponding multiple choice questions were moderate ranging from 0.30-0.54; p<0.01.
Conclusions
Essay questions have changed the culture of teaching and learning in WR2. There is greater emphasis on concept learning, organization of ideas, clear and articulate use of language. Students are learning in an environment where emphasis is on self-awareness and self-reflection. Instructional and assessment methods are designed to nurture active interchange of ideas between students and faculty rather than recall and factual knowledge.