I have read two of the medical cases in recent issues of JIAMSE which deal with examination question “security”. Our experience at the University of Louisville, School of Medicine bears on this case.
For years our course in Medical Microbiology and Immunology released the examinations to the medical students. After many years, it became difficult to come up with new, relevant questions. Even in a dynamic field such as ours, it becomes hard to ask a question in new, valid ways. We would keep stems and change answers, or change stems for same answers. Anyway, this all became moot, when our school went to Block Testing. We run classes for five semester weeks, give the students four days free, and test them all day Friday of the sixth week on all the subjects presented in the five weeks (Streips, et al. JIAMSE, 2006, 16:10-18). The questions are scrambled and presented in 6 sections of 50 questions each. The test is carefully assembled, proofread, validated, integrated, and made as close to NBME type question sets, as possible. The test is also sequestered. Once they take it, the section is removed and the students don’t see the test again until review. The test is only revealed with answers in a review session, where they can’t take notes but can challenge questions. This allows them to discuss questions with peers and learn in that way, as well. However, with 300 questions it is impossible for them to pass on this test to next year students, aside from a question or two. We have a bank of questions which obviates this method as well. We have had no changes in success rates for questions in question analysis as long as we have done the examination process this way.
I feel we have the best of both worlds and the best answer for the cases proposed in the Journal. The students “see” the questions after taking the test, can discuss and learn, challenge answers, but cannot pass the question on to later classes. That way there is no copy of the exam floating around, which would be accessible to some people but not others. Also, students cannot study from our question set for the exams, but must study the material as they would for the USMLE Step 1.