IAMSE 2002 Annual Meeting - Drs. Aggie Butler and David Swanson

WORKSHOP SESSION TITLE:	Writing Clinical Vignettes for Test Items

SESSION LEADER(S):	Aggie Butler, Ph.D.and David Swanson, Ph.D., National Board of Medical Examiners, Philadelphia, PA U.S.A.

OTHER PRESENTERS:

HANDOUTS or SLIDES

This session was designed to help faculty members improve the quality of multiple-choice questions written for their examinations. Through informative presentations and discussions, faculty members were able to explore structural issues relating to constructing quality multiple-choice items for testing of basic science content. An overview of item formats concentrating on the traditional one-best-answer item formats that require examinees to select the single best response was presented. One-best-answer (A-type) questions make explicit the number of options to be selected and are the most widely used multiple-choice-item format. Several examples of well-constructed A-type items were presented. Each consisted of a focused item-stem (e.g., a clinical vignette) and a lead-in question, followed by a series of options with one correct answer for each item.

Information regarding extended-matching (R-Type) items was also presented. Extended-matching items are organized into sets that use one list of options for all items in the set. Examinees are instructed to select the one-best answer. Examples of well-constructed matching sets were presented. Each R-type item includes four components:

a theme
an option list
a lead-in statement
at least two item stems

Participants were able to explore issues relating to constructing quality questions by recognizing and avoiding technical item flaws. The two types of technical item flaws described included testwiseness and irrelevant difficulty. The participants discussed flaws related to testwiseness that make it easier for some examinees to answer the question correctly, based on their test-taking skills alone. Flaws related to irrelevant difficulty that make the question difficult for reasons unrelated to the trait that is the focus of assessment were also discussed. Several examples of technical item flaws were presented and discussed during the session.

The session also provided an opportunity for the participants to discuss general guidelines and basic rules for item construction for the basic sciences. Item stems illustrating the difference between a question assessing recall of an isolated fact and a question assessing application of knowledge were presented. Several examples of items using patient and laboratory vignettes to assess application of knowledge were presented and discussed.

Each participant was provided with a printed copy of the manual “Constructing Written Test Questions for the Basic and Clinical Sciences, which is published by the National Board of Medical Examiners. The manual includes the material that was presented during the session and additional templates that can be used to assist item writers develop items for the basic and clinical sciences. Copies of this manual can be obtained at www.nbme.org.

**** **** ****

(Editor's Note: click on "Handouts" at the top of this page for direct access to this manual.)