The following abstracts have been accepted for presentation at the this meeting in Poster format in the category of Assessment.  Those selected for Oral Poster Presentations are so designated in the far right column.
 
 

A1	EVALUATION OF STUDENTS ENROLLED IN PROBLEM-BASED LEARNING COURSES IN A VETERINARY CURRICULUM Larry G. Adams*, DVM, PhD, Robert L. Bill, DVM, PhD, S. Kathleen Salisbury, DVM, MS, Laurie D. Adams, DVM, Department of Veterinary Clinical Sciences, Purdue University School of Veterinary Medicine, West Lafayette, IN 47907-1248 U.S.A. Evaluation of students enrolled in problem-based learning (PBL) courses is an important part of the success or failure of PBL courses. The Doctor of Veterinary Medicine (DVM) curriculum at Purdue University School of Veterinary Medicine includes a series of four 3-credit PBL courses offered in the first 2 years of the 4-year DVM program. In these courses, we wanted to evaluate multiple skills including understanding of content, group process skills in the tutorial sessions, problem-solving skills, learning/teaching abilities, information management skills, and interpersonal and communication skills. The evaluation tools were designed to evaluate these different skills by utilizing a combination of subjective evaluations of PBL group skills, written examinations, and oral examinations. Evaluation of student and group performance in tutorial sessions is done through a combination of subjective evaluations by faculty tutors, student peers and self-evaluations. We utilize a combination of formative and summative feedback evaluations done three times per semester. Formative evaluations are done to provide constructive feedback to the students about skills they are doing well and any suggestions for improvement that would help the group learn better. The formative evaluations are not used to determine grades. The summative evaluations provide feedback to the students on specific areas including student participation in the tutorial sessions, problem-solving skills, learning/teaching abilities, and interpersonal and communication skills. The summative evaluations include numerical scores that are used to determine a percentage of the course grades. The formative evaluations and summative evaluations are completed by the student (self-evaluation), group members (peer evaluation), and the faculty tutor. Self and peer evaluations each constitute 5% of the course grade while faculty evaluations constitute 10 to 24% of the grade. Written examinations are given to evaluate student understanding of specific content related to each case studied during the semester. In addition, a portion of the written exams are derived from the content learned in prior semesters of the PBL courses because learning within a professional curriculum is geared towards career application of the material, rather than passing a given course or semester. Written examinations constitute 40 to 55% of the course grade. At the end of each semester, individual oral examinations are given. The students are given 30 minutes to read the case and make some notes followed by a 30-minute oral examination during which additional case information is provided. The oral examination allows for assessment of problem-solving skills, communication skills, and knowledge base. The students individually problem-solve through an unknown case and are questioned by two examiners using a standard set of questions. The oral examination contributes 20 to 30% of the overall grade. Using a variety of evaluation tools allows assessment of many different skills.	ORAL
A2	AN ONLINE, WEB -BASED TRIPLE JUMP EXAMINATION TO ASSESS COMPETENCIES IN A COMPETENCY CURRICULUM W. Marshall Anderson* and Patrick W. Bankston, Indiana University School of Medicine, Northwest Center for Medical Education, Gary, IN 46408 U.S.A. Objective:  In 1999 Indiana University School of Medicine implemented a Competency Curriculum involving nine separate competencies. A statewide triple jump examination was administered to the M1 class at the end of their academic year to assess several competencies. The Northwest Center for Medical Education (NWCE) has been using triple jump exams since 1990 as part of their Problem-Based Learning (PBL) Curriculum. The objective of this project was to create and administer an online, web-based triple jump exam for M1 students in the first semester to demonstrate its applicability to the entire statewide system. This exam was also dministered online in 2000.  Methods:  A PBL case was converted to hypertext markup language. CT scans and histological slides were digitized and included on certain pages. Submit forms for each page were produced in Javascript and Perlscript. A computer lab was reserved for two days to administer the test.  Results:  M1 students at NWCME completed the exam on networked computers over the exam period. Pages were printed by the students for their personal copy and submitted via e-mail to a course instructor. Submissions were identified only by student ID number, day and page number. Individual pages were printed, collated, and graded by a committee of faculty. Three competencies assessed: life-long learning, problem solving, and communication.  Conclusions:  In 1999, only one student's submission was lost in transit, but was easily recovered. In 2000 no submission were lost in transit. Answers were much more legible and concise than previous handwritten triple jump exams. This method of triple jump examination is being offered statewide in 2001. Five of the nine statewide medical education centers have signed-up to administer this exam online.	ORAL
A3	ASSESSMENT OF PHARMACOLOGY INFORMATION WITHIN A MULTIDISCIPLINARY SOPHOMORE MEDICAL CURRICULUM Carl L. Faingold, Ann C. Kelson and George A. Dunaway*, Departments of Pharmacology & Medical Education, Southern Illinois University School of Medicine, Springfield, IL 62794 U.S.A. The ability to assess student acquisition of important information that is specific to basic science disciplines can be problematic in recently developed multidisciplinary integrated medical curricula. At SIUSM a new assessment vehicle, which simulates the context in which the knowledge will eventually be used in clinical practice, is designed around patient cases that are chosen by representatives of the major second year disciplines. The approach taken by the faculty at SIUSM involves a three-day assessment presented three times per academic year. Each assessment contains four standardized patients (SPs) and 6-8 Objective Structured Clinical Examinations (OSCEs). After each SP and OSCE patient encounter the student completes an objective assessment that involves questions on both clinical and basic science information. The computer-based questions relate directly to the patient material in the SPs and OSCEs with any topic in the differential diagnosis for each case forming the potential basis for the questions from each discipline. Pharmacology questions arising from each case allow the important issues of drug-related information, which were identified by the faculty content experts within the Dept. of Pharmacology, to be tested under the broad rubric of the Pharmacology Mental Algorithm. This algorithmic approach is presented to the students at the beginning of the year and reinforced at the beginning of each organ system as a standard format for dealing with any information relating to drug use is as follows:  PHARMACOLOGY MENTAL ALGORITHM (A-J)  A. Drugs clinically available that have the desired pharmacological actions required to treat the pathophysiology of the medical problem;  B. Mechanism(s) of drug action;  C. Factors which influence the pharmacokinetics of each drug;  D. Potential adaptation processes altering target cell responsiveness (e.g. tolerance, desensitization, dependence); E. Clinical usefulness of the drug(s) to be used relative to the therapeutic goals established for the patient;  F. Contraindications (absolute and relative),  G. Adverse (side) effects;  H. Drug-induced effects, including those related to intentional drug abuse, that can be mistaken for disease signs and symptoms; I. Alterations of clinical laboratory tests related to drug administration;  J. Risk/Benefit ratio of therapy (therapeutic index). These ten topics form the basis for questions about any pharmacological agent that might be used for any condition in the differential or for any drugs that might have been taken for any condition in each patient's medical history. This expands the scope of exam content beyond that which traditionally forms the basis for case-based assessments. Experience with this approach, thus far, indicates that important discipline-specific concepts in Pharmacology can be effectively tested using this format.	ORAL
A4	A 360-DEGREE FEEDBACK APPROACH TO DEVELOP AND ASSESS FIRST YEAR MEDICAL STUDENTS' CLINICAL SKILLS Edward Gurza, M.D.*, and Pamela Derstine, Ph.D. Loyola University Stritch School of Medicine, Maywood, IL 60153 U.S.A. Feedback is an essential component of the teaching-learning cycle. It clarifies performance criteria, helps students learn to evaluate their own level of performance, and focuses students' efforts to improve their skills. In 1995, we adopted a 360-degree feedback approach for first and second year medical students that facilitates the development of the knowledge, skills, and attitudes necessary to attain proficiency in core clinical skills.The core component of this curriculum is the small group sessions that include periodic, focussed, formative encounters with standardized patients (SPs). Students receive structured feedback from the SPs, from a peer-partner, from a faculty facilitator, as well as from self-reflection, prompted from audio- or videotapes of each encounter. As first year students develop specific skills and gain experience, they practice more advanced skills, such as taking sexual, drug, alcohol, tobacco, and physical abuse histories as they encounter these topics in lecture or their readings. Second year students learn physical examination skills, integrating the history and physical with an emphasis on sound clinical reasoning. Summative SP encounters take place at the end of each of the four semesters and contributes significantly toward the final course grade. Faculty facilitators regularly evaluate student non-cognitive competencies (participation, preparation and skill development) in non-SP small group sessions using a rating form based on a clear, unambiguous rubric. Students reflect, in written form, on their strengths and weaknesses and develop strategies to improve. Each student meets with their faculty facilitator twice each semester to discuss issues raised by these assessment tools. Although facilitator evaluation of student performance in small groups contributes significantly toward the final course grade, the ongoing assessments are formative. No grade is given until the end of the course since competence in complex skills takes time to develop. Faculty facilitators become part of this learning community by being evaluated each week by their students. These forms are collected by the course director, who gives feedback to individual faculty on their teaching skills. Thus, the feedback process used to develop student skills, knowledge and attitudes is intimately tied to the course structure.
A5	A PROGRESSIVE CREDIT ASSESSMENT TECHNIQUE (PCAT) FOR IMPROVING THE PRECLINICAL EXPERIENCE Hartwig, Walter C.*, Touro University College of Osteopathic Medicine, Vallejo, CA,  94558  U.S.A. Can a different assessment technique be valid and at the same time improve student attitude and comprehension? The Progressive Credit Assessment Technique (PCAT) is proposed as a system that neutralizes many student anxieties while compiling a more individualized assessment of strengths and weaknesses than is achieved by traditional methods. The PCAT proposed here combines computer-based testing software with a customized accounting of student progress in separate subjects (anatomy, physiology, etc) using a traditional multiple-choice format. One advantage of PCAT is that it requires students to correctly address ALL items in each subject test bank, therefore eliminating the shifting standard of "passing competence". In return, students pace their own testing sessions, which may relieve some aspect of testing anxiety. PCAT software debriefs the student after each testing session and automatically recycles questions that were addressed incorrectly. The system is not perfect, but its design offers a positive shift in student experience without compromising the rigor of traditional assessment techniques.
A6	A MODIFIED TRIPLE JUMP EXAMINATION FOR USE IN AN ESSENTIALS OF CLINICAL MEDICINE COURSE Debra L. Klamen, MD, MHPE*, Department of Psychiatry, University of Illinois at Chicago, Chicago, Illinois 60612 U.S.A. Background: Triple jump examinations have been reported in the education literature since at least 1993 (Smith, R. 1993). Although the use of this format, and its apparent ability to assess clinical problem solving, is intriguing, the sheer volume of faculty time involved is a deterrent to its more wide-spread use, especially with large class sizes. The University of Illinois College of Medicine at Chicago enrolls approximately 190 medical students in its M2 class on a yearly basis. The modified triple jump exam described below is an attempt to maximize the triple jump’s assets, while minimizing faculty time and effort in the proceedings. Design of the exam: Each student in the M2 class, upon completion of the Essentials of Medicine course, would be scheduled to complete a focused history and physical examination on a standardized patient. Since the patient case was written by faculty involved in the course, all aspects of the case (history and physical findings) are known. Students will answer questions immediately upon exiting the interview, such as "What is in your differential diagnoses at this stage? Please list the top 4 diagnoses in order of their probability, with the diagnosis you consider most probable listed first." Students will not be allowed access to any material other than notes taken during the focused examination. Students will then be given a sheet of questions including: "Explain your rationale for choosing the two diagnoses you did, and explain why you put them in the order that you did. What did you learn in your research that was the most influential in changing or retaining your top two diagnoses?". Students will be responsible for turning in answers to the sheet of questions four days later, after they have researched their patient’s problems and findings in the medical literature. A key will be developed, and faculty will grade both the initial questions, and those turned in 4 days later. Inter-rater reliability will be achieved closely monitored. We hope to administer this exam for the first time to the M2 students in May, 2001. Our findings will be compared to the previous year’s examination results, which were administered in an open book essay format. Using this technique, we hope to better understand what the M2 students know "cold" and how well they use the medical literature to inform themselves about a clinical problem.
A7	PERFORMANCE-BASED ASSESSMENT FOR SECOND-YEAR MEDICAL STUDENTS: INTEGRATION OF NEUROLOGICAL EXAM SKILLS AND NEUROSCIENCE CONTENT Kathryn Lovell*, Jane Turner, Brian Mavis, Daniel Murman. Michigan State University, East Lansing, MI 48244 U.S.A. Michigan State University College of Human Medicine instituted a mini-OSCE four years ago to assess the performance of second-year medical students on curricular elements related to the neurological exam taught in Clinical Skills and neuroscience basic science content taught in a 4-week intensive Neuromusculoskeletal course. The performance-based assessment (PBA) was conducted after students completed exams over both the clinical skills and the basic science content. For the past 2 years, faculty members have served as standardized patients; in prior years classmates served as SPs. At the beginning of the PBA the SPs are trained to simulate the clinical expression of a specific case. Each student 'doctor' receives the chief complaint and vital signs of the SP, performs a modified neurological exam on the SP, completes portions of a written record with instructions to localize the lesion and to list and justify the most likely diagnosis, and then answers computer-based questions about the case, including interpretation of a radiology image. The answers given at the last station comprise a small part of the final exam score for the basic science course. At the last station, each student is given feedback about the completeness of the neurological exam that he/she performed. The written record and neurological exam performance for each student is categorized as 'above average', 'satisfactory', or 'needs improvement', and the feedback is returned to the student. Students have provided highly positive feedback about the value of the experience as a simulated patient encounter. By serving as SPs and by reading the written products, faculty have gained a much richer understanding about the abilities of students. PBA results in past years have resulted in curricular changes to enhance the understanding of second-year students. These changes have included the production of a digital video screening neurological exam demonstration with interactive questions related to localization of function being tested, and revisions of the procedures used in teaching the exam skills.	ORAL
A8	A SIMPLE, COST-EFFECTIVE APPROACH FOR PRESENTING ONLINE, INTERACTIVE EXAMS Julia Perhac, Jack Madderra, and Terrence W. Miller*, Ph.D. Touro University College of Osteopathic Medicine, Vallejo, CA 94592 U.S.A. Our objective was to create internet-based, interactive histology laboratory quizzes that could be easily and inexpensively administered. The design that was developed could serve as a template for online exams in most didactic- and laboratory-based classes. Each quiz is an html document prepared as a "form" that includes jpeg-compressed light micrographs and several multi-choice questions. A student fills in his or her name, chooses the answers, then hits a "submit" button. This transfers the results to a text file residing on the internet service provider (ISP) server, and the text file is subsequently emailed to the instructor. Text files are comma-separated values that can be imported into an Excel spreadsheet and easily graded. After hitting the "submit" button, the student is presented with a confirmation page that serves as a proof of completion the quiz. Student comments regarding the quizzes has been largely positive. Students like the fact that quizzes can be taken from home or campus at times convenient for them. They also feel that the quizzes are a "low-pressure" way of forcing them to keep pace with the material. Eliminating in-class quizzes provides students with more time to complete laboratory assignments while lab instructors are present. This approach has two major shortcomings as currently implemented: a) the design does not provide safeguards against cheating, and b) the student does not receive an immediate graded response. These problems can be addressed relatively easily and approaches are discussed. Implementation of the quiz requires that either the server is enabled for common gateway interface (CGI) or the ISP provides CGI support. This format can be used with any number of questions as well as other question formats (e.g. fill-ins). It is therefore applicable for a variety of subjects. This is a common, simple, no-cost service that can conceivably be maintained by the instructor. It is therefore ideal for schools with a limited budget and a small management information services department.
A9	LONGITUDINAL ASSESSMENT OF STUDENT/TRAINEE KNOWLEDGE AND SKILLS AT OHIO UNIVERSITY COLLEGE OF OSTEOPATHIC MEDICINE AND CENTERS FOR OSTEOPATHIC RESEARCH AND EDUCATION Malcolm Modrzakowski*, Ph.D., Ronald Portanova, Ph.D., Joseph Jollick, Ph.D., and Michael Adelman, D.O., Ohio University College of Osteopathic Medicine, Athens, OH 45735 U.S.A. The field of medicine is evolving and we as medical educators must also change to meet the demands of the profession and of the medical student. Ohio University College of Osteopathic Medicine (OUCOM) recognizes this need and has created programs to initiate medical education reform. Our goal for curriculum reform has been to develop a continuum of physician training. This training is a seamless, osteopathic medical curriculum, integrated both vertically (across time) and horizontally (across disciplines) which encompasses the period from matriculation to the completion of primary care residency. To accomplish the task of assessing the accumulated basic science, clinical and interpersonal skills of trainees over the seven year span of medical education, we are exploring comprehensive, integrated techniques which include Progress Testing and Objective Structured Clinical Examination (OSCE). Progress testing is a method of assessing the time dependent acquisition and retention of knowledge based on student performance over successive tests and not on the result of a single examination. The OSCE, which is a performance based assessment where the learner is tested on what he or she can "do" rather than just what he or she "knows", allows for assessing skills learned over a specific time period, such as the completion of specialized clinical rotations and/or the skills, acquired throughout the continuum of the student's medical education. These two assessment techniques can help medical educators guide student learning throughout the curriculum. A status report of our experiences to date will be presented in a poster with pictures and text.
A10	ASSESSING CLINICAL NEUROLOGICAL EXAM SKILLS IN FIRST YEAR MEDICAL STUDENTS: A NEW CARD GAME Sandra L. Shea, Ph.D.*, Richard W. Clough, Ph.D., William R. Hamilton, M.D., School of Medicine, Southern Illinois University, Carbondale, IL 62901 U.S.A. In the fall of 2000, Southern Illinois University School of Medicine (SIUSOM) implemented a new curriculum with the goal of merging the best points of the previous Standard and Problem-Based Learning curricula. This necessitated a number of changes in teaching and evaluation techniques which reflect, among other things, the fact that the entire class goes through all stages of training and exams simultaneously. An integral goal of the new curriculum is to increase the amount of clinical training students receive in the first year of medical school by utilizing Standardized Patients (SPs), mentoring experiences and formal clinical training sessions with guest and resident clinical faculty. Following our 10 week Sensorimotor Systems and Behavior unit, we assessed each student’s performance on our basic neurological exam (BNE). Rather than simply ask students questions about this exam in a written format, we required them to demonstrate their proficiency with the exam. However, we had neither the time nor the clinical faculty to have every student conduct the entire exam. The compromise we struck was predicated on basic sampling technique. Students had to be prepared to do all of the BNE because they did not know which questions would be asked of them. They entered a physical exam room with an SP, a physician and 5 stacks of colored cards. All rooms had the same 5 stacks of cards. Each colored stack contained tasks specific to one BNE area: cranial nerve exam; mental status exam; spinal segments, motor and proprioception; sensory dermatomes; cerebellar functions. There were 30 cards (samples will be at the poster). Students drew one card from each colored stack. They were allowed to "throw out" any ONE card if they couldn’t do or didn’t understand that particular skill, but they had to replace it with another card from the same colored stack. They had 5 minutes to do the 5 tasks described on the cards. They then went to another physical exam room with a different SP and a different physician, drew 5 more cards (eliminating any they had already done) and performed 5 more BNE skills. Having two physicians evaluate each student added an element of quality control. The physicians were in accord on all of the BNE testing standards. Students were given their grade on completion of the task. No student ran out of time. Any student who performed fewer than 7 of 10 tasks correctly was assigned to remediation the next day and had to re-do any tasks they had not performed correctly. This test format required the students to be ready to perform any portion of the BNE while allowing us to sample only part of it. This format also gave students experience with 2 SPs and feedback from 2 different physicians. Finally, this format had the benefit of providing the physicians a different set of tasks for every student who entered the room. With 4 physicians, each seeing 18 students, these variations helped maintain focus. Interestingly, no student required remediation with this method and only 1 student had a grade lower than 80%. Content knowledge, neurological basis of the BNE and other aspects of the unit were evaluated by other means (discussion at poster).	ORAL
A11	A FORM FOR ASSESSMENT OF STUDENTS IN PROBLEM-BASED LEARNING SESSIONS Carol F. Whitfield, Ph.D.*, Pennsylvania State College of Medicine, Hershey, PA  17033  U.S.A. Assessment of students in problem-based learning (PBL) sessions is always difficult and mostly subjective. Its subjectivity causes students great concern for the variability in facilitators' interpretation of what behaviors should be exhibited by average, better-than-average or excellent students. There is considerable variability in the facilitators’ interpretation of these behaviors despite the written descriptions of these classifications given to them before the sessions. Ten percent of our facilitators assign grades that are more than 1 standard deviation above and below the mean facilitator grade, and some facilitators' mean grades for a group have a 0 standard deviation. Currently, a portion (5 to 18%) of each course grade is determined by PBL grades assigned by facilitators. Students and faculty at our institution agree that if grades are not given for PBL or a pass/fail grading system is used, students will not work as hard or accomplish as much in the sessions. Therefore, I developed a grading method that eliminates some subjectivity and is more acceptable to students and faculty. The method uses seven categories of assessment: participation in hypothesis generation, participation in discussion of learning objectives, clarity and depth of explanations, progress in group participation, professional behavior, use of resources, and group skills. Four to six statements describing characteristic behaviors or levels of performance are listed for each category. One of these statements describes the ideal student while the others may describe too little, too much or other less desirable levels of performance. Each statement is given a point value not seen by the facilitator. The facilitator chooses the one statement in each category that best describes the student. The form containing these checked statements is scanned into a computer that assigns the grade as a percentage of the maximum points that the ideal student would be awarded. Students have given strong support to this method of assessment, and the curriculum committee has enthusiastically accepted it. The pilot test for this newly developed form will occur in parallel with the old assessment form in the Spring 2001 semester. It remains to be seen if significant changes in the PBL portion of the course grade changes significantly upon use of the new form.