Assessment
104 – One step ForWard and NO steps back!
NBME subject exam scores and transition to an integrated clinical curriculum
Kirstin Nackers, Raquel Tatar, Eileen Cowan,
Laura Zakowski, Katharina Stewart, Sarah Ahrens, David TIllman, Laura Jacques,
and Shobhina Chheda
UW-Madison SMPH
PURPOSE We used NBME subject exam scores and pass rates to study how moving from a traditional clerkship model to integrated clinical blocks affected student acquisition of medical knowledge. Recent AAMC data shows that two-thirds of medical schools are currently undergoing/planning substantial curriculum changes. (1) During periods of curricular change, it is desirable to track assessment outcomes to inform leadership of possible negative impacts to students during the transition.
METHODS The Legacy Curriculum at University of Wisconsin, School of Medicine and Public Health followed the standard medical school educational model, including traditional, department-based, clinical clerkships. The ForWard Curriculum began in 2016 and these students entered new integrated clinical experiences in January 2018. Our Neurology and Psychiatry clinical experiences are integrated with Internal Medicine in Acute Care. Family Medicine and Ambulatory Internal Medicine are integrated in Chronic and Preventative Care. OBGYN and Pediatric experiences are integrated in Care Across the Life Cycle. Finally, Surgery, Anesthesia, and other procedural specialties are integrated in Surgical and Procedural Care. We compared NBME scores and passing rates between the final cohort of Legacy M3 students and first cohort of ForWard students for Adult Ambulatory Medicine, Medicine, Neurology, OBGYN, Pediatrics, Psychiatry, and Surgery. Institutional passing thresholds were not changed.
RESULTS NBME scores and passing rates to date for the ForWard Curriculum cohort demonstrate no statistically significant differences from those of the Legacy cohort (p<0.05). Further monitoring of these data is necessary, however these results indicate no substantive negative effects to student acquisition of medical knowledge during the transition period to the integrated curriculum.
CONCLUSIONS Limitations
include relatively small sample size due to examining only two cohorts of
students. Strengths include examining the same standardized assessment
measures. References âCurriculum Inventory and Reports (CIR) – Initiatives –
AAMC.â Association of American Medical Colleges, AAMC,
aamc.org/initiatives/cir/427196/27.html. Accessed 2018 October 15.
Assessment
106 – DO MCQ QUIZ SCORES PREDICT STEP 1 SCORES TO THE SAME DEGREE IF THE QUIZZES
ARE NO LONGER PART OF THE COURSE GRADE?
Kathryn Moore, Karly Pippitt, Candace Chow, and
Jorie Colbert-Getz
University of Utah School of Medicine
PURPOSE Scores from multiple-choice question (MCQ) assessments share a moderate-to-strong relationship with Step 1 scores. It is unknown if this relationship varies by the stakes of the assessment (i.e., whether the assessment is graded). If scores from graded assessments predict Step 1 scores, but scores from identical but ungraded assessments do not predict Step 1 scores, this may suggest students are not motivated to perform their best for ungraded/formative assessments. If, however, there is no difference in predictive ability, this would provide evidence that assessments offer valuable learning experiences even if ungraded and formative in nature.
METHODS Participants were University of Utah SOM students matriculating in 2014 (N=100) and 2016 (N=131). Students in the graded cohort completed 17 MCQ quizzes across 6 courses in years 1-2 while students in the ungraded cohort completed 19 MCQ quizzes. Both cohorts completed Step 1 at the end of year 2. Quiz performance contributed to 36%-50% of course grades for 2014 matriculates (graded cohort). The testing environment and majority of quiz questions were the same for the 2016 matriculates, but performance did not contribute to course grades (non-graded cohort). Correlations between quiz and Step 1 scores were computed; linear regression was used to determine if quiz scores significantly predicted Step 1 scores.
RESULTS Average quiz scores were 81%(SD=6%) for the grade cohort and 80%(SD=6%) for the non-grade cohort. There were strong correlations between quiz and Step 1 scores for the grade cohort (R=0.79) and non-grade cohort (R=0.63). Quiz scores significantly predicted Step 1 scores for the grade cohort, F(1,98)=165.81, p<0.001, and non-grade cohort, F(1,129)=84.59, p<0.001.
CONCLUSION Regardless if used
for grading or formative purposes, quiz performance shared a strong relationship
and predictive ability with Step 1 performance.
This may suggest that assessments need not count towards course grades
in order to motivate students.
Assessment
108 – A BENCHMARK-ANCHORED PATH FOR SUCCESS ON STEP 1 AND LEVEL 1 LICENSURE
EXAMS
Maria Cole, Kerin Fresa, Marcus Bell, Dawn
Shell, and Linda Adkison
University of Missouri School of Medicine,
Philadelphia College of Osteopathic Medicine, and Trinity Medical Sciences
University School of Medicine
PURPOSE Residency program directors increasingly use a USMLE Step 1 or COMLEX Level 1 score as an important screening point for applicants. Low scores and repeated failures can result in poor outcomes for students. Thus, the development of benchmarks supported by data that identify student progress and provide evidence of âpreparednessâ lead to better prepared students with better individual and institutional outcomes. This study is a collaboration between three different medical schools to compare data and develop shared best practices.
METHODS Data were collected for analyses from a US 6-year allopathic medical school, an osteopathic medical school, and a Caribbean allopathic medical school. All schools collect data for students taking USMLE Step 1 and one school collects data for students taking COMLEX Level 1. These data include: course performance, NBME discipline exam results, commercial 8-hour simulation exam results, an internal comprehensive exam, and the NBME Comprehensive Basic Science Exam results. Data for multiple cohorts are combined and association studies were performed to determine Pearsonâs coefficient and significance.
RESULTS Outcomes demonstrate the following for COMLEX Level 1 exams: Specific course performance can be associated with Level 1 scores COMSAE D, an internal comprehensive exam, and an 8-hour COMBANK exam are correlated with Level 1 scores. Outcomes demonstrate the following for USMLE Step 1 exams: NBME basic science exams and a commercial 8-hour exam are correlated with Step 1 scores The NBME Comprehensive Basic Science Exam is correlated with Step 1 scores. These results led to institutional-determined benchmarks for each student and students to achieve prior to registering for USMLE Step 1 or COMLEX Level 1. The benchmarks led to improved studentsâ performances for first time pass rate and mean score.
CONCLUSION These studies
demonstrate the power of shared information and collaboration between schools
and support schoolsâ efforts for studentsâ success.
Assessment
110 – Leveraging Student Perception of Assessment Performance to Support
Learning
Brock Mutcheson, Andrew Binks, Renee Leclair,
and Emily Holt
VTCSOM
PURPOSE: The overall goal of the study was to determine and refine the efficacy of student-support services intended to improve student performance. For this particular analysis, the research team investigated and described the extent to which student perceptions of their own assessment performance were linked to data aberrations detected using traditional data forensics.
METHODS: The study so far has included 102 first year medical student observations and seven assessments; three formative and four summative exams. In total, there have been 1090 item-level observations. During exam review sessions, participating first year medical students indicated the type of error they believed they had made on a questionnaire developed through a literature review conducted by the research team. The types of errors were categorized into two major groups; Type A, a test-taking error (e.g. poor question interpretation, misreading a question) or Type B, a lack of content mastery (i.e. they didnât know the answer). Student perceptions were investigated by student demographic and academic performance characteristics and various item-level characteristics. For this analysis, the team focused on data aberrations detected on individual items an examinee was anticipated to answer correctly based on response patterns, but that the examinees actually answered incorrectly (Meijer 1994). The estimated error rates were then associated with the preliminary summary findings.
RESULTS: Several data aberrations were identified and the error rate was found to be significantly correlated with multiple type-A error categories (i.e. poor test-taking) identified by students.
CONCLUSIONS: This analysis
provided validity evidence for one intended use of the questionnaire and
promising evidence for studentsâ abilities to recognize and explain assessment
errors. Moreover, this analysis demonstrated the value of including an
alternative post-hoc measure to the assessment feedback process with the
ultimate goal of improving learning support.
Assessment
112 – Retention of Medical Knowledge across the First Year of Medical School
Carrie Bailes, Mary Caldwell, Renee Chosed, Anna
Blenda, and Matthew Tucker
University of South Carolina School of Medicine
Greenville
PURPOSE Medical students are tasked with absorbing a vast amount of medical knowledge. Because of this, it is important to assess how much of that knowledge they retain as well as the depth of memory for that information. Here we retested students on a subset of questions from their Molecular and Cellular Foundations of Medicine summative exam 10.5 months after they first sat for the exam. In addition to re-assessing student performance using the same multiple choice format (cued recall), we also tested how well students could answer the questions without seeing the answer choices (free recall).
METHODS Second year medical students (N=46, 25 Females) reported to the same location of the original exam and used the same testing software. Fifty of 104 questions were selected from the original exam based on whether the item could be answered from memory without seeing the answer choices. The question stem was displayed with a text box below it to allow for the free recall of an answer. After an answer was entered, the original answer choices were shown, and students selected the best answer.
RESULTS The studentsâ average on the original exam was 87.2±5.5%. 10.5 months later, students answered correctly 53.9±9.6% of the items, which was 62% of their original scores. Free recall rates (i.e., recollection without seeing the answer choices) were considerably lower, with students correctly answered 15.8±9.2% of the questions. Initial exam performance did not correlate with retest scores.
CONCLUSIONS Understanding how
much information is retained in medical school is important for evaluating
teaching effectiveness and the difficulty of course content. Understanding
differences in free vs. cued recall may provide insights about how deeply
information is memorized. Testing protocols can be developed to compare
retention rates between modules and across different time spans.
Assessment
114 – CAUSES OF VARIATION IN THE PREDICTIVE VALIDITY OF FORMATIVE ASSESSMENTS IN
AN ORGAN-SYSTEM BASED PRECLINIC CURRICULUM
Jason Booza, Paul Walker, and Matt Jackson
Wayne State University School of Medicine
PURPOSE In 2018, the Wayne State University School of Medicine launched a new preclinical organ-system curriculum with faculty-authored weekly formative assessments coupled to end-of-unit summative exams prepared through the National Board of Medical Examinerâs customized assessment system. Preliminary evaluation indicates that while the formative assessments were moderately-to-highly predictive of summative performance, there was variation in predictive validity of low and high performers. Formative assessment tended to overpredict for low summative performers and underpredict for high summative performers. While medical education literature is replete with assessment best practices, there is a paucity of studies addressing the variation of learnerâs performance on formatives assessments. Our purpose is to understand causes of variation affecting student learning and assessment, as well as to address this gap in the literature.
METHODS A mixed-method approach analyzed formative assessment practices routinely captured through ExamSoft. Data was collected from weekly formative assessments completed on or off campus during an open period from Friday afternoon to Sunday evening. Randomization was achieved with a stratified random sample survey of medical students across 3 levels of summative performance.
RESULTS We found high summative performing students took the formative assessments earlier and spent less time preparing. They were also more likely to use the post-exam review and performance summary features of the formative assessments in preparation for the summative exams. These students viewed the system as being a valuable part of their learning strategy. Low and mid summative performers completed formative assessments late and many reported that they did not utilize the post-exam review and performance summary features.
CONCLUSION The predictive
validity of formative assessments appear be affected by the strategy of
preparation, use of the performance summary features, and self-fulfilling views
of the formative assessment process.
These findings may shape targeted interventions to improve student
learning strategies.
Curriculum
204 – INTEGRATING SCIENTIFIC RESEARCH INTO MEDICAL EDUCATION THROUGH A
COMPREHENSIVE ELECTIVE COURSE
Marek Svoboda, Yasmin Kamal, and Roshini
Pinto-Powell
Geisel School of Medicine at Dartmouth
PURPOSE One of the goals of evidence-based medical education is to familiarize future physicians with the scientific method in order for them to comprehend scholarly literature and communicate it to their patients. However, many medical students lack the skills necessary to participate in research themselves. Here we describe an implementation of a comprehensive elective course which is designed to equip interested students with these skills through a series of workshops, one-on-one mentorship, and pursuit of a summer research experience.
METHODS The elective course has been implemented for one year. Participating students are selected among first-year medical students based on their motivation to conduct basic, translational, or clinical research. No prior experience is required. Throughout the year-long course, students attend a series of 10 one-hour workshops designed to teach them the skills necessary to engage in research. Topics covered by invited faculty experts include resources management, clinical trial design, scientific writing, and funding opportunities available to medical students. Additionally, each student is paired with a peer mentor to help them navigate the research community. At the conclusion of the course, students complete and present a Specific Aims page based on their proposed study.
RESULTS Nine students participated in the first year of the elective (2017-2018). All students secured a research position for the summer following their first year of medical school and four students also obtained nationally competitive funding. 78% of the students indicated that participation in the elective significantly helped them find and secure these positions and 88% learned valuable research skills not taught in their medical curriculum.
CONCLUSIONS M1 participation
in a well designed research elective has the potential to provide immediate
value in the acquisition of a summer research position and to stimulate a
lifelong interest in scientific inquiry.
Curriculum
218 – USING A CULINARY HEALTH EXPERIENCE TO TEACH TEAMWORK SKILLS: A NEW
INTERPROFESSIONAL EDUCATION EXPERIENCE
David B. Trinkle, David W. Musick, Renee
LeClair, Nathaniel L. Bishop, Cynda A. Johnson, and Joalenn Tabor
Virginia Tech Carilion School of Medicine and Jefferson
College of Health Sciences
PURPOSE To design a new interprofessional culinary health curriculum for first year medical, nursing and physician assistant students (total=135).
METHODS As medical schools focus more on healthy communities, there has been a corresponding expansion of teaching on nutrition and healthy cooking concepts. Students in the early years of training must learn about clinical nutrition and the dietary needs of special populations. Other relevant concepts that play a role in culinary health include population health, clinical dietetics, health disparities, quality and safety, and service learning. Interprofessional team activity in a large commercial training kitchen also offers many analogies to fast paced hospital care teams. Working with two partner institutions, we launched a culinary health program that involved three components: expanding studentsâ nutrition knowledge base, team-based meal preparation and service, and community outreach. Interdisciplinary student teams worked together during two lab sessions in a commercial kitchen wherein they cooked and served five different types of meals based on special patient populations. They also completed a nutrition oriented community service project involving inner city youth, performed a nutrition âself-analysisâ exercise and worked through clinical nutrition cases.
RESULTS Students provided feedback on standardized course evaluation forms, rating the culinary health experience at 3.75 on a 5-point scale. Excellent suggestions for course enhancement were received. There was a range of student âbuy inâ regarding the analogy of a commercial kitchen being similar to a busy clinical environment.
CONCLUSION The successful
implementation of a culinary health program involving students from three
disciplines requires abundant logistical support. Similar programs, during either preclinical
or clinical years, may be scalable and easily accomplished. Strengths included a large student base from
three disciplines and the mandatory curricular requirement for the
program. Limitations included a single
school intervention and one yearâs worth of experience and student feedback
data.
Curriculum
222 – Reconnecting the Mouth to Medical Education
Mariah Rudd, Richard Vari, and Cynda Johnson
Virginia Tech Carilion School of Medicine
PURPOSE: The absence of oral health education in medical education has resulted in physicians who are poorly equipped to diagnosis, treat, refer and collaborate on matters of oral health and disease. This poster describes the successful integration of an oral health curriculum, with measurable outcomes, into the medical curriculum.
METHODS: In the first two years of the curriculum all medical students are now required to engage in thirty-six hours of oral health education which includes a lecture presented by an international expert on oral health, labs on anatomy/oral health, PBL patients with dental diseases, lectures on oral manifestations of systemic disease, oral cancer screening, and inclusion of the oral exam into the HEENT exam. In year 3 oral health is a component of the Family Medicine Clerkship. Assessments include a HEENOT (O=oral) exam with standardized patients, a written exam in year 2, student satisfaction of the oral health week, and a retrospective chart review comparing documentation of oral components within student (n=100) and attending (n=100) HEENT exams.
RESULTS: The average score on the oral written exam has been 89%. Qualitative findings from dental week evaluations have been extremely positive. The retrospective chart review revealed that students were more likely to include the documentation of periodontal tissue (students 12%, attendings 9%), dentition (students 46%, attendings 15%), oral cancer screening (students 49%, attendings 32%) and mucous membranes (students 60%, attendings 35%) within their HEENT examinations.
CONCLUSION: In a partnership
with Delta Dental, who provided financial resources for the curriculum, VTCSOM
has completed 5 years of educating and training future physicians on the
importance of oral health.
Curriculum
228 – A PUBLIC HEALTH CURRICULUM: MAKING IT REAL
Carol Nichols, Barbara Russell, and Kathryn
Martin
Medical College of Georgia at Augusta University
PURPOSE Undergraduate medical education curricula must include medical consequences of common societal problems. We have designed and implemented an interactive, pre-clinical curriculum where students learn about the role of public health in mitigating these problems at the local, state, national, and global levels.
METHODS The first year curriculum included a public health module that began with an Ebola outbreak simulation to highlight global health. Social determinants of health, the opioid epidemic, and health policy issues at the state and national levels were discussed. Students participated in a poverty simulation to learn about community health. The second year curriculum included Disaster Day training, an overnight bus tour to rural Georgia, and a trip to the Centers for Disease Control and Prevention (CDC). During Disaster Day, students activated a mobile hospital, triaged patients, participated in airway and breathing simulations, and received âStop the Bleedâ training. The trip to rural Georgia exposed students to healthcare in underserved areas. During the CDC visit students explored global health, chronic disease prevention, emergency preparedness, and infectious diseases.
RESULTS Students investigated the top health issues in GA and developed Public Service Announcements and poster presentations that addressed various aspects of these issues. Additional public health issues were explored through student-led case presentations. Evaluations of how well instructional materials/methods aided understanding of topics, the extent to which the content contributed to their development as a future physician, and overall quality of the curriculum all increased as a result of these curricular changes.
CONCLUSIONS The inclusion of
horizontally and vertically integrated public health content into the
pre-clinical curriculum has given students unique opportunities to participate
in interactive, experiential learning sessions that highlight common societal
problems and the role of public health in mitigating those problems.
Curriculum
232 – Collaborative Testing in Medical Education: Student Perceptions and Knowledge Retention
Jennifer Eastwood, Katherine Kleinberg and, David W. Rodenbough
Burrell College of Osteopathic Medicine
Purpose At BCOM, we utilize collaborative testing, in which students take exams individually, then again in small groups. Prior research shows collaborative testing enhances short-term knowledge retention. We sought to understand how students perceive the value of collaborative testing and whether collaborative testing is associated with long-term knowledge retention.
Methods Quantitative data included student exam performance during the Renal 1 course, and on a post-test delivered 8 months later in Renal 2. We tested knowledge retention for two conditions: exam review and collaborative testing. After individual testing in Renal 1, groups of 2-3 students received half of the test questions with answers to discuss and half of the questions without answers to complete. The post-test included 10 comparable questions per condition from the Renal 1 exams. Knowledge retention was determined for each student as post-test score/pre-test score, and a paired t-test was performed. Qualitative data included responses to prompts on studentsâ perceptions of collaborative testing and exam review collected during Renal 1. Two researchers analyzed the data using grounded theory.
Results Quantitative results showed knowledge retention to be significantly higher for the collaborative testing condition as compared to the exam review. Qualitative results showed that students engaged in more discussion in the collaborative testing condition and valued the teamwork skills it fosters. However, students expressed that not receiving correct answers was detrimental to their learning.
Conclusion Collaborative
testing enhances retention of knowledge and may be effective for students to
maintain knowledge through board exams and beyond. Our findings suggest that
the discursive process fostered by answering questions collaboratively is
instrumental to retaining knowledge. Our data, however, do not support
studentsâ perception that receiving correct answers is critical for learning.
This may support promotion of an epistemic shift for students, encouraging them
to value deliberative process over correct answers.
Instructional Methods
310 – TRAINING IMPROVES STUDENT PERFORMANCE AND PERCEPTIONS IN SMALL GROUP
LEARNING
Mari K Hopper
Indiana University School of Medicine
PURPOSE Fifty percent of course contact time in Indiana University School of Medicineâs (IUSM) ârenewedâ curriculum was dedicated to small-group learning. Course evaluations indicated that students did not value the approach, perhaps due to lack of training in this methodology. This study set out to determine if training in small group dynamics would improve student participation in and perception of this learning method.
METHODS Following IRB approval, case-based small-group sessions were audiotaped on two occasions prior to training (Pre), and two sessions following training (Post). Recordings were evaluated and scored by trained evaluators using a rubric including the following categories: Participation, Shared Roles, Focus on Learning Objectives, Approach, and Interpersonal Interaction. Category scores were averaged across the three evaluators both Pre and Post. To assess student perceptions, a 15-question survey was administered at three time periods: 1) before Pre small group sessions; 2) after recording Pre sessions and directly prior to training; and 3) after Post small group sessions. Topics such as personal preparation, interpersonal interactions, and prior undergraduate experience were included in the survey and rated using a Likert scale scoring one through seven.
RESULTS Although work is ongoing, preliminary analysis of audio recordings indicate that participation increased following training, with members participating more equally and encouraging input from others more frequently. There was little change noted in role sharing and addressing learning objectives. Survey responses following training revealed that students enjoyed small group sessions more, contributed more equally, and had fewer tangential discussions when compared to pre-training.
CONCLUSION These data
indicate that training in small group dynamics prior to participation in small
group learning activities will improve student discussion performance and
perceptions of the learning experience.
Instructional Methods
332 – COLLABORATIVE DEVELOPMENT OF CONCEPT-BASED CASES FOR COGNITIVE INTEGRATION
Tracy Fulton, Michael Dell, Leslie Fall, David
Harris, James Nixon, Ann Poznanski, and Amy Wilson-Delfosse
University of California, San Francisco School
of Medicine, Case Western Reserve University School of Medicine, Aquifer and
Geisel School of Medicine at Dartmouth, University of Central Florida College
of Medicine, and University of Minnesota Medical School
PURPOSE The Aquifer Sciences curriculum is a resource for the development of tools to support learnersâ cognitive integration. We have recruited teams from six medical schools to develop virtual patient cases, each focused on the assessment of a single Integrated Learning Objective (ILO), and thus a single clinical decision, from the curriculum. These cases will provide clerkship students with deliberate practice in cognitively integrating, transferring and applying their basic science knowledge to safe and effective clinical decision-making.
METHODS Each school recruited a team of basic science educators, clinical science educators, and senior students to collaboratively design cases. Teams were provided case authoring guides and webinar training on authoring clinical decision questions, basic science justification questions, and integrated explanations. Teams were then assigned a âcalibration caseâ (i.e. provided the same concept, ILO and common condition) in order to define best practices and to ensure a consistent approach to authoring cases that draw on the concepts, teaching points, and Harm Statements of the ILO required to make the decision.
RESULTS At this time, each school has completed a draft calibration case with questions and model explanations. Teams took different approaches to involve students, the number of answers for the clinical decision question, the length of the basic science justification, and the process for hand-off among and between team members. Most teams circulated drafts electronically but incorporated at least one team meeting. Results from the next phase of the project, in which each school will develop multiple cases to support either knowledge transfer or integration, will also be discussed.
CONCLUSIONS Curricula that support
learnersâ cognitive integration are challenging to design. The case development
framework described here requires learners to demonstrate an understanding of a
basic science concept in making a provisional decision, leading to an
entrustment action.
TBL/PBL
500 – “Describe What You See”: A Pilot Project in Converting
Problem-Based Learning from Text to Videos in Preclinical Medical Student
Psychiatry Education
Mary Steinmann and Sean Ferrell
University of Utah
PURPOSE Problem-Based Learning (PBL) is a commonly-utilized teaching method in preclinical medical student education. PBL consists of cases that students work through in small groups with a faculty facilitator, who may or may not be a content expert. At our institution, the cases are typically presented in text format with progressive disclosures as the case unfolds. However, important psychiatric concepts, including mental status examination findings, are difficult to learn. Video has been used widely in psychiatric teaching. In an effort to improve student engagement and understanding of observable psychiatric findings, we developed and piloted a series of progressive-disclosure cases that use videos of simulated psychiatric patient encounters to stimulate small group discussion and problem solving, facilitated by faculty who are not content experts.
METHODS Three progressive-disclosure video cases were developed to teach depression, psychosis, and peripartum mood disorders. The videos employed psychiatric residents as actors to reduce cost and enhance authenticity. The cases were piloted during weekly 1-hour PBL sessions in the second-year medical student Brain and Behavior course. A separate facilitation guide was developed describing key teaching points of each case disclosure, and offering suggestions to help faculty who were not experts in behavioral health facilitate student discussion. A 30-minute training session was held with faculty facilitators prior to each session to review the case and answer questions. The activity was assessed on standard end-of-course evaluations.
RESULTS 120 second-year medical students participated in the pilot project. Each case took approximately 1 hour for groups to complete. 92% of students on end-of-course evaluations reported that the videos used in PBL increased the authenticity of the cases. 91% of students reported that PBL sessions enhanced their learning of course content.
CONCLUSION Video-based PBL is
an engaging method to teach psychiatric principles including mental status
examination/observation, diagnosis, and treatment in preclinical psychiatric
education.
TBL/PBL
508 – Evaluation of Premedical Module at Duke-NUS Medical School
Lee Cheng Jie Irene and Sarada Harichand
Bulchand
Duke-NUS Medical School
PURPOSE: Duke-NUS medical school, a graduate entry medical school, the first in South East Asia, adopts Team-based learning (TBL) in delivering preclinical content. To recruit prospective MD students, we developed a Premedical Module using TBL as the main pedagogy to drive critical thinking and communication skills. The module, runs for 16 weeks, and is offered to local undergraduate students across faculties. This abstract describes the project and its outcomes.
METHODS: The module has run for 7 years. A wide range of topics in biomedical science were delivered in TBL format together with team assignments and field trips. The course was facilitated by research fellows. Students who completed the module with outstanding performance were shortlisted as Premed Scholars. These individuals received clinical shadowing experience to gain insight into the roles of medical professionals in Duke-NUS. To evaluate the effectiveness of the Premedical module as a recruitment tool, we analyzed studentsâ reaction to the module, using thematic analysis on session evaluation reports, and also determined the application and acceptance rate to the MD programme.
RESULTS: 169 students took the module from 2010 to 2015. 91 students were selected as Premed Scholars. Common perceptions of the module included an enriching exposure to different aspects of cutting edge medical research, and a positive learning experience through TBL which gave an insight into the unique pedagogy used at Duke-NUS. On average, in each academic year, 24% of Premedical modules takers applied, and among those who applied an average of 72% were accepted to the MD programme. Among the Premed Scholars, on average 50% applied and 78% were accepted to the MD Programme.
CONCLUSION: The Premed module
provides an opportunity to attract high quality candidates. Experiencing the
same teaching pedagogy of the MD programme, and routine engagement with the
medical school facilitates quality applicants and entrants to the MD
programme.
Technology and Innovation
606 – Evaluating the effectiveness of 360° videos in promoting engagement in
medical education
Neeral Shah, Nathaniel Larson, and Vivian Chan
Department of Medicine, University of Virginia
PURPOSE As 360° video technology has become more advanced and accessible, new ideas regarding its utilization in medical training have emerged. With this study, we have sought to examine whether 360° videos promote increased engagement over standard 2D videos among medical students.
METHODS We enrolled 39 fourth-year medical students to watch two 4-minute videos of exercises in an anatomy lab in a 360° 3D format with an immersive headset or in 2D format on a laptop computer. Every two minutes, students were asked if they were thinking about any thoughts unrelated to the video and to rate their engagement on a scale of 0-100. Following the videos, participants ranked on a scale from 0-100 their concordance with 14 statements related to engagement, practicality, and interest in the technology. Participants were also asked to describe what they perceived as the positives and negatives of their video experience.
RESULTS While watching the videos, the average engagement reported by the 360° video group was higher at each time point than the engagement reported by the 2D group. Further, the engagement remained high in the 360° group through the 6 and 8 minute timepoints. In the post-video survey, the 360° group reported a statistically significantly higher average engagement in seven out of eight of the relevant statements. 360° video was rated as more practical and interesting than 2D. There was no significant difference in the perceived ease of learning. Additionally, when asked to give feedback, more individuals in the 360° group cited feeling engaged or involved.
CONCLUSIONS Overall, our
results suggest that use of 360° video may improve engagement for short videos
used in medical education.
Technology and Innovation
612 – THE USE OF VIRTUAL REALITY WITH CLINICAL CASE STUDIES TO FACILITATE
LEARNING OF 3-DIMENSIONAL ANATOMICAL CONCEPTS IN CARDIOLOGY
Roberto Galvez, Laura Shackelford, Jenny Amos, and
Judith L Rowen
Carle Illinois College of Medicine, University
of Illinois at Urbana-Champaign
PURPOSE There has been recent interest towards utilizing virtual reality (VR) as an educational tool. As with any new tool it is important to determine what educational content VR is most effective in delivering. We describe a clinical case-centered VR task that was used to effectively teach first-year medical students the 3-dimensional aspect of collateral circulation, an anatomical property that is often difficult to teach with traditional methods.
METHODS Groups of four students were given access to computers running the VR version of Organon (a commercially available anatomy program). Students were provided an overview of the software and then presented with a clinical case of a patient exhibiting symptoms consistent with aortic coarctation. They then completed the following tasks: 1) propose the most likely cause for the patientâs symptoms; 2) using the VR tools, diagram the most likely route for blood flow around the occlusion if the coarctation was A) preductal or B) postductal. Upon completion, each group submitted a single document with their proposed answers. Students also received further examination on this content in weekly quizzes (NBME-style, from Firecracker).
RESULTS All 32 students completed the tasks to a sufficient level of competency within the 1 hour time. Weekly quiz performance indicated student competency; performance on 3 related questions was 97% for Carle Illinois students, compared to 76% for other US students. Upon examination of the activity as a whole, it 1) successfully educated the students on the predetermined objectives, 2) stressed a collaborative teamwork approach to learning, and 3) utilized VR to effectively assist with the educational objective.
CONCLUSION As medical schools
move towards utilizing newer technologies it is important to remain cognizant
of what educational content is best conveyed with these technologies. We
provide an example of educational content that was successfully provided
through VR.
Other
700 – A MODERN APPROACH TO MEDICAL EDUCATION: COLLABORATIVE ANATOMY-PATHOLOGY
PROJECT
Jessica Sohn, Bonnie Brenseke, Howard Reisner,
Terence Mitchell, and Bruce Newton
Campbell University School of Osteopathic
Medicine
PURPOSE: The Collaborative Anatomy-Pathology (CAP) Project provides medical students a deeper understanding of anatomy, histology, pathology, and clinical medicine via linking disease seen at gross and microscopic levels with clinical implications of the disease.
METHODS: First year medical students document lesions found during gross anatomy dissection. Samples are collected and undergo routine histologic processing. The resulting glass slides are scanned to permit virtual microscopy. After extensive literature review, participating students construct scientific posters, under faculty guidance, for presentation at research symposia. Examples of poster topics have included hypertensive nephropathy, teratoma, prostatic adenocarcinoma, and porcelain gallbladder. After presenting, students complete a 10-question Likert survey to assess the impact of the project on their medical education.
RESULTS: The CAP Project has been conducted for the past five years. Twenty-eight students participated in the first year and 49, 77, 63, and 98 participated in subsequent years (315 in total). The majority of surveyed participants agreed or strongly agreed the program increased their interest in anatomy (96%), pathology (86%), and research (78%). The participants also noted an increased ability to perform literature searches (76%) and compose a poster suitable for presentation at a scientific session (94%).
CONCLUSIONS: Integration of
the basic and clinical sciences is one of the primary goals of modern medical
education. Outcomes showed students were more observant during their
dissections as they looked for pathological processes; and they learned how to
compose and present a scientific poster as well as how to use medical
informatics to research various disease states. The CAP Project demonstrates
that anatomic dissection can be part of an innovative and integrative teaching
strategy in medical education. The low cost (~$8/H&E slide), ease of
preparation, and reliance on available material make this an attractive option
for many professional programs. Examples will be shown.
Other
708 – ASSESSING GENDER DIFFERENCES IN INFLUENTIAL FACTORS AND CAREER GOALS FOR
FOURTH-YEAR MEDICAL STUDENTS WHO CHOOSE INTERVENTIONAL RADIOLOGY
Rebecca F. Y. Minner
University of Arkansas for Medical Sciences
PURPOSE Women remain underrepresented in interventional radiology (IR) with women comprising only 13.3% of IR residents and 9.2% of IR academic faculty. The purpose of this study is to better understand how influential factors differ between men and women who choose IR residencies, as well as gather data on the intended career paths of men and women choosing IR as a career.
METHODS The IRB-approved study uses de-identified 2015-2017 Graduation Questionnaire data from the Association of American Medical Collegeâs national survey of fourth-year medical students. The data were collected during the transitional time from the IR fellowship training model to the IR residency training model. A focused analysis was on women (n=118) and men (n=483) choosing IR. These respondents were compared regarding factors that influence their decision to choose a specialty and regarding intended career path plans using a chi-squared test or Fisherâs exact test, as appropriate.
RESULTS Women were more likely than men to indicate that mentoring was a strong influence (p=0.0010), whereas men were more likely to indicate that their future family plans were a strong influence (p=0.0047). Women were more likely than men to indicated future participation in research (p=0.0496), teaching (p=0.0402) and public health (p=0.0335) during their career.
CONCLUSION To combat the IR
gender gap, there should be an increased effort by IR physicians to encourage
and facilitate mentoring relationships with medical students. Since, women are
more likely to want to enter into research, teaching, and public health
settings, it is important to utilize this research to better recruit female
medical students into IR and guide students to resources that allow them to
achieve their desired career goals.
Other
714 – PHYSICIAN ASSISTANT FACULTY ADVISEMENT DURING CREATION OF AN ULTRASOUND
STUDENT INTEREST GROUP
Ian M. Smith and Sarah J. Garvick
Wake Forest School of Medicine Dept of PA
Studies
PURPOSE Use of ultrasound in medicine is becoming a necessary skill set. However, the intensive training required for advanced competency in ultrasound is prohibitive for most medical curricula. One method for students to gain advanced training is an optional student interest group. This abstract gives recommendations for successful advisement during initial implementation of a Physician Assistant (PA) Ultrasound Student Interest Group (USIG) at an institution with dual campuses.
METHODS All students are introduced to medical ultrasound early in the curriculum, as they receive instructions on basic utilization of ultrasound and techniques for evaluation. A subset of students advocated for the creation of a USIG desiring additional training. Although the group is student-led, faculty with clinical experience in medical ultrasound have been essential in the development. Faculty involvement included alignment of educational goals, oversight of bylaw creation, guidance for institutional protocols, and preservation of the USIG over time. Furthermore, having dual campuses required close faculty support to guarantee continuity of content through lectures from subject matter experts, workshops, and labs at each program location.
RESULTS Sixty out of 89 students (67.4%) participated in the inaugural year. Participants from both campuses (n=20 and n=40) demonstrated an interest in increasing their ultrasound skills. Students participated in 8 workshops during their first year of PA education. Topics included a focus on point of care ultrasound (POCUS) skills, among others. Workshops were advertised to other medical learners throughout the institution to encourage interprofessional education and collaboration.
CONCLUSION Establishment of a
USIG is vital for students who want to accelerate their training in medical
ultrasound for clinical practice. Faculty advisement is imperative to ensure
design of a valuable and successful resource that allows for a relevant, goal
oriented experience that is sustainable for PA students.
Other
716 – Physiciansâ Perspective on their Preclinical Medical School Curriculum
Judith Binstock and Maria A. Pino
Touro College of Osteopathic Medicine and NYIT
College of Osteopathic Medicine
PURPOSE This preliminary study is the first to ask physicians how they would change their medical school preclinical curriculum for best-practice medicine. The purpose of this research project was to answer: (1) Did osteopathic physicians consider their preclinical curriculum supportive of best-practice medicine? (2) What suggestions would these physicians have to improve their preclinical curriculum?
METHODS A 12-question survey was emailed to the 2011-2018 alumni of Touro College of Osteopathic Medicine, Harlem, NY. Questions were asked on demographics and type of practice, but focused on the assessment of specific basic science courses by asking: âShould have Moreâ, âSufficientâ or Should have Lessâ. Additional questions included interest in having clinically relevant basic science presentations and the importance of including other topics, e.g. basic statistics, early patient exposure, clinical cases, etc. A statistical analysis (SPSS) was performed and the relationships between variables was tested using a Pearson Chi-square (a=.05)
RESULTS (1)Demographics: The respondents (N=122) were 55% male and 44 % female; most were practicing or associated with an academic hospital (57%) and 48% were primary care while 52% were non-primary care physicians (including Emergency Medicine); (2)Basic Science Courses: Pharmacology(41%), Pathology(39%) and Physiology(39%) ranked the highest as âShould have Moreâ while Histology/Lab(45%) and Embryology(35.5%) ranked the highest as â Should have Lessâ; (3) 60% were interested in clinically relevant basic science presentations; (4) Early patient exposure (65%), increasing clinical cases (70%) and analysis of journal articles(70%) were ranked highest as âShould have Moreâ. A statistical analysis between variables and questions regarding preclinical courses found little significant differences and no overall patterns.
CONCLUSION Pharmacology,
Pathology and Physiology were identified as âShould have moreâ as were clinical
cases, journal article analysis and early patient exposure. Asking practicing
physicians what they feel would help them in âbest-practiceâ medicine is
crucial in designing future medical school curriculums.
Other
718 – CONTINUING MEDICAL EDUCATION FOR PRIMARY HEALTHCARE PROVIDERS TREATING
YOUNG ADULTS WITH AUTISM
Megan McCrohan and Mary Dereski
Oakland University William Beaumont School of
Medicine
PURPOSE National data indicates that 66,000 individuals with Autism Spectrum Disorder (ASD), will reach the age of adulthood annually; therefore, many primary care providers will likely treat ASD patients. Previous studies indicate that knowledge of ASD, along with appropriate accommodations, can improve healthcare delivery.
METHODS A module was developed describing typical ASD behaviors, including clinical approaches to enhance the healthcare experience. The module was distributed to primary healthcare providers. Efficacy for increasing knowledge and awareness of ASD was assessed through pre- and post-module surveys.
RESULTS Twenty-one primary care providers chose to enroll in the research study online (n=14 Family Medicine, n=5 Internal medicine, with the remaining from Pediatrics, Internal Medicine/Pediatrics, and Preventative Medicine). Participant responses indicated that autism knowledge about the correct number of ASD individuals turning 18 annually, increased by 64% after taking the module. When asked âHow familiar are you with the challenges that patients on the Autism spectrum face when transitioning to adult primary care?â, respondents increased their familiarity from 38% in the pre-survey to 100% in the post-survey. When asked âif a patient or their previous physician has not disclosed that your new patient is on the Autism spectrum, would you ask the patient to fill out a form?â, pre-module responses increased from 50% indicating âdefinitelyâ or âyesâ to 92% for post-module responses. In the post-survey, 50% of participants indicated that they would definitely approach the delivery of healthcare to a patient with ASD differently, with 100% stating they would make accommodations after completing the module.
CONCLUSION This study suggests that a module providing ASD information and potential patient-related accommodations may increase healthcare provider awareness regarding how to improve healthcare delivery for this population. It may, therefore, be warranted for primary care providers to have access to this information prior to treating patients with ASD.