Session 1 – Assessment
1 – Student Academic Performance Factors Affecting Matching into First-Choice Residency and Competitive Specialties
Scott Helf
PURPOSE: To identify academic performance factors affecting matching into first-choice residency and highly competitive specialties using the Careers in Medicine® competitiveness ratings.
METHODS: The authors conducted a study of 1,726 graduates from their institution between 2010 and 2017 and assessed pre- and post-admission academic variables for associations with matching into first choice and specialties of varying competitiveness.
RESULTS: Approximately half (53.9%, n=931) of graduates matched into their first choice. Matching into first choice was associated with passing COMLEX Level 2 CE (P=0.01), PE (P=0.02) on the first attempt, and higher COMLEX Level 2 CE and USMLE 2 CK scores (<0.001 and 0.002; 14.1 and 3.9-point difference in mean scores respectively). Pre-clinical GPA (P=0.002) and highest MCAT score (P=0.02) were also associated, however differences in means were <1 point for both. Matching into highly competitive specialties was associated with passing COMLEX Level 1 (P <0.001), Level 2 CE (P <0.001), USMLE Step 1 (P <0.001) and Step 2 CK (P=0.03) on the first attempt. Mean scores of students matching into high- versus low-competitiveness specialties differed as follows: COMLEX Level 1 62.7 points, Level 2 CE 50.5 points, USMLE Step 1 13.6 points, Step 2 CK 7 points (all <0.001), as did mean pre-clinical GPA (2.4 points, P <0.001).
CONCLUSION: Licensing exam performance is important for matching into first-choice residency and highly competitive specialties. However, differences in scores were not as pronounced for matching into first choice as for highly competitive specialties. These results may help faculty prepare students and inform curriculum design to improve matching.
2 – Interpreting Results of the National Graduation Questionnaire
Emmanuel Mbaku Ngu
PURPOSE: Every year medical Schools around the country require their out-going 4th year students to complete an exit questionnaire (GQ), which requires subjective input, regarding student perceptions about their preparation for clerkship, internships as they enter the next phase of their medical training. Student responses to the subjective questions regarding their basic science learning during the 4 years of their medical education are used by the LCME and other accreditation agencies, to obtain national comparison about the quality of education that students are receiving. Those medical schools falling below the national average in different basics science areas are subjected to critical review of the subject matter implicated, resulting in an overhauling and numerous revisions of how courses are taught. This abstract looks at whether there is any type of correlation between the results of GQ and performance in step-1 USMLE scores.
METHODS: The relationship between the results of the GQ questionnaire on basic science subjects (such as pharmacology, biochemistry) and students’ performance on step-1 USMLE at both Augusta University and the University of Georgia Medical Partnership for 2017, 2016, 2015 and 2014 academic years were examined for any correlations.
RESULTS: For all four years examined, there was no correlation between the results of GQ and corresponding students’ performance on step-1 USMLE scores. In all the cases the results of the GQ questionnaire were significantly lower compared to the national average than the correspond performance on step-1 USMLE.
CONCLUSIONS: The results of the GQ may provide useful information in some domains of the curriculum. However, they has no bearing on how well students have learned the basic sciences.
3 – A Mock Exercise Contributes to an Improvement in Students’ Performance and Satisfaction with Essay Exams
Jenny Fortun
Purpose: We typically use essays to evaluate students’ critical thinking and ability to apply foundational knowledge to novel clinical scenarios. Students have expressed difficulty with the use of open ended questions, lacking a clear understanding of what questions are asking or how much information to provide.
Methods: For the current class, we introduced a mock essay two weeks prior to their first essay. We compared the numerical performance of the current year (mock essay) with the prior class (no mock essay) and student satisfaction via surveys.
Results: After conclusion of the mock essay, 80% of students felt that this exercise was extremely/very useful. The numerical performance in the first essay was significantly higher for the current class (72% vs. 67%, p=0.002). The current class performed significantly higher in questions that required students to explain mechanisms of disease (different question, similar level of complexity; 76% vs. 53%, <0.001), and to interpret the results of the same diagnostic test (different gene; 65% vs. 58%, p=0.03). The numerical average in multiple choice questions (MCQ), which occurred within one week of essay 1 and tested similar material, was not statistically different between the two cohorts (p=0.12). The correlation between essay 1 and MCQ (Spearman correlation) was 0.423 (<0.001). In the end-of-course-survey for the prior year, there were 67 unsolicited comments regarding essays being misleading and having unclear expectations. This year, there were only three students with similar unsolicited comments. When the current year was asked about their preferred assessment method, the majority of the students preferred to be tested by MCQ (67%), compared to essays. Some students had concerns about subjectivity of essays and grading (33%), whereas others highlighted the strengths of essays (36%).
Conclusions: We conclude that incorporating a mock exercise before the essay is very valuable to improve students’ performance and satisfaction.
4 – An Integrated “First Patient” Discoveries Activity Assesses Multiple Competency Domains
Anna C. Edmondson
PURPOSE: A “First Patient” Discoveries Activity was introduced to provide students the opportunity to investigate the life and death of their first patient – their body donor. The goal of this activity is to integrate basic and clinical sciences and assess competencies including observational medicine skills, evidence-based reasoning, and awareness of how public health and socioeconomic factors contribute to health and disease.
METHODS: First year medical students (n=~190/year) are divided into 24 dissection groups. While dissecting, students complete a body donor observation form, noting pertinent anatomical findings. Pathologies are biopsied and submitted with a report indicating the reason for the biopsy and differential diagnosis. Based on observations, pathological findings, and research, students hypothesize their patient’s cause of death (COD) and submit a final report including a discussion of observations and pathological findings relevant to public health and socioeconomic factors in their patient’s life. The assignment is evaluated using a grading rubric. Students’ perceptions are gathered from a curricular evaluation survey (n=105).
RESULTS: Students performed satisfactorily, with an average final report score of 84.1%. Most students met or exceeded expectations on discussions of how behavioral, public health, cultural/societal, and socioeconomic factors impacted the presentation and outcome of disease in their patient (average scores: 85.6%-87.8%). Students performed well with diagnostic reasoning, scoring 89.9% on pathology submissions, yet struggled with demonstrating observational medicine skills, scoring 69.5% on the body donor observations form. When surveyed, students (59%) felt the activity promoted observational medicine skills, and 50% felt it increased their awareness of the impact non-life threatening pathologies have on quality of life.
CONCLUSION: The “First Patient” Discoveries Activity has been successful in helping students integrate information and promoting mindfulness of how pathologies affect patients’ quality of life. Students are learning observational medicine skills but require additional opportunities to practice and improve this skill.
Session 2 – Assessment
1 – Educational Effect of Assessments: Essays can Promote Study Strategies that Develop the Clinical Reasoning Process in First Year Medical Students
Helen Tempest
Purpose: Typically, first year medical students have had limited opportunities to practice problem solving skills using a clinical case. We believe that how students study for essays impacts the development of this skill set.
Methods: Students were administered three essays biweekly in the first course in medical school. We evaluated whether students use different study strategies when preparing for essays and whether their perceived learning was influenced by how they studied. For this, we examined their responses using guided questions in the end-of-course survey.
Results: When students were asked whether they studied differently for multiple choice question (MCQ) compared to essay exams, the majority (63%) indicated they did. The survey results indicated that 53% of students focused on diseases and symptoms/signs rather than molecular details when studying for essays. These students reported they reorganized their study strategy by starting with disease presentation, and correlating patients’ signs and symptoms with molecular and cellular mechanisms learned in lectures. All of these students commented on the benefits of studying for essays using these strategies. Of note, 19% of the class reported using less time trying to memorize details. On the other hand, 6% of students reported essays required more memorization. Interestingly, 19% of students reported working in groups, practicing giving explanations and integrating lecture material. 13% of students reported focusing on content delivered via active learning pedagogies (i.e. case based learning and flipped classrooms) rather than lectures, and 18% of students reported using less practice MCQ banks. When specifically asked how many hours they spent studying for essays, 55% of students indicated they studied throughout the course and their studies overlapped with studying for MCQ test.
Conclusions: Most students focused on disease presentation and correlation with foundational sciences, as expected. Essays throughout medical education are useful to further develop this skill set.
2 – We need a SIX Pack… for Faculty Development
Heather Billings
PURPOSE: The importance of providing consistently high-quality, value add, applicable and relevant faculty development to our medical educators is more important than ever. Our challenge was to create a series of high quality stand-alone, mini workshops that can be delivered by faculty to faculty, within their own practice environment.
METHODS: A 2016 systematic review of more than 100 faculty development programs suggests that the most successful programming includes, evidence-informed design principles, relevant content, experiential learning, opportunities for practice, application, feedback and reflection, and intentional community building.
The SIX-Pack Faculty Development project includes a collection of Succinct, In-person, eXpandable faculty development sessions that can be taken off of the “shelf” (ie downloaded from a website), reviewed and adapted by the facilitator, and delivered, in person to a diverse audience – when and where appropriate. The SIX-Packs include:1) objectives, 2) powerpoint slide deck, 3) facilitator guide, 4) activities, trigger videos, case scenarios, 5) resources, and deeper dive references, and 6) evaluation tools.
RESULTS: Five SIX-packs have been created and are being facilitated by faculty across the enterprise.. Initial quantitative and qualitative data is being collected through evaluation forms designed to collect data across Kirkpatrick’s evaluation continuum – ample results will be reportable by Spring 2018. The Successive Approximation Model (Design, Develop, Evaluate, repeat) is utilized for continuous improvement and evaluation data is fed back into the design and revision process (Allen, 2012).
CONCLUSIONS: Anecdotal testimonials suggest this model of faculty development is effective, easy to use, relevant, and a good use of participant’s time. Early lessons learned and educational technologies utilized are advancing the skills and competencies of a large group of medical educators, education specialists and instructional designers within the institution. Additional stakeholders benefiting from this project include Program Directors, Department Chairs, and Education Leadership.
3 – Automated Assessment of Student Learning and Curriculum Mapping Using Machine Learning
Ian S. Haworth
PURPOSE: Assessment of student learning of curricular outcomes is dependent on mapping of outcomes to exam questions. Similarly, evaluation of coverage of outcomes in a curriculum requires mapping of these outcomes at the lecture (event) level. The goal of this work is to show that machine learning can be used as an objective and time-saving approach to both of these tasks and that the results can be used to identify gaps in student learning and in curricular coverage of outcomes.
METHODS: Curricular outcomes, including institution-specific and national outcomes, were mapped to exam questions and event content using an algorithm that matches content between documents and scores each outcome from 0 (unrelated) to 1 (strongly related) for a particular question or event. To evaluate student performance, the score for a student on a question (0 or 1 for multiple choice or a fractional score for essay format) was applied for each outcome included in the question. In this way, a percentage score was obtained for each outcome for each student across multiple exams. A similar analysis was performed to determine coverage of outcomes in events.
RESULTS: Data will be presented for mapping of outcomes to a curriculum and exams. A complete curriculum map will be shown to provide evidence of the accuracy of this approach and identify areas of weakness that require further study. The utility of machine learning for “real-time” tracking of student performance based on outcomes will be demonstrated. Results from a search procedure will be discussed that identifies gaps and overlap in the curriculum. This information is used to evaluate reasons for general weaknesses on particular outcomes in a class of students.
CONCLUSION: Machine learning is a new approach to assessment and mapping that permits objective evaluation of student learning and curricular content at the outcomes level.
4 – The Acquisition and Progression of Communication Skills During Pre-Clerkship Curriculum: Does Early Exposure to Clinical Skills Improve Medical Student Performance?
Michelle Yoon
PURPOSE: We sought to determine whether or not a new, integrated, systems-based curriculum that includes early clinical encounters improves pre-clerkship medical student performance in communication skills.
METHODS: Data were collected from two class cohorts of students in the old (N=338) and new (N=341) curricula, whom were respectively assessed on communication skills performance using the Essential Elements of Communication (EEC) tool. We compared the progression of, and end of the pre-clerkship EEC scores on an observed structured clinical examination (OSCE) using a one-way ANOVA and independent students’ t-tests, respectively.
RESULTS: Despite lower EEC mean scores (62.8% vs 69.8% < 0.01), on their first encounters, students in the new curriculum performed as well as or better than students in the old curriculum on the end of pre-clerkship OSCE (63.49% vs 61.88 <0.05). The EEC domains of “Gather Information”, “Build a Relationship”, and “Sharing Information” were statistically significant factors in explaining the variance of OSCE EEC scores.
CONCLUSIONS: In spite of having lower communication skills scores initially, students in an integrated, systems-based curriculum that featured early clinical experiences demonstrated slightly higher scores on an end of pre-clerkship OSCE assessment compared with students in an older, discipline-based, 2 x 2 curriculum. Early exposure to clinical skills provides a firm foundation in communication skills. Our findings support current curricular trends across many medical schools internationally, emphasizing more integration, earlier clinical experiences, and more emphasis on and/or inclusion of communication skills within medical school curriculum and instruction.
Session 3 – Curriculum
1 – Correlating Preclinical Ambulatory Care Specialty Exposure with First-year Medical Student Performance on an Objective Structured Clinical Examination
John E. Nolan, III
PURPOSE: Preclinical Ambulatory Care exposure offers medical students early opportunities to improve clinical performance; however, it is unknown whether exposure to generalist care rather than specific specialties results in variable performances on standardized patient examinations (SPEs). We intend to compare mean Objective Structured Clinical Examination (OSCE) scores for students by ambulatory specialty exposure.
METHODS: One hundred ninety-seven first-year medical students were exposed to one of ten outpatient specialties before taking a two-station OSCE. Performance was assessed in the domains of communication, elicitation of patient concerns and history-taking. OSCE scores were measured and compared by specialty and on the basis of generalist versus specialist preceptors.
RESULTS: By specialty, student performances in all domains did not significantly differ. Students with generalist preceptors outperformed peers with specialist preceptors in eliciting concerns from patients in their first case (1.00 vs. 0.98; p = 0.03), but results did not significantly differ in the second case or in the total score for that domain. No other scores significantly differed.
CONCLUSION: Our results allow for the recruitment of preceptors from a variety of outpatient specialties to provide preclinical students with a longitudinal ambulatory care experience without concern for significant differences in clinical development or performance in communicating, attaining clinical histories or eliciting patient concerns on an OSCE.
2 – Exodus from the Classroom: Student Perceptions, Lecture Capture Technology, and the Inception of On-Demand Preclinical Medical Education
Uzoma Ikonne
PURPOSE: In recent years, medical educators have noticed a decline in classroom attendance. Therefore, understanding student preferences for content delivery and the relationship between student attendance and learning outcomes may enhance curricular design and best practices for technology-supported learning.
METHODS: Attendance data were collected, for first-year students, during the 2015-2016 academic year in non-mandatory class sessions using audience response technology and were linked to student academic performance outcomes. Pearson correlations were calculated to quantify the relationship between classroom attendance and academic performance. Students were surveyed to characterize factors that influenced attendance decisions and compare perceived convenience, efficiency, and effectiveness between classroom attendance and lecture capture technology. Preferred modality to receive didactic content was assessed, and open-ended questions about the advantages/disadvantages of lecture capture, classroom attendance, and vodcast (video podcast) were included. Open and axial coding were used to analyze responses to these questions.
RESULTS: A 78% reduction in first-year student classroom attendance was measured from the beginning to the end of the academic year (<.001). The relationship between classroom attendance and academic performance (r=0.17, P=.29) and the partial correlation between them (r=0.18, P=.08) were not significant. Students perceived lecture capture recordings as more convenient, efficient, and effective than classroom attendance, and vodcasting was the preferred method of content delivery. Major themes associated with the open-ended questions were effective or ineffective time management, enhanced interaction, learning advantages or learning challenges, positive or negative content characteristics, and no disadvantages.
CONCLUSIONS: There was a significant decrease in first-year classroom attendance during the academic year, but we found no significant relationship between classroom attendance and academic performance. Students perceived lecture capture recordings as a practical alternative to attending class; however, vodcasts were the preferred modality to receive didactic content. The findings of the current study may be useful for developing learning-centered curricula at medical institutions.
3 – Integration of Addiction Medicine into Organ Systems Curriculum
Marta Ambrozewicz
PURPOSE: The challenge of integrating and refining existing addiction medicine curriculum into the new systems based curriculum is critical. With the sharp rise in the incidence of drug overdose related events, it is important to develop a physician workforce capable of impacting the estimated 60,000 deaths and hundreds of billions in total economic costs in the US each year due to addiction. While the topic of addiction is addressed in medical schools, thoughtful integration of concepts and providing opportunities for students to apply their knowledge is crucial.
METHODS: In an effort to address this, at Eastern Virginia Medical School, we integrated addiction medicine into the Heart-Lung-Kidney module. In collaboration with addiction medicine experts and basic science faculty, we designed an interactive and case-based session to meet our learning objectives. The session addressed the impact of opioids, alcohol, tobacco, cocaine and amphetamine on cardiovascular, respiratory, and renal systems. Pathophysiology, short term and long term effects of the drugs were discussed along with management aspects. Student knowledge was assessed throughout the session using the audience response system. Emphasis was made on addiction as a biological disorder and chronic disease.
RESULTS: Over 87% of the students (n=150) rated the session positively and multiple comments similar to, “unique and engaging presentation of the material” and “an excellent way of combining all the material we had learned” were present in response to open ended questions.
DISCUSSION: With the nation facing sharp rise in the incidence of misuse of drugs, it is timely that future doctors are trained adequately to manage these patient populations. Our aim is to improve detection, and ultimately outcome for those with addictions as well as equip physicians with the knowledge and skills to provide such patients with quality care.
4 – Returning to Basic Science in the 4th Year: A Flexible Elective in Pathophysiology of Disease Provides Meaningful Learning Opportunities
Margaret M. Maimone
PURPOSE: The curriculum at SUNY Upstate Medical University, requires students to complete 2 credits of elective work in basic science after completing clinical clerkships. To provide students with an opportunity to pursue basic science education within any area of interest, I created an elective that allows students to research the pathophysiology of any disease of interest with maximal flexibility.
METHODS: The Pathophysiology of Disease elective was designed with 3 core learning objectives in mind: 1) to analyze and synthesize current literature; 2) to create a learning module that reviews this literature; and 3) to present this knowledge to an appropriate audience. Students have flexibility in their choice of disease and the format of their learning module, from powerpoint presentations to case reports submitted for publication. Because students work independently, they can complete their work within any time frame. Grades (pass/fail) are based on faculty evaluation of the learning module and a reflection paper. Course evaluation methods include student surveys and analysis of reflections.
RESULTS: 23 students completed the elective in the first year, and 40 students are currently registered in its second year. Student surveys and reflections indicate the vast majority of students find the elective useful and would recommend it to others. They appreciate the flexibility in topic choice and timing that allows them to work during interview season, and they acknowledge honing their skills in public speaking and conducting literature reviews. They also enjoy the opportunity to conduct an in-depth exploration of a disease topic of their own choice.
CONCLUSIONS: The Pathophysiology of Disease elective has allowed students to return to the basic sciences in their 4th year in a way that students find meaningful to their development as future physicians.
Session 4 – Curriculum
1 – Development of Physiology Core Concept Framework to Facilitate Integration with Clinical Decision Making
David M. Harris
PURPOSE: Most physiology curricula, as well as national medical physiology learning objectives, are organized by organ system. However, clerkship students are expected to integrate knowledge between organ systems to justify clinical decisions. The traditional, organs-based framework is not ideal to facilitate this process. The purpose of this abstract is to present an integrative framework of physiology core concepts that promote better clinical decision making.
METHODS: This framework has been developed as part of the Aquifer Sciences (formerly MedU Science) initiative, a collaboration with IAMSE, to create a curriculum to promote student excellence in clinical decision making. The first step was to identify core concepts in physiology which would be used to develop learning objectives. As part of the curriculum, learning objectives need to be related to 1 of 5 clinical decisions (Diagnosis, Work-up, Course of Illness, Prevention, Management). We developed a framework of core concepts based on the three highest rated concepts (homeostasis, cell-cell communication, flow-down gradients) highlighted in previous studies.
RESULTS: There are ten core concepts in this framework: Regulation (Reg) of mean arterial pressure, Reg of electrolytes (K+ and Ca2+), Reg of osmolarity and blood volume, Reg of body temperature, Reg of pH, Reg of oxygen and CO2, Reg of reproduction, Reg of blood glucose, flow-down gradients, and cell-cell communication. Many of the other core concepts identified by previous reports are included within descriptions of each core concept or overlap among our selected core concepts. For example, learners need to understand negative feedback to address many of our core concepts.
CONCLUSION: The benefit of this framework is that it can be used to help learners to transfer knowledge across organ systems and improve clinical decision making.
2 – Don’t Re-Invent the Wheel, Re-Purpose It: Using Existing Cases to Integrate Basic Science Content and Differential Diagnosis Using Integrated Illness Scripts
Brian Wilcox
PURPOSE: Retention and transfer of basic science core concepts to clinical settings is crucial for developing clinical reasoning and medical decision-making. Learners who retain basic science knowledge have improved retention of clinical knowledge and improved diagnostic accuracy when faced with clinical challenges. Studies have demonstrated that integration of basic science learning and clinical learning, at the level of individual teaching sessions/cases, causes improved retention and transfer to difficult clinical cases. Many institutions find it challenging to integrate basic science content when teaching about differential diagnosis. Learners often forget the basic science information or fail to apply it, in patient encounters.
METHODS: This presentation will cite evidence supporting utilizing Venn diagrams and illness scripts to meaningfully integrate and transfer students’ basic science concepts into the process of developing a differential diagnosis. The methodology focuses on using these tools to illustrate the basic science that causes distinguishing features in alternative diagnoses. This approach provides a template for educators to easily apply these tools within cases that already exist in their curriculum, and with patients during clinical rotations.
RESULTS: These tools were piloted with first-year students during PBL sessions at two medical schools. Students reported enjoyment and satisfaction with using these tools for “getting to the core causes” of the key findings. Integration of the method into sessions required minimal class time or faculty development. Additional examples will be provided that demonstrate this process to integrate basic science into diagnostic reasoning in virtual patients with common conditions.
CONCLUSION: Most schools already utilize TBL or PBL cases, simulated and/or virtual patients to teach diagnostic reasoning. Venn diagrams and integrated illness scripts can be easily integrated into these cases to emphasize key basic science mechanisms underlying the clinical disease processes, thereby improving students’ diagnostic accuracy, and basic science retention and transfer.
3 – Use of a Curriculum Database to Drive Development of Cases that Foster Cognitive Integration of Foundational Science Knowledge and Clinical Decision-Making
Tracy B. Fulton
PURPOSE: Clear approaches to and tools for improving the ability of medical students to cognitively integrate foundational science knowledge into clinical decision-making are needed. The Aquifer Sciences initiative, the result of a national collaboration between IAMSE and Aquifer (formerly MedU) and has produced a robust curriculum consisting of clinically ‘integrated’ learning objectives (LO) that connect key clinical conditions to core basic science concepts through relevant clinical decisions. Each LO is paired with a basic science teaching point, a harm statement, and learning aids such as recall tools and common analogies. The Aquifer Sciences curriculum and the database in which it is housed can now provide the foundation for the development of cases that authentically leverage and cement basic science understanding, with the goal of improved recall and transfer.
METHODS: The Aquifer Sciences database is searchable by clinical condition, basic science discipline, core concept, and decision area. We are in the process of 1) defining a structural framework for case development by experimenting with approaches to searching the curriculum database and 2) using the database to identify and develop high-yield pilot cases. Each case begins with a patient scenario at the point of a crucial clinical decision that requires the application of basic science concepts and will cause harm if basic science understanding is not effectively utilized. Pilot cases are being defined that illustrate the application of at least ten core basic science concepts. Case conditions will be chosen to highlight important basic science understanding across multiple disciplines and clinical specialties.
RESULTS: The framework for case development, the curriculum database, and pilot cases will be presented.
CONCLUSION: The Aquifer Sciences curriculum database provides a critical resource for the development of cases which foster cognitive integration of basic and clinical sciences within the minds of students.
4 – Pathways for the Integration of Basic Science Content in Phase 1 Medical Education
Amy E. Medlock
PURPOSE: Reform in undergraduate medical education calls for the integration of knowledge, skills, and values. Harden’s Integration Ladder posits that integration is best achieved when learning is transdisciplinary and student-centered. Hurdles that we have encountered in designing sessions that meet these criteria include: (1) differences in experiences and teaching styles between educators, (2) high student/teacher ratios, (3) few existing teaching methodologies/formats to guide the development and implementation of integrative and active learning sessions, and (4) time constraints. The purpose of this poster is to outline the processes used and provide examples of newly designed sessions to overcome these barriers to integration.
METHODS: To overcome these challenges, we have successfully created integrated large group sessions in our curriculum by: (1) developing sessions that have two session “co-leaders” who negotiate the necessary session learning objectives, content, and assessment items; (2) utilizing student-centered methods that allow students to construct meaning; and (3) sharing teaching material and assessment items with experts in other disciplines and requesting input to construct learning activities that are transdisciplinary.
RESULTS: We have developed multiple integrated sessions and assessment items for our curriculum using the methods outlined. The sessions created include Cell Structure, Organelle Function, and Protein Trafficking; Muscle Metabolism; Liver Metabolism; Eye and Visual Pathways; Ear and Auditory Pathways; and Endocrine Pancreas. This team-based approach to integration has carried over into better integration of learning objectives, and the development of assessment items that better reflect USMLE Step 1 items.
CONCLUSIONS: Hurdles to integrating different basic science disciplines in Phase 1 medical education can be overcome through deliberate planning and cooperation of multidisciplinary teams to coordinate, develop, and implement various aspects of a given learning activity. While there is still work to do, overall, the development of integrated sessions has been important in removing artificial barriers that impede learning.
Session 5 – Instructional Methods
1 – Friday Formative Quizzes: A Group-format Review Designed with Input from Learners
Jessica Morgan Jones
PURPOSE: To promote recall-based versus recognition-based study in pre-clerkship students, Georgetown University School of Medicine implemented a weekly formative quiz followed by small group discussion throughout the pre-clerkship phase of its new Journeys Curriculum. This self-assessment activity was designed to provide feedback and encourage deeper study for understanding.
METHODS: Developed with input from students during the curriculum reform process, the weekly Friday Formative Quizzes (FFQ) were implemented in August 2017. Questions used the same format as the summative exams. Students work individually on their quiz questions for 15 minutes, then spend the remainder of the time discussing the questions in-depth in small groups. Faculty were invited to attend and participate in discussions. We monitored attendance and collected initial feedback via focus-groups at four weeks, and via end-of-module evaluations at seven weeks.
RESULTS: Nearly all of the students (90-95%) attended the FFQ every week, discussion was vigorous, and most students stayed for the entire allotted time. The FFQ were the most highly rated aspect of the pre-clerkship sciences curriculum. However, students were confused regarding the purpose of the quizzes, voicing concern about their performance and wanting it moved to Mondays to allow weekend study. We had to reiterate that the FFQ was to help students self-assess and plan/focus their weekend study. Students also requested written explanations for all questions and attendance from all faculty who had submitted questions, changes that have been implemented.
CONCLUSIONS: Regular formative quizzes followed by small group discussion are popular with students. However, students may not understand its self-assessment purpose, which is important for students to guide their study and realize the full benefits of the activity. Next steps include continued evaluation of the FFQ and its impact on student study.
2 – Student Perceptions of Using the Pathways of Human Metabolism Map for Learning, Studying, and as a Reference on Assessments
Doug Spicer
PURPOSE: To face the healthcare challenges presented by metabolism-related chronic diseases, students must be able to apply key metabolic concepts to clinical practice. Learning tools that support finding, interpreting, and applying core knowledge, rather than rote memorization, are needed. One tool that facilitates this shift was developed by Stanford University School of Medicine biochemistry course directors, who created a medicine-oriented metabolic map specifically designed for use in teaching, learning, and assessment. The Pathways of Human Metabolism (PHM) map has been vetted and approved by the Association of Biochemistry Educators. In this study, we describe how the Pathways of Human Metabolism Map has been utilized in three different medical school curricula. We also report on students’ perceptions and utilization of the map during both the learning and assessment phases of their curricula.
METHODS: First or second year students from three institutions were surveyed on their perceptions of using the PHM Map as a study aid and as a reference on assessments. Student responses were coded using Grounded Theory to identify overall themes and patterns that emerged.
RESULTS: Major themes that arose from students at all three institutions will be presented, including appreciation for not having to memorize pathways, for having a tool to help visualize integration between the pathways, and for being able to focus on clinical applications. Another theme that arose was concern about the potential to be disadvantaged on board exams. Quantitative results suggest a relationship between students’ perceptions of the map’s usefulness for assessments and the amount it was used in studying.
CONCLUSIONS: Despite significant differences in curricular structure, students across three institutions had similar reactions to the use of the PHM map. Initial results suggest that second year students are more likely to express concerns about the impact of the map on preparation for board exams.
3 – Using a Series of Simple Puzzle Phantoms to Encourage Medical Student Understanding of Ultrasound Fundamentals: Scaling Up
Lauren Maloney
PURPOSE: As ultrasonography is introduced to medical students, basic ultrasound physics and machine functions are often discussed with immediate progression to scanning human anatomy to evaluate organ structure and function. This approach bypasses a focus on learning the psychomotor skills required to manipulate the ultrasound probe to evaluate the structure of a three-dimensional object using two-dimensional information generated by the ultrasound. We developed a series of low-cost, homemade, gelatin-based ultrasound puzzles to facilitate understanding of rotation, depth, and alignment, using familiar objects in a transparent phantom. This allows students to connect the visual structure of an object to its ultrasound appearance. After a small preliminary study suggested the puzzle phantoms to be an effective teaching tool, we aimed to scale the sample size up to a full medical school class.
METHODS: 134 medical students participated in 30 minute sessions. First, they scanned an opaque puzzle phantom and answered questions about its structure. Next, they scanned three clear puzzle phantoms that each targeted a single fundamental principle, with immediate deliberate feedback provided by expert faculty. Additionally, students completed a worksheet that reinforced the objective of each puzzle phantom. They then rescanned the opaque puzzle phantom, and completed a survey online. The cost per student for phantom materials was less than $2.00.
RESULTS: Students demonstrated an increased ability to use alignment, depth, and rotation to answer questions about structures contained within an opaque puzzle phantom. The majority of students agreed that the experience was a productive use of their time, that their new skills would be useful in their careers, and that they would recommend the experience to others.
CONCLUSIONS: Reinforcing our previous preliminary results, our scaled-up series of low-cost puzzle phantoms seem to be an effective, well received technique to encourage medical student understanding of fundamental ultrasound probe manipulation.
4 – An Innovative Method for Teaching Advanced Clinical Skills in Lifestyle Medicine
Magdalena Pasarica
PURPOSE: Lifestyle medicine is emerging as an important but under-addressed topic in the education of future medical provides. Innovative educational approaches targeting learner’s skills and confidence in this field are needed. We developed an innovative outcome based curriculum for advanced clinical skills in lifestyle medicine. The learning outcomes were to conduct a comprehensive lifestyle history while displaying respect and confidence.
METHODS: Two sessions were delivered to medical students in their first clinical year during the Medicine Clerkship. Session-1 (30 min) was a self-learning online case base module. Session-2 (1.5h) was delivered face-to-face by two expert educators (a clinician and a psychologist who modeled the appropriate skills and attitude), and utilized a live role-play for practice of the targeted skills with immediate faculty feedback. The online module was evaluated by learners using three questions. Learner’s self-perceived confidence was assessed using a 5-point Likert scale. Communication and history taking skills were assessed by an Observed Standardized Clinical Encounter station (OSCE).
RESULTS: Curriculum was delivered 7 times (total 150 learners) and was enhanced with each iteration based on peer and/or learner feedback. During the OSCE, learners performed the targeted skills and displayed respect above the minimally required level. Learners reported increased confidence in practicing lifestyle medicine. Most learners (95%) agreed that the on-line module was a valuable, easy to navigate tool and appreciated the realistic succinct examples/situations; however, 10% wanted to see more visuals and 20% wanted a module with management recommendations (which was to follow in the curriculum).
CONCLUSIONS: This active learning approach combining a face-to-face and an on-line session resulted in ability of conducting a comprehensive lifestyle history, while displaying respect and confidence. Other institutions looking to integrate innovative active lifestyle medicine sessions and/or advanced clinical skills into their curriculum could use the same approach.
Session 6 – Student Support
1 – Financial Education Among Medical Students: An Evaluation of Financial Preparedness Using a California Paradigm
Emily Van Kouwenberg
PURPOSE: Medical education is associated with significant debt burden. The goal of this study was to explore the financial preparedness of trainee medical professionals by examining medical students in California as a paradigm.
METHODS: A cross-sectional study was designed to assess medical student financial preparedness using a 29-item electronic questionnaire administered between November 2010 and February 2011. The survey was sent to MD and MD/PhD students at participating allopathic California medical schools.
RESULTS: Among the participating schools, the response rate was 422 of 2026 students (20.8%). 140 participants (33.8%) reported receiving some form of financial education during undergraduate college, and 113 (27.3%) during medical school. 243 students (56.8%) first obtained financial information from a family member, 12 (2.9%) from friends, 35 (8.5%) from a financial advisor, 72 (17.4%) on their own, and 14 (3.4%) responded “other;” 46 students (11.1%) reported having no financial education. 253 participants (61.1%) responded that undergraduate college is the best time to introduce financial education, while 112 (27.1%) indicated that medical school is the most appropriate time. Projected student estimates of total educational debt upon medical school graduation varied, with $200,000-$249,000 being the most common range (107 students, 26%). Of all survey respondents, 312 (75.9%) responded that they felt they were in control of their finances.
CONCLUSION: Most medical students do not receive adequate financial education. Given the large sums of projected debt accrued during undergraduate and medical school, most medical students felt that financial education should be introduced early in their curriculum, optimally before medical school.
2 – Being Well at the University of Louisville: A Comprehensive Wellness Initiative Promotes Health, Resilience, and Compassion in Medical Students
M. Ann Shaw
PURPOSE: Professional burnout can negatively affect provider wellness and patient care. The University of Louisville (ULSOM) developed Being Well, a multifaceted initiative promoting wellness at the organizational level. Twenty-five diverse, interdisciplinary activities were developed and offered for free to students, residents, faculty, and staff—including several evidence-supported wellness training programs. This study reports preliminary student wellness outcomes after initiating Being Well.
METHODS: Participation was tracked, and ULSOM students’ 2015/2016 AAMC Medical School Year Two Questionnaire (Y2Q) were compared pre/post. Additionally, a comprehensive wellness inventory was administered that included self-report subscales measuring: resilience, health, compassion, depression, life satisfaction, happiness, work quality, and burnout. All currently enrolled students (M1-M4) were invited to complete the survey in July 2017. This study has IRB approval.
RESULTS: Being Well student participation varied across programming (Mind Body Medicine: n=12, Koru: n=220, Compassion Cultivation: n=45) and yoga/tai-chi classes (n=39 each). 2015/2016 Y2Q data showed improvements for quality of life (m=39.9/41.3—higher quality), perceived stress (m=5.9/5.5—lower stress), and burnout (m=9.9/9.3—lower disengagement; m=12.4/11.8—lower exhaustion). Wellness inventory respondents (n=386) reported high health (m=14.6/16), resilience (m=22/30), and compassion (m=115.4/140). Rising M2 wellness resembled incoming M1s on most subscales, although M2s reported higher levels of nonclinical burnout (t=4.0, <0.001). Rising M3s/M4s also reported higher levels of depression than pre-clerkship students (t=3.5, <0.001).
CONCLUSION: Being Well initiatives initially targeted medical students, but expanding these efforts to multiple levels across the continuum contributed to culture change supporting wellness at the organizational level. Baseline outcomes parallel wellness trends in medicine (e.g., heightened stress during clinical rotations). Paired, longitudinal data will track student (along with faculty, staff, and resident) outcomes to understand whether Being Well participation improves wellness. Our experience developing Being Well provides insight, like the importance of interdisciplinary collaboration and variable programming, for other academic medical centers to initiate similar programs.
3 – Assessing Metacognitive Regulation Skills in Students in a Professional Program
Cortny Williams
PURPOSE: Students with advanced metacognitive regulation skills can select and modify learning strategies based on outcome. These skills can be further applied in their profession as the graduate is expected to be an expert who can evaluate diagnostic analysis and treatment options to distinguish bias from the physician and patient perspective. Here we will survey the level of metacognitive regulation skills that chiropractic students possess entering the program. The results of the surveys will be used to guide effective teaching strategies that develop metacognitive regulation skills.
METHODS: Students are surveyed twice in a 10 week term, after their first exam, and just prior to taking final exams. The first survey assesses student monitoring and evaluating of exam one strategies, and planning for subsequent exam study strategies. The second survey assesses the extent to which students followed their study plan from the first survey, and calculates difference in time investment in exam preparation between the surveys.
RESULTS: Two cohorts of students have been offered the surveys. Data collection ends in December 2017. Preliminary results indicate that 45% of students initially reporting that they study over a 24-48 hour period increased their frequency of study. Further, 65% of students planned to change study habits and identified new study skills to try. 75% of these students self-reported that they incorporated new study habits, while 23% self-reported that they did not have time or resources to use their new study plan.
CONCLUSIONS: Students that participated in the survey were exposed to new study strategies, asked to modify study strategies based on outcome. Teaching strategies that model elements of the survey, enforcing good study habits and encouraging frequent use of metacognitive regulation, can increase the development of metacognitive skills.
Session 7 – Technology & Innovation
1 – The Virtual Microscopy Database – Sharing Digital Microscope Images for Research and Education
Michael Hortsch
PURPOSE: Virtual microscopy has become the predominant modus of teaching the structural organization of tissues and organs, replacing light microscopes and glass tissue slides that were used in a traditional histology or pathology laboratory setting. Although virtual microscopy image files can easily be duplicated, creating them requires quality histological glass slides and expensive microscope equipment that are not always available, especially at new institutions in developing countries. This has set limits to the generation and distribution of virtual image files at a wider scale.
METHODS: The Virtual Microscopy Database (VMD) is a new Internet-based resource to make virtual histology and pathology image files available to educators worldwide (http://www.virtualmicroscopydatabase.org). It is a virtual image file-sharing website that allows easy access to a large number of virtual images that have been uploaded by other VMD users. With support from the American Association of Anatomists and MBF Bioscience, VMD registration and use of the VMD are free of charge. However, the VMD site is restricted to faculty and staff of research and educational institutions, who need to register.
RESULTS: Currently, the VMD site offers >2600 virtual image files. The available files are organized into 15 collections originating from different universities. The majority of images are either light or transmission electron micrographs of normal cells and tissues. A number of pathological and immuno-stained specimens are also available and new images and collections are constantly being added. Currently, the VMD has over 500 registered users from 56 different countries.
CONCLUSION: The Virtual Microscopy Database is a new, Internet-based resource for histology and pathology educators and researchers. It allows for the sharing of virtual microscopy image files. The VMD resource will help to elevate the quality of histology and pathology instruction and make these subjects more accessible to students at educational institutions worldwide.
2 – Knowledge and Perceptions of Ultrasound in a Clinical Anatomy Course for Allied Healthcare Graduate Students
Saskia D. Richter
PURPOSE: Similar to medical education, allied healthcare (AHC) programs are recognizing the importance of integrating diagnostic ultrasound (US) into curricula, including anatomy, to develop practitioners capable of using US at the point-of-care for improved patient outcomes. This study examined AHC program differences in knowledge and perceptions following initial exposure to musculoskeletal ultrasound (MSK-US).
METHODS: MSK-US was introduced to graduate AHC students (athletic training [AT,n=11], occupational therapy [OT,n=54], and physical therapy [PT,n=63]) during the upper extremity module of an anatomy course. MSK-US basics were presented via online presentation before a one-hour hands-on MSK-US session where students imaged structures in small groups following faculty demonstration. Students were assessed on knowledge (structure identification), application (identify structure on US using corresponding cadaver), and perceptions via online and lab practical assessments utilizing multiple-choice, matching, and Likert-scale items. Kruskal-Wallis and one-way ANOVA, with pairwise comparison with Bonferroni correction, were used to assess performance and perceptions between groups. Statistical significance was set at p=0.05, 2-tailed.
RESULTS: No difference between groups existed in structure identification (p=0.47), application (p=0.42-0.63), or perceptions (p>0.05). Students successfully identified (3.45±1.2 out of 4) and applied (1.72±0.56 out of 2) anatomy structures on US images. Students from all programs agreed or strongly agreed MSK-US: 1) instrumentation is easy to use (85.9%), 2) improves cross-sectional anatomy understanding (84.2%), 3) should be included in anatomy (59.1%), 4) will likely be used in clinical practice (60.6%).
CONCLUSIONS: By presenting MSK-US in a precise and limited manner during first exposure, students were successful and reported having a positive learning experience. Similar to literature in medical programs, a variety of students benefit from, and enjoy, the integration of MSK-US in anatomy. Further, AHC students perceive MSK-US will be utilized in future clinical practice. Faculty should consider a phased approach when integrating MSK-US into AHC curricula.
3 – Innovative use of audience response system to assess teaching methods and implement rapid change
Mari Hopper
PURPOSE: Although audience response systems (ARS) are frequently used to formatively assess student learning, this technology is rarely employed to formatively assess teaching. Aim of study was implementation of novel approach using ARS to assess teaching methods in order to enact rapid improvement.
METHODS: Second-year students of large multi-center medical school participated anonymously. Study involved nine sequential, three-hour pulmonary class sessions delivered live on one campus and livestreamed to others. Following each session, ARS delivered three Likert scale and one open-ended question: 1)What degree have you utilized higher order skills; 2)Rate level of engagement; 3)Estimate percentage of time focused; and 4)What went well/suggestions for improvement. Following each session, course leaders considered student input and adjustments made before delivery of next session. Faculty also quantified parameters including number of cases, key concepts, videos, and slides per session and “Bloomed” embedded questions.
RESULTS: Approximately 60% of students responded to ARS questions. Results indicated consistent improvement from session one to nine in all parameters: number of students perceiving very high use of higher order skills increased from 10% to 30%, students rating the session as highly engaging rose from 55% initially to 80% for final session, and ability to remain focused for more than 70% of the class time rose from 55% of students to over 75%. Comparison of student perceptions to objective measures is underway.
CONCLUSION: Innovative ARS use allowed faculty to immediately gather feedback from diverse students at distributed locations. Feedback guided change leading to improved student engagement, focus, and utilization of higher order skills. Students were also highly appreciative of opportunity to impact change in the class while participating.