Session 1 – Assessment
1 – Assessing Undergraduate Medical Education Competencies in Obesity Medicine: Observed Structured Clinical Encounter
Magdalena Pasarica
PURPOSE: There is a gap in resources for assessing obesity medicine education. We developed an obesity Observed Structured Clinical Encounter (OSCE) station to assess the medical education competencies and Entrustable Professional Activities (EPAs) that align with obesity medicine.
METHODS:Â Â We deployed the OSCE in the first clinical year in the medical curriculum. The educational objectives were: 1) to develop recommendations for the management of obesity; 2) to obtain relevant patient information in a sensitive manner from patients with obesity; 3) to make confident, informed and evidence-based recommendations for the management of obesity. The OSCE was content validated by clinical content experts. A modified Angoff method was used to establish the cut-off score for minimal competency. Following the educational session, but before the assessment, learners rated their level of confidence in obtaining a lifestyle medicine history.
RESULTS: The minimally competent level was set at 74.5% of the maximum averaged points for the Standardized patient checklist. Only 5 out of 116 leaners did not pass. The overall OSCE score was 82.5+/-6.0 (Mean+/-SD). After the educational activities, the majority of students were confident in their ability to manage patients with obesity, ranging from slightly confident (3%), to confident (21%), to very confident (50%), to strongly confident (24%). The overall OSCE score, as well as the scores for the individual components of the OSCE were NOT significantly correlated (p=NS) with the learners perception of confidence.
CONCLUSIONS: Students were able to perform well on the Obesity OSCE station, despite a wide range of confidence levels, suggesting a misalignment between confidence, knowledge and skills for some students. Future studies should focus on increasing alignment of learnersâ knowledge, skills and confidence in obesity management with real patients. Other institutions looking to evaluate their obesity management curriculum could use this obesity OSCE station.
Oral Award Nominee:
2 – Medical Student Equanimity: Staying Cool Under Pressure but Maintaining Empathy Towards Others
Robert Treat
PURPOSE: Equanimity is a state of composure that results from an attitude of acceptance toward stressful conditions. It is a critical element of resilience that is often necessary for medical students to meet the challenges of medical school. However, the capacity to adapt quickly to stressful academic challenges may require an emotional detachment that undermines interpersonal behaviors such as empathy.
The purpose of this study is to analyze the relationship of medical student empathy and perceived stress as mediated by equanimity.
METHODS: In spring 2017, 115/500 M1/M2 medical students completed these self-reported surveys: RS-25 Resilience Scale to measure five-item factor of equanimity (scale: 1=strongly disagree/7=strongly agree); Interpersonal Reactivity Index to measure seven-item empathy (1=doesnât describe me well/5=describes me well); 10-item Perceived Stress Inventory (1=never/5=very often). Pearson correlations and stepwise multivariate linear regressions used for predicting empathy from equanimity and perceived stress. Inter-item reliability via Cronbach alpha. IBMÂź SPSSÂź 24.0 generated statistical analysis. This research approved by the institutionâs IRB.
RESULTS: Equanimity sample scores had a mean (sd) = 28.5 (3.9) which was above the instrumentâs midline score of 20 (range=5-35). Empathy mean (sd) = 26.2 (4.3) was above midline score=21 (range=7-35). Perceived stress mean (sd) = 25.4 (5.5) was below midline score=30 (range=10-50).
Equanimity (alpha=0.68) was significantly correlated to empathy (r=0.4/<.001) and perceived stress (r= -0.5/<.001). Empathy (alpha=0.81) was significantly correlated to perceived stress (alpha=0.85, r= -0.3/<.007).
Three-stage linear regression results for equanimity mediation of perceived stress and empathy: (1) empathy on perceived stress (RÂČ=0.11/<.001/beta= -0.3), (2) empathy on equanimity (RÂČ=0.16/<.001/beta=0.4), and (3) empathy on equanimity and perceived stress (RÂČ=0.29/<.001/betas=0.5 and 0.0, respectively).
CONCLUSIONS: Medical student equanimity mediated the adverse effects of perceived stress on empathy. Students are well-equipped to interact calmly and empathically with patients in the clinical settings during their final years of medical school.
Oral Award Nominee:
3- Should Selectivity Indicies of Undergraduate Institutions be used as a Criteria for Medical School Admissions?
Scott Helf
PURPOSE:Â Medical school admissions committees often use selectivity of applicantsâ premedical colleges as a competitive admissions factor. Barronâs Profiles of American Colleges publishes an annual selectivity index, from 0 to 9, ranking pre-medical schools from not ranked (0), non-competitive (1), to most competitive (9), respectively. Does Barronâs selectivity index predict medical student success? And, if the answer is âno,â should it no longer be considered for admission into medical school?
METHODS:Â 2,746 matriculant records for 11 admissions cycles for the classes of 2008 to 2018 were grouped into their respective selectivity cohorts, per the Barronâs selectivity index of their premedical baccalaureate college. Relative medical school successes were compared, as measured by their preclinical grade point averages (PCGPAs) and performance on national licensing exams (NLEs).
RESULTS:Â T and Mann-Whitney tests were performed, and the significance of P values corrected using the Bonferroni method. When comparing the selectivity cohorts, no statistically significant differences (< 0.05) were observed for PCGPAs, or all three levels of NLEs, including preclinical, clinical, and post-graduate levels.
CONCLUSION:Â When comparing selectivity of matriculantsâ pre-medical colleges as according to Barronâs selectivity index, there is no observable statistical difference in medical school performance between the most and least selective groups. This appears to be the case for PCGPAs and all three levels of NLEs. Our study suggests that Baronâs selectivity index does not reliably predict medical student success. Therefore, it may not be appropriate to consider Baronâs selectivity index as a competitive factor for admission into medical school. For greater generalizability, additional studies of this and other selectivity indices may be warranted.
Oral Award Nominee:
4 – Discriminant Validation of a New Test of Evidence-Based Medicine
Diana Glendinning
PURPOSE: Evidence-based medicine (EBM) is medical practice that uses the best, current evidence to inform decision-making. Medical schools teach EBM to prepare graduates for future clinical practice and lifelong learning. A validated test of EBM has potential use for assessment and research. Existing validated instruments are limited by either content, delivery or grading. We have developed a new test of evidence-based medicine knowledge and skills for use in medical schools. The test was developed collaboratively by researchers at Penn State College of Medicine and Rutgers, Robert Wood Johnson Medical School. It has 17 multiple-choice questions delivered through an internet testing system. The purpose of the current research is to assess the test for discriminant validity.
METHODS: We defined 2 cohorts of student learners. Beginners are medical students before formal EBM training; Intermediates are medical students who have completed a pre-clerkship EBM curriculum. Students were recruited from medical school classes at both institutions. Volunteers were administered the test using REDcap software at designated times and locations.
RESULTS: We have tested 43 students to date. Test scores are significantly different (t=4.015, df=41; P=0.0002) between Beginners (mean 51% ±13) and Intermediates (mean 73% ±16). Mean individual test item difficulty also varied between beginners (mean 52.1%, ±28.3) and intermediates (mean 72.79% ±25%).
CONCLUSION: Preliminary discriminant validity testing of a new test of EBM suggests it is effective for distinguishing between beginner and intermediate groups of medical students. Further testing is planned to include EBM experts. The test has previously been tested for content validity. If further testing demonstrates discriminant validity, the test may be used to easily assess EBM knowledge and skills in medical student populations.
Session 2 – Assessment
1 – Post-Exam Feedback with Question Rationales Improves Re-Test Performance of Medical Students on a Multiple-Choice Exam
Beth Levant
PURPOSE:Â Question response feedback is widely used in formative and summative testing to facilitate learning, particularly with computer-based systems. This study compared the effects of two types of delayed feedback (correct response/correct response + rationale) provided to medical students by a computer-based testing system following a summative exam.
METHODS:Â The preclinical medical curriculum at the University of Kansas Medical Center uses a two-exam system for summative assessments in which students test, revisit material, and then re-test (same content, different questions), with the higher score used to determine the studentsâ grades. Test and re-test scores from the midterm multiple-choice examinations from the Infectious Disease module, which occurs in the Fall of Year 2, were compared between academic year (AY) 15/16, which piloted rationales on 8 of 75 questions, and AY16/17, which had rationales for all questions. The exam items were 93% identical between years.
RESULTS: Exam difficulty and discrimination of the test and re-test were similar in both years. The average increase in score on the re-test was 0.89 ± 6.21% (n=192) without rationales and 4.08 ± 6.60% (n=197) with rationales (<0.000). Rationales also resulted in fewer students with a change in score on the re-test of <0% (<0.01) and more students with an increase of >10% (<0.001). The effect of the exam item rationales was greatest in students with entering composite MCAT scores of 0-23 (<0.01) and 24-29 (<0.001) or Year 1 GPAs in the bottom quartile (<0.001). The exam item rationales were highly rated by students (M=4.22 on a 5-point Likert scale).
CONCLUSION:Â Delayed feedback with exam question rationales resulted in a larger increase in exam score between the test and re-test than feedback with correct response only, suggesting improved learning. This effect was greater in academically weaker students.
2 – Use of a Two-Step Examination Process in a First-Year Medical Anatomy Course: Impact on Student Experience and Performance
Melanie L. Korndorffer
PURPOSE :Â The objective of this study was to analyze and quantify the impact of a two-step high stakes examination process on student experience and performance in a first-year medical anatomy course.
METHODS:Â First-year medical students (n=175) in an Anatomy course completed four individual examinations, each followed by a team examination using identical questions. A paired t-test was used to determine mean differences between individual and team exam scores; a repeated measures ANOVA was used to determine mean differences in performance among different subgroups of students (top 27%, middle 46%, bottom 27%) and teams with different compositions of individual members (more in top 27%, middle 46%; bottom 27%, or even mix). Students rated the value of the team-based exam on block evaluations using a 5-point Likert scale and also provided qualitative feedback. Comments were analyzed and coded based on common themes.
RESULTS:Â Team mean scores exceeded Individual mean scores for all examinations. While all subgroups (top 27%, middle 46%, bottom 27%) showed significant improvement in scores from individual to team exams, students in the bottom 27% experienced the greatest gain. Teams containing a larger proportion of students in the bottom 27% scored lower on team exams than those with higher proportions in the middle and top 27%. All results significant at <.05. More than 90% of students believed that team exams improved understanding of Anatomy content and served as an effective exam review. Qualitative feedback revealed the following themes: value of collaboration and peer teaching to solve problems and increased confidence in knowledge.
CONCLUSION:Â A two-step individual and team-based examination is an effective method to improve student scores on high stakes examinations, increase confidence in knowledge, promote peer learning, and increase student engagement. The team-based exam provides a method for exam review which promotes learning and facilitates knowledge retention.
3 – Assessing the effectiveness of Objectively Structured Clinical Teaching (OSCT) in standardized & real patients in undergraduate clinical training – A pilot study
HMP Singh
PURPOSE:Â To evaluate the effectiveness of Objectively Structured Clinical Teaching (OSCT) in assessing studentâs performance using standardized & real patients.
METHODS:Â This study is a randomized block design involving 58 students from year 5 MBBS program. The students were divided into two groups and were assessed for their performance in Medicine, O&G and Primary care (PCM). Two groups of patients were used during this trial, real patients (RPs) and standardized patients (SPs). SPs were first selected based on their communication and computer skills followed by training in a workshop intended to imitate the presentation of the RPs. The studentsâ assessment was based on their performance in history taking, examination and management. Studentsâ performance was then assessed by subject experts under strict guidelines. The scores of students were compared between SP and RP groups, between three examination stations and between disciplines. The rotational exam results were compared with their OSCT performance. Perception of students, examiners and SPs were also recorded.
RESULTS: The mean scores of students in the RP (62.6±6.3) and SP (59.9±10.9) group were not statistically significant. Studentâs performance in O&G was significantly higher (p value 0.02) in management station compared to Medicine & PCM. Studentâs performance in all other stations among the three disciplines was similar with no significant differences. The correlation between the three stations within each discipline showed no significant results. The comparison of the rotational exam scores (65.3±5.5) and OSCT scores (61.1±8.5) was not statistically significant which indicates concurrent validity of the methodology. The students, examiners and SPs perceived OSCT as a good and reliable teaching and examination tool.
CONCLUSION:Â OSCT using standardized patients is an effective tool for clinical teaching to reduce inconvenience to the real patients and for effective student learning.
4 – Effect of Basic Sciences Grades and Preparatory Courses on USMLE Step 1 Performance Amongst Medical Honor Students in a Caribbean Medical School
Ryan Chen
PURPOSE:Â To examine the relationship between the studentâs weighted mean percentage grade (WMPG) during basic sciences curriculum and the studentâs performance on the United States Medical Licensing Examination (USMLE) Step 1 among high academically achieving medical students. Also, to determine whether taking a commercial preparatory course significantly improves Step 1 score, as published data documenting the effectiveness of such third-party programs are lacking.
METHODS:Â A survey was conducted on four cohorts of honor students at St. Georges School of Medicine, who maintained at least 90% WMPG during the basic sciences (n=326). All those surveyed had recently completed Step 1, and were asked to report their three-digit score along with their WMPG grouped into tertiles. The average Step 1 score was calculated for each group, after which One-way ANOVA, and studentâs T-tests were used to compare between groups. Students were also asked about commercial review courses taken and their perceived usefulness for the purpose of Step 1. The average Step 1 score was compared between these groups as well.
RESULTS: 190 (58.3%) of the students responded to the survey. Overall, the average Step 1 score was 250.0 ± 8.9. Students belonging to a higher WMPG bracket had significantly higher Step 1 scores compared to those in a lower bracket (<0.05). On the other hand, Step 1 performance between the course takers and non-takers were statistically indistinguishable for every course surveyed. The subjectâs perceived usefulness of that course also had no statistically significant effect when compared with ANOVA.
CONCLUSION:Â The studentsâ academic performance during basic sciences years as represented by their WMPG is a strong predictor of Step 1 performance. However, neither participating in a commercial coaching course nor student attitude regarding its usefulness affected the outcome, suggesting the possibility of a metacognitive, self-regulatory pattern in student learning behavior.
Session 3 – Curriculum
Oral Award Nominee:
1 – Aligning Expectations: Comparing Student and Clerkship Director Perspectives about Readiness to Perform the Core Entrustable Professional Activities at the Start of the Clerkship Curriculum
Victor Soukoulis
PURPOSE: The AAMCâs 13 Core Entrustable Professional Activities (EPAs) identify tasks that students should be prepared to do with indirect supervision on day one of residency. Use of EPAs within an assessment program requires ensuring that learners are aware of performance expectations and prepared to use information from assessment to create individualized learning plans to achieve entrustment. Prior studies have demonstrated a gap between studentsâ self-assessment and supervisorsâ ratings of studentsâ abilities. We sought to understand student and supervisor perceptions at the time of transition from the pre-clerkship to the clerkship phase of the curriculum to better prepare teachers and learners for trust-based assessment.
METHODS: Students were surveyed prior to the first clerkship rotation, indicating the level of supervision they felt they needed to perform each EPA and their confidence with this assessment. After the first clerkship block, directors of each of the core clerkships were asked to assess the level of supervision a âtypicalâ third year student needs to perform each EPA at the beginning of the clerkship year. Level of supervision scores designated the need for joint execution, direct supervision, or indirect supervision.
RESULTS: Students (n=75) rated their readiness to perform EPAs with lower levels of supervision than clerkship directors (n=16) did (2.89 vs 2.19; <0.001). For eight EPAs, there was a statistically significant difference in their perspectives. Studentsâ scores related to lower levels of supervision correlated with higher confidence in their self-assessment (<0.0001).
CONCLUSION: The discrepancy between student and clerkship director ratings of the level of supervision required presents a challenge for enacting EPA-based curricula that use trust and shared expectations for performance as a foundation for assessment and decision-making. Addressing this gap will facilitate students assuming authentic roles with graduated autonomy in clinical settings.
Oral Award Nominee:
2 – Introducing a Self-Care Curriculum in Medical School
Alan L Hull
PURPOSE: Physicians and medical students are experiencing burn-out and depression at increasing and startling rates. Examples and outcomes of self-care curriculum programs that help students develop and practice resilience are limited and few studies address student response to these programs. This studyâs aim was to  gauge student perceptions of a required self-care curriculum introduced during the first 10 weeks of our medical school program.
METHODS: A student-led committee planned and implemented a 2.5 hour self-care fair during orientation week for the entering class in 2017 to introduce students to self-care concepts (nutrition, exercise, yoga, mindfulness) followed by seven 2-hour weekly sessions focused on blending underlying physiology, health care systems, and behavioral science principles with developing self-care skills (resilience, learning management, sleep, mindfulness, emotional intelligence, nutrition). Surveys were electronically administered at the end of the program with 2 email reminders. The study was approved by Cleveland Clinic Institutional Review Board.
RESULTS: 26 of 32 students (81%) completed the survey. Average scores ranged from 2.9 to 3.7 on the 21-item 4-point agree/disagree scale asking whether objectives for each session were met. In addition students agreed/strongly agreed that the curriculum fostered better self-care (96%), increased awareness of the power of self-care (96%), increased their understanding of the importance of incorporating physical and mental health activities to prevent burnout (96%) and increased their ability to recognize the signs and symptoms of burnout (88%). Several specific suggestions for adding or changing content to the curriculum were identified. We are planning a student âchallengeâ during the school year to reinforce self-care skills.
CONCLUSION: Students overwhelmingly supported a self-care curriculum that emphasized knowledge and skills early in the first year of medical school. This study supports the development of further activities to reinforce the knowledge and behaviors necessary to improve and retain resilience.
Oral Award Nominee:
3 – eQuality at the University of Louisville: An Integrated LGBTQ Health Curriculum to Improve Physician Competency
Leslee J. Martin
PURPOSE: Lesbian, gay, bisexual, transgender, and queer (LGBTQ) individuals experience bias in health care. Provider training to address disparities is inadequate, contributing to unmet medical needs. eQuality, the University of Louisvilleâs integrated LGBTQ health education intervention, was launched to address calls from various medical associations to mitigate these gaps. eQuality aims to drive educational and systems innovation through a robust LGBTQ health curriculum that assesses student attitude, knowledge, and clinical skills to ultimately improve the competency of the physician workforce and enhance LGBTQ patient care.
METHODS:Â eQuality was implemented in 2015-16 with 50.5 hours of new or modified LGBTQ health- related curriculum that combines health systems and basic science content and is required for all students. Content spans MS1-MS4 years and includes: institutional climate reviews, faculty development programs, unconscious bias trainings, clinical skills assessment with transgender/genderqueer standardized patients, healthcare community engagement through continuing medical education initiatives, and extensive participation from local LGBTQ community members.
RESULTS: Outcomes are extensive, including: LGBTQ-specific student curriculum, climate improvement initiatives, student implicit attitude debriefing, and student LGBTQ health knowledge, attitude, and clinical skills assessment. Knowledge scores increased more for the intervention group (M=7.9) versus for control (M=2.6, <.001). There were some student attitude improvements, such as more favorable attitudes toward inclusive sexual histories for intervention versus control (F=22.3, <.001). Demonstrated knowledge transfer to clinical skills included 63.5% of students knowledgeably discussing hormone use with transgender/genderqueer standardized patients.
CONCLUSION:Â eQuality contributes broadly to provider LGBTQ health competency. Assessment breadth and incorporation of LGBTQ community members strengthen the approach of this innovation. Ongoing challenges include engaging clinical leadership to inform physician role modeling. This presentation highlights the diverse, encouraging outcomes of integrated LGBTQ health competency curriculum to serve as a template for other academic programs considering similar programs, and published eQuality resources are available to aid these efforts.
Oral Award Nominee:
4 – Reconnecting the Mouth to Medical Education
Richard C. Vari
PURPOSE: The absence of oral health education in medical education has resulted in physicians who are poorly equipped to diagnosis, treat, refer and collaborate on matters of oral health and disease. This poster describes the successful integration of an oral health curriculum, with measurable outcomes, into the medical curriculum.
METHODS: In the first two years of the curriculum all medical students are now required to engage in thirty-six hours of oral health education which includes a lecture presented by an international expert on oral health, labs on anatomy/oral health, PBL patients with dental diseases, lectures on oral manifestations of systemic disease, oral cancer screening, and inclusion of the oral exam into the HEENT exam. In year 3 oral health is a component of the Family Medicine Clerkship.  Assessments include a HEENOT (O=oral) exam with standardized patients, a written exam in year 2, student satisfaction of the oral health week, and a retrospective chart review comparing documentation of oral components  within student (n=100) and attending (n=100) HEENT exams.
RESULTS: The average score on the oral written exam has been 89%. Qualitative findings from dental week evaluations have been extremely positive. The retrospective chart review revealed that students were more likely to include the documentation of periodontal tissue (students 12%, attendings 9%), dentition (students 46%, attendings 15%), oral cancer screening (students 49%, attendings 32%) and mucous membranes (students 60%, attendings 35%) within their HEENT examinations.
CONCLUSION: In a partnership with Delta Dental, who provided financial resources for the curriculum, VTCSOM has completed 5 years of educating and training future physicians on the importance of oral health.
Session 4 – Curriculum
1 – Radiology in a Preclinical Medical School Curriculum: Students’ Acceptance of a New Interactive Teaching Format
Alexander Boscia
PURPOSE: Radiology is generally not well represented in preclinical medical school curricula. Aligning the teaching of basic radiology with that of anatomy has, however, been shown to improve medical studentsâ knowledge of both topics. Given curricular time constraints, the most effective teaching format for radiology in the preclinical setting is still unclear. This prospective study aims to assess the acceptance of interactive radiology small-group teaching sessions (IRSGS) among 1st-year medical students and to analyze preferred teaching/learning methods for radiology.
METHODS: An IRSGS aligned with the 1st-year anatomy curriculum was introduced in year one along with traditional lectures. IRSGS was facilitated by 11 Radiology residents and senior faculty. Students worked with a DICOM viewer previously downloaded to their personal laptops and identified normal anatomy on selected radiographic images. A survey of 45 questions was conducted among 184 1stâyear students addressing preferred teaching/learning methods and acceptance of IRSGS including a self-assessment of radiology knowledge.
RESULTS: 106 students completed the survey (58%). While 25% of the students found Radiology interesting, 92% considered Radiology an important discipline; 98% stated it should be taught in a preclinical curriculum. Per self-assessment, 62% felt (very) confident to identify normal anatomy in a chest radiograph and 27% in cross sectional imaging but were much less confident in identifying pathology (29% and 6%).
Internally distributed support material and Radiology Websites (radiopaedia [33%], Wikipedia [5%]) tied as the most preferred resource for learning radiology (37%); 29% of students used lectures as a substantial resource. The most-preferred method of teaching radiology was expert facilitating small-group discussions (77%)
CONCLUSION: Radiology should be integrated into a preclinical curriculum preferably in alignment with anatomy. Small group interactive Radiology sessions are well-accepted by preclinical medical students while lecture-based teaching of radiology is less appreciated.
2 – A Novel Approach to Global Health Education: Teaching Applicable Skills and Capacity Building in Medical School
Amy Miner
PURPOSE:Â As interest in global health (GH) education continues to grow, significant gaps have become apparent; paucity of relevant and applicable skills training, deficiency of multidisciplinary involvement, insufficiency in capacity building and inability to accommodate scheduling demands have all been noted. We addressed these gaps by developing a multidisciplinary course using evidenced-based, applicable hands-on training which incorporated flexible scheduling, training in capacity building and opportunity for in-country application. This abstract describes the curriculum.
METHODS:Â The course is a hybrid of online learning and hands-on skills training relevant to current GH needs. Students and instructors are recruited from multiple disciplines: Nursing, Physician Assistant and Medicine. Foundational knowledge is gained through USAID online modules. Hands-on skills training consists of Helping Babies Breathe and Helping Mothers Survive workshops which teach basic lifesaving skills in maternal and newborn global health, prepare students to teach these skills, and provide fundamental principles of capacity building. Following course completion, students are encouraged to assist in teaching the course the following semester and then teach in resource-limited countries.
RESULTS:Â 33 students have completed the course, 20 of whom have taken the training to the countries of Nepal, India, Guatemala and the Dominican Republic. Course graduates praise the flexible schedule, multidisciplinary approach, applicability of course content and opportunity for continued learning afforded by the course.
CONCLUSION:Â The relevant and applicable content presented in this multidisciplinary course provides an opportunity to move beyond a traditional global health model of simply providing care, to a global health experience which affects lasting change in resource-limited settings.
3 – Educating Medical Students to Become Great Teachers: A Longitudinal MS4 Medical Student as Teacher Elective (MST)
Elisabeth Schlegel
PURPOSE. Medical education bodies worldwide consider teaching a physician core skill. Fostering workplace learning, we introduce a longitudinal medical student as teacher (MST) elective combining didactic foundations of educational theories, strategies, and curriculum design with expectations to teach within different courses of the medical curriculum during the 4th undergraduate year. The purpose of this innovation is offering a comprehensive program, reinforced with teaching experience, with the intent to provide opportunities preparing students for roles in medical education and teaching excellence during residency.
METHODS. Commencing in the 2017/18 academic year after a 2-year trial of a 2-week block elective rotation, educational boot camps (22 hours), experiential learning as participating educators (30 hours), independent study (15 hours), and a capstone project will prepare MS4 students to master pedagogies across a variety of instructional settings. A visual timeline, reflective online teaching logs, faculty debriefings and mentoring, and an educational project enable students to successfully complete the elective.
RESULTS. The inaugural five MS4 students co-facilitating a variety of SOM sessions have been commended as near peer facilitators by faculty and students. The self-directed learning approach is compatible with MS4 responsibilities including acting internships and residency interviews allowing sufficient time to acquire teaching skills, but competing responsibilities challenge coordinating MS4 teaching opportunities. By March 2018 data will be available from semi-structured exit interviews evaluating program structure and learning objectives, and from individual learning assessments of reflective teaching logs, debriefings, journal club presentations, and completion of their final capstone project including a presentation.
CONCLUSION. The MST program providing educational content and structure, reflection, and diverse teaching opportunities can easily be adopted by medical schools with elective time and faculty resources. Lessons learned include operating a flexible timeline and enhanced communication of upcoming teaching opportunities for accommodating individual solutions. Improvements will be reported based on student feedback.
4 – Continuous Professional Development Course: A Way to Enhance Medical Studentsâ Progression on Competency Goals
Leslie Ruffalo
PURPOSE:Â Professional development is central to studentsâ success through medical school, yet there is no generally accepted model to best integrate these aspects in the medical school curricula. In response, our institution implemented the Continuous Professional Development (CPD) course. The CPD course spans the third and fourth year of medical school with the goal of ensuring studentsâ continued professional development towards meeting our schoolâs global competencies for medical students.
METHODS:Â Each student is assigned to a CPD director who monitors and tracks the studentâs progression along our school’s global competencies, rotation evaluations, and progress on a student completed self-assessment tool. The objective of this study was to evaluate medical studentsâ experiences and perceptions of the CPD course since its inception three years ago. Medical students complete a survey at the end of the third and fourth year that serves as the CPD courseâs summative evaluation. The survey measures studentsâ satisfaction on multiple CPD components. We used statistical software to analyze quantitative data. We analyzed open-ended questions using content analysis techniques.
RESULTS:Â Five hundred and seventy four medical students have completed the CPD course over three years. Overall satisfaction with the course was rated at 3.77 (SD=0.77) on a five point Likert scale (1=Unsatisfactory and 5=Outstanding). Students rated course components (cumulatively) as highly acceptable (mean=3.77, SD=0.84). Â Students valued individual meetings with CPD Directors (mean=3.97, SD= 0.93). Students also highly regarded their CPD Director (mean=4.27; SD=0.86).
CONCLUSION:Â Professional development is an important component of medical education and requires the use of innovative strategies to integrate into the curriculum. Our CPD course provides students with one-on-one mentoring and feedback on competency progression and individualized advice on meeting their career goals. Ongoing assessments are needed to improve the quality and value of such courses in medical education.
Session 5 – Instructional Methods
1 – A Retrospective Study of Interactive Pharmacology Instruction: Did the Flip Flop?
Emmanuel M. Ngu
PURPOSE:Â Recent trends in medical education indicate that a more self-directed learning approach for assimilating foundational basic science content is eminent. The primary objective of this study is to evaluate the effectiveness of interactive changes made to foundational pharmacology instruction over a four-year period. This abstract describes the impact of flipping the pharmacology classroom on student learning.
METHODS:Â The impact of increased active learning strategies on medical student performance in a foundational pharmacology course was examined over four consecutive years. Dual approaches for content transfer were examined: 1) a more traditional lecture-based format, with active learning and 2) a case-based, flipped classroom design to encourage the use of condensed, prerecorded lectures, and assigned readings for in-class application. Analysis of student weekly examination scores was performed. In addition, end-of-module and professor evaluations for each system-based section were used as additional measure of impact.
RESULTS:Â The more traditional, lecture-based instruction in foundational pharmacology was associated with higher student performance on the weekly examinations. Furthermore, online student evaluations of pharmacology professors teaching in each system-based module were higher with the more traditional approach. In addition, overall end-of-module evaluations were lower with the case-based, flipped classroom design.
CONCLUSION:Â âFlippingâ the pharmacology classroom, resulted in decreased learning, as measured by examination performance. The end-of-module and professor evaluations were highly correlated with this finding. Overall, it appears that, based on the findings of this study, a less self-directed learning approach to foundational pharmacology facilitates greater student satisfaction and more effective learning.
Oral Award Nominee:
2 – Effectiveness of Simulation-Based Multidisciplinary Teamwork Training of Physical Restraint in Psychiatric wards
Chih-Wei Yang
Purpose: Physical restraint is inevitable in some situations for psychiatric emergencies. During the procedure, optimal multidisciplinary teamwork is crucial for preventing complications for both the staff and patients. We developed and evaluated the effectiveness of a simulation-based multidisciplinary teamwork training of physical restraint in psychiatric wards.
Methods: This is a before-and-after study design. Medical staffs, including residents, registered nurses, and security guards, in psychiatric wards of National Taiwan University Hospital were all invited. Trainees, grouped of 5, were asked to deal with a standardized simulated psychiatric physical violence which required physical restraint. Each session lasted 10 minutes, followed by video-assisted debriefing. Assessments, including 10-point-scale self-efficacy of management (Score M) and teamwork (Score T), for each trainee were used before and after each simulation session. Changes of the scores were further analyzed using linear logistic regression to compare the learning outcomes of trainees of different wards, experiences of critical care, and disciplines (residents vs. registered nurses vs. security guards).
Results: During August 2016, totally 35 medical staffs (7 groups), including 5 residents, 24 registered nurses, and 6 security guards, were recruited. Assessments for each trainee improved significantly after simulation-based training (Score M 6.71 to 8.71 (p < 0.001), and Score T 6.6 to 8.6 (p < 0.001)). Linear logistic regression analysis revealed that self-efficacy of medical staffs without previous critical care experiences and security guards improved significantly more than others.
Conclusions: Simulation-based multidisciplinary teamwork training of physical restraint in psychiatric wards is effective and highly recommended in psychiatric wards.
3 – Evaluating Medical Student Engagement During an Innovative “Immune Response Simulation” Session Targeted to Enhance Understanding of Abstract Immunological Principles
Nyla Dil
PURPOSE: Undergraduate medical education is moving towards engaging students in active learning as opposed to heavily relying on traditional lecture based pedagogy. Immunology is a complex discipline to comprehend and most entering medical students have little to no prior knowledge of this intricate discipline. We developed a novel hands-on âimmune response simulationâ session to engage students in active learning of immunological principles and improve their integration with medical microbiology. We are developing tools to gauge, monitor and access student engagement during this novel active learning session. This abstract describes this medical education project and its outcomes.
METHODS: Students are taught didactic immunology lectures followed by this active learning session where groups of students utilize their immunology knowledge to stimulate an immune response scenario to five major classes of pathogens. Students prepare these 10 min long enactments playing roles of a host immune cell or a pathogen. Faculty facilitates this learning session and administers the assessment tools. The session outcomes are evaluated by administering pre- and post-immunology tests. Three integrated aspects of student engagements: cognitive, behavioral, and affective are measured.
RESULTS: Preliminary results indicate that there is a significant increase in immunology knowledge gained and that students were highly engaged in learning immunology through this innovative pedagogy.
CONCLUSION: Immune response simulation is an effective pedagogical novelty where students are actively engaged in learning complex immunological principles in fun, hands on experiential learning session. This novel pedagogical intervention might prove valuable in engaging students in other medical science disciplines as well.
Session 6 – Instructional Methods
1 – Integration of Simulation Early in Undergraduate Medical Education Enhances Student Critical Thinking
Mary O. Dereski
PURPOSE:Â The aim of this study was to assess medical student understanding of asthma pathophysiology, patient presentation, and possible treatment modalities through engagement in a clinical simulation (SIM) activity. Asthma exacerbation was selected to provide an opportunity for participation in an emergency situation within a high yield/low risk environment. This study also gauged studentsâ perception of their preparedness for engaging in an emergent patient situation and the value of that participation.
METHODS:Â Oakland University William Beaumont School of Medicine faculty worked collaboratively with the School of Nursing to develop an acute asthma exacerbation SIM scenario. Two classes (2016 and 2017) of first year medical students (n=244) participated in the SIM session. Tests of asthma related knowledge were administered pre-SIM activity and post-SIM debriefing along with an optional student satisfaction survey.
RESULTS:Â A paired t-test was run on pre- and post-tests of asthma knowledge (2016 n=127, 2017 n=117). The scores increased significantly (<.001) for both years. Students who responded (70%) to the scaled satisfaction survey strongly agreed (SA) or agreed (A) that the SIM activity integrated background knowledge, fostering critical thinking skills (>93%) and increased self-confidence to encounter an acute asthma situation (>80%). Students thought that prior respiratory sessions adequately prepared them to engage in the SIM (>68% SA/A), and the debriefing facilitated their understanding of the clinical scenario (>90% SA/A). Greater than 95% (SA/A) thought the experience was well worth the time they invested.
CONCLUSION:Â A disease-based clinical SIM scenario involving first year medical students is effective in supporting basic science principles while bridging to potential treatment modalities and clinical outcomes. Engaging first year medical students in a SIM is valuable from a knowledge acquisition and clinical preparedness standpoint. SIM integration into the first year medical student respiratory course was a valuable educational engagement strategy for first year medical students.
2 – Students’ Perceptions of the Physician’s Role in Patients’ Behavior Change?
Denise Kay
PURPOSE:Â Future physicians receive minimal training in human behavior or motivation. Medical students may question whether influencing a patientâs choices or behaviors is even the physicianâs role. We explored undergraduate medical studentâs perceptions of the usefulness of Prochaskaâs Transtheoretical Model (TTM) and Miller/Rollnickâs Motivational Interviewing (MI) to their future practice.
METHODS:Â Students completed an online module in the internal/family medicine clerkships that: (1) reviewed TTM and demonstrated, through video, the relational dynamics of MI, (2) Assessed studentâs understanding of TTM and their ability to apply the MI approach, (3) Captured studentâs beliefs about the usefulness of TTM and MI in their future practice.
RESULTS: 57 students completed the module. 97% endorsed the statement âI believe these tools are applicable in health care settings and/or will be useful to me as a physicianâ.  While two students agreed the tools were applicable in health care settings, they were not sure how theyâd be useful to them as physicians. One student did not see the tools as relevant to patient-doctor interactions or how theyâd be of use in their practice.
48 students provided a defense for their selected response about the usefulness of TTM and MI in their future practice. Themes identified in their responses included: (a) beliefs about the physicianâs role, (b) the observed or proven effectiveness of the TTM and/or MI, (c) the doctor-patient relationship, and (d) the potential of these tools as pathways to patient change.
CONCLUSION:Â In health care, expertise about chronic disease is useless if it canât eventually translate to sustainable changes in patient behavior. When provided with a model for conceptualizing patientâs readiness for change and a tool by which to interact with patients about change, the majority of students saw TTM and MI as applicable to health care and useful for physicians.
3 – Cloud-Based Learning: Interactive Team-Based Learning Instruction using Google Drive
Anne M. Dersch
PURPOSE:Â There is a need for instructional methods that require little preparation and utilize constructivist principles to promote directed-independent learning, interpersonal skills, and higher-order cognitive skill levels. Here we describe and evaluate a novel instructional method using a cloud based document-editing application: Cloud-based learning (CBL).
METHODS:Â CBL consists of a didactic lecture reviewing foundational concepts, a team activity where students collaborate to create a study tool document using Google Drive while professors provide real-time feedback, and an interactive lecture using iclickers and student-created content. We used CBL to integrate microbiology and immune-pathomechanisms of respiratory infections for first-year medical students. CBL evaluation consisted of formative assessment and a survey to compare CBL with Team-based learning (TBL), flipped class (FC) and didactic lecture (DL). Statistical analysis included ANOVA and paired t-test.
RESULTS:Â CBL provides a high average (92.1%) level of student satisfaction across six question items. The average CBL collaboration scale was significantly higher than DL (P=0.0002) and FC (<0.0001). The average CBL knowledge gained (7.2; P=0.0003) and classroom directed-independent learning (6.8; P=0.0001) scales were significantly higher when compared to FC. CBL provided significantly higher instructor feedback (67%) than TBL (29%; P=0.0023), DL (12.9%; <0.0001), and FC (6.5%; <0.0001). The preparation time to study for CBL is similar to DL, but significantly less than FC and TBL. In terms of amount of feedback received, students ranked CBL preferable to DL (1.7 vs 3.5; <0.0001) and FC (1.7 vs 3.2; <0.0001). A paired t-test of the pre- and post-formative assessments showed a significant average mean difference (<0.00001).
CONCLUSIONS:Â We have developed an interactive instructional method that allows medical students to learn content without prior preparation in a good learning environment using Google Drive with instant professor feedback. If successful, CBL may improve medical student outcomes when compared to other instructional methods.
Session 7 – Student Support
1 – Predictors of USMLE Step 1 Outcomes: Charting Successful Study Habits
Ryan Guilbault
PURPOSE: The United States Medical Licensing Examination (USMLE) Step 1 is a test that determines many aspects of medical studentsâ careers in the U.S.A. and Canada. Various studies have reviewed the impact of study patterns and resources on Step 1 scores, yet few have sufficiently investigated the predictive power of one of the most touted Step 1 study methods: practice questions and tests. Thus, the aim of this study was to compare the predictive value of practice questions and tests to other potential predictors of Step 1 scores.
METHODS: An electronic survey regarding Step 1 study habits was sent to Mercer University School of Medicine class of 2019 students following their completion of Step 1. Student self-reported data were then used for correlation and regression analyses on the effects of various study habits on Step 1 scores (n = 49).
RESULTS:Â Our results showed that higher pre-clinical curriculum grades (r = 0.58, p < 0.01), higher National Board for Medical Examiners practice exam scores (r = 0.65, p < 0.01), and greater number of practice questions completed (p = 0.53, p < 0.01) were all correlated with higher Step 1 scores. Regression analysis corroborated these findings and indicated that pre-clinical curriculum grades were the strongest predictor of Step 1 scores; a one-unit increase in pre-clinical letter grades was associated with a 12-point increase in Step 1 scores (b = 1.99, SE = 2.99, p < 0.01).
CONCLUSION:Â Despite student emphasis on practice questions and duration of dedicated Step 1 study time, pre-clinical curriculum grades are the best predictor of Step 1 success. Students, professors, and curriculum directors/deans should emphasize focused study on pre-clinical curriculum content throughout the basic science curriculum to increase the probability of higher scores on the USMLE Step 1.
Oral Award Nominee:
2 – How Emotional Exhaustion, Cynicism, and Personal Accomplishment Impact Personal Perception of Wellbeing
Becky Gates
PURPOSE:Â Because of their impacts on health outcomes and patient care, burnout and wellbeing of healthcare professionals are of nationwide interest. All members of the healthcare team contribute to the education of physicians-in-training; thus, optimizing the wellbeing of each professional group improves the learning environment. The purpose of this study was to determine the relationship between components of the Maslach Burnout Inventory (MBI) â depersonalization/cynicism (DP), emotional exhaustion (EE), and personal accomplishment (PA) â and perception of personal wellbeing. Results may help guide the implementation of role-specific strategies to improve system-wide wellbeing and the student learning experience.
METHODS:Â Medical students, residents, nurses, attending physicians, and advanced care practitioners (ACPs) at the institution were sent an online survey measuring risk of burnout (MBI) and perception of personal well-being (investigator-developed question, WB). Ordinal regression was performed to determine the impacts of DP, EE, and PA on WB.
RESULTS:Â The study sample included 90 medical students, 535 frontline nurses, 396 physicians and ACPs (providers), and 121 residents. DP, EE, and PA explained a significant proportion of variance in WB in providers, residents, and medical students (r2 =0.411, r2 =0.537, r2 =0.419), and a smaller proportion of variance in WB in nurses (r2 =0.285). When compared with DP and PA, EE had the greatest impact on WB in all professional groups. DP significantly impacted WB in residents and nurses but not in other groups. PA significantly impacted WB in all groups but was most pronounced in students and residents.
CONCLUSIONS:Â EE has a consistent influence on perception of wellbeing in all groups, but DP and PA have varying effects depending on group. More research is needed to determine specific factors that influence perceived wellbeing and risk for burnout with sensitivity to role. Additional work is needed to examine the effectiveness of tailored role-specific strategies in promoting wellbeing.
3 – For Students by Students: a Pilot Study of a Self-Assessment Tool: Student-Led Assessment Project 2018 (SLAP18)
Avery Ford
PURPOSE:Â Literature suggests that students who engage with the production and discussion of student-generated content may have increased retention and test performance. In the present study, the relationship between studentsâ use of PeerWise ? an online platform that enables students to author, answer and discuss questions ? and retention, as measured by scores on summative examinations. To date, the effect of student-led formative assessment on academic performance at the graduate level has not been studied in the United States.
METHODS:Â Students from the Medical Health Sciences (MHS) Masterâs program at Touro University Nevada voluntarily participated in this project. As a capstone project, five students from the MHS18 cohort explained the basics of item writing to their peers. Sections of lecture content were assigned to each study group. Every student was advised to write at least two questions per assigned lecture topic and answer a minimum of 20 questions prior to each summative examination. Data were analyzed using JASP and R statistical software.
RESULTS:Â More than 80 % (33 out of 40) of the students in the class authored questions and/or submitted answers to questions. Data analysis revealed that the number of answers submitted was a significant positive predictor of improvement in exam score explaining 16% of variance in exam score change.
CONCLUSION:Â Although the project is still ongoing, the preliminary data corroborate previous findings which indicated that practicing retrieval through formative assessment assists students in improving their scores on summative assessments.
When a bank of questions is not provided to students, they should be encouraged to use the PeerWise platform to author questions and answer the maximum number of questions authored by their classmates in order to be better prepared for summative assessments.
4 – Factors Predicting Successful Performance on USMLE Step 1
Rachel VonLuehrte and Sara Barnett
PURPOSE:Â All medical students must take the USMLE Step 1 exam to graduate and become a physician. We surveyed third year SLU students about the resources they used, their study habits and their Step 1 score to understand if certain instructional methods led to higher scores and or lower stress and anxiety.
METHODS:Â Students at Saint Louis University who have completed their second-year course work and have taken the USMLE Step 1 exam were sent an email to participate in a survey which asked about the resources they used, their study habits, their Step 1 score as well as other well-being questions. The survey was distributed and administered online and the responses were collected as anonymous, coded and analyzed.
RESULTS:Â 53 medical students completed our survey. The mean Step 1 score (235) was significantly higher than the national average Step 1 score from 2016 (228). Over 75% of students used First Aid, Pathoma, and Sketchy Medical Bugs as resources and 50% ranked these three resources within the top ones they used. The only resource that made a significant difference in the average step scores was Firecracker. Overall there was significant positive correlation between the results of NBME and UWorld Practice tests and studentsâ Step 1 Score. Lastly there was a significant positive correlation with the CBSE Practice Exam and studentsâ Step 1 score.
CONCLUSION:Â This survey allowed us to identify resources that will be beneficial to students while studying for Step 1 to better advise future studiers and help them achieve their Step 1 goal score. Better identification of study resources can lead to better study strategies and less stress and anxiety overall.
Session 8 – Technology & Innovation
1 – Cadaver vs. Microsoft Hololens: A Comparison of Educational Outcomes of a Breast Anatomy Module
Valeda Yong
PURPOSE:Â Microsoft HoloLens (HL) mixed reality technology offers students a novel modality to visualize clinically important anatomical structures, such as the breast, which are uniquely challenging to discern with the naked eye in traditional cadaveric dissection. In this study, we developed a 3-D anatomical model of the breast and integrated it into a dynamic, educational module on the HoloLens. Here, we report the educational outcomes and overall impressions of medical students learning breast anatomy through our module, as compared with traditional dissections.
METHODS:Â A mixed reality breast anatomy module was created using AutoDesk 3DSMax and integrated into the HoloLens device. 38 first-year medical students were then recruited and divided into two groups: one participated in the HL module prior to dissecting the cadaveric breast, and the second dissected the cadaveric breast prior to the HL module. Before and after each teaching modality, participants answered seven comprehension-based breast anatomy questions and a survey about their experience.
RESULTS:Â Our preliminary results show that scores on the comprehension-based questions significantly improved more after the HL module than after the dissection (p=0.0209). Additionally, students appear to react more positively to the HL module than the dissection, regardless of which they were presented with first (p = 0.0008).
CONCLUSIONS:Â The present study demonstrates that students show better comprehension outcomes of breast structures using the HL. This is likely due to the ability to visualize structures that cannot be seen in the cadaver. Additionally, students appear to react more positively about learning breast anatomy through the HL module than through cadaveric dissection. Our results will be important in informing a future technology-driven medical education, especially in using mixed reality to supplement what a traditional cadaveric dissection may lack.
2 – OneNote: More Than Just a Place for Students to Compile Notes
Sarah McCarthy
PURPOSE:Â Among medical students at the Larner College of Medicine, OneNote is typically used to compile and annotate PowerPoint slides and notes from lectures. This allows students to keep everything conveniently in one place and easily share with fellow students. However, OneNoteâs capabilities expand far beyond this usage. Thus, we were interested in how OneNote could be used in the context of the current shift in medical education to more active and self-directed learning. Further, we wanted to test the feasibility of including faculty members in document sharing. We created a project that enabled first year medical students to create, update and share with faculty and classmates timelines of embryological development.
METHODS:Â Student groups (n=17) each developed a timeline based on embryological development in one shared notebook. Each groupâs timeline was visible to other groups and faculty. There were five class sessions dedicated to developing the timeline. Each session also included clinical cases which referenced the timeline to integrate their knowledge while answering clinically-relevant questions. Examination scores and end-of-course evaluations between 2016, when only traditional lectures were given, and 2017, with the new active learning curriculum, were compared.
RESULTS:Â There was no difference in overall examination scores between 2016 and 2017. While the sessions were well-reviewed by the students, they would have preferred to have more time to compile the timeline. Interestingly, the students continued working on the timeline outside of class to prepare for upcoming examinations.
CONCLUSIONS:Â To our knowledge, this is an innovative approach to teaching embryology as well as utilizing OneNote within the classroom. In the future, students will complete a version of their own timeline before each session to better utilize class time. OneNote as a collaborative tool for students and faculty in higher education has the potential to expand the classroom beyond its four walls.
3 – Enrollment Intentions of Learners for Open Online Courses in Different Educational Settings
Peter GM de Jong
PURPOSE:Â Open online courses such as Massive Open Online Courses (MOOCs) offer educators new opportunities for integrating high quality materials in their regular teaching programs. In order to make online resources useful for learners, the quality of the materials, the instructional design, and the integration with other learning activities need to meet their intentions. Enrollment intentions of MOOC learners have been described before, but not in different educational settings.
METHODS:Â A MOOC on Clinical Kidney Transplantation has been developed at Leiden University Medical Center in 2015. The course has been offered in different educational settings between June and December 2017. These setting were a regular course for third year medical students (compulsory), an international summer school program (elective), and a MOOC for the general public worldwide (voluntary). The enrollment intentions of the participants were assessed with the Online Learning Enrolment Intentions Scale as published by Kizilcec and Schneider (2015).
RESULTS: A total of 81 surveys were completed by medical students (36), summer school participants (16) and learners worldwide (29). In all three settings, the majority of the learners subscribed to the course with intentions of âgeneral interestâ, ârelevance to school or degree programâ, and âpersonal growth and enrichment.â In regard to the other learners the medical students scored lower on every intention, except on ârelevance to school or degree programâ and âexperience an online course.â The medical students hardly mentioned reasons for âcareer changeâ or âmeet new people.â The general learners score higher on âcareer changeâ and âearn a certificateâ, while summer school learners want to âmeet new people.â
CONCLUSIONS:Â Intentions of learners to attend the online course differ considerably and seem to be related to the educational setting. It seems plausible to take intentions into consideration in course design in order to improve the course usefulness.
4 – Promoting Interprofessional Communication in Rural Settings Through IPE Telemedicine Simulations
Richard Van Eck
PURPOSE:Â The Interprofessional Education Collaborative (IPEC) has identified four interprofessional competency domains,1 yet, teamwork looks different when it occurs in large urban medical facilities where multiple specialties are represented, versus low-resource rural settings with few healthcare practitioners on staff. Telemedicine technologies in rural healthcare settings may enhance IPC, yet the use of telemedicine requires additional experience and training for best results. Healthcare education must find ways to promote these skills for all students.2,3
METHODS:Â This intervention incorporated students from five healthcare professions into three longitudinal simulation scenarios that follow a single patient (STEMI) in a rural healthcare setting from diagnosis, to intervention/treatment, and end-of-life. Telemedicine (delivered via our ROBOTS computer tablets on Segway-like pedestals that allow audiovisual participation from a distance) is authentically integrated in each scenario, with each of five healthcare professions playing a role. We developed a modified version of the CATS instrument to measure IPC4 and telemedicine skills and conducted interrater reliability measures.
RESULTS: We have run three scenarios with at total of 273 students in social work, occupational therapy, physical therapy, nursing, and medicine. The resulting 65+ recorded video sessions are being analyzed with our modified CATS instrument. We have also conducted three group debriefings and two focus group interviews which are being coded and examined using thematic analysis. Data will be presented and the the modified CATS instrument will be made available for use by other  institutions. Methods and materials required for implementation will also be shared.
CONCLUSION:Â Despite the significant logistical issues involved with scheduling, response has been overwhelmingly positive and has generated interest in and requests for additional telemedicine longitudinal simulations. We have also developed a means of integrating telemedicine into virtually every other simulation without any additional cost in resources or time, thus extending the power of telemedicine components throughout the simulation curriculum.
Session 9 – Other
1 – How Students Choose E-Learning Resources – The Importance of Convenience and Familiarity
Michael Hortsch
PURPOSE:Â Electronic learning resources are popular with todayâs students. However, how they choose their favorite e-learning tools has not been investigated in detail and therefore is not sufficiently understood.
METHODS:Â The popular SecondLookâą Histology review tool was offered in three different formats to students participating in two histology courses at the University of Michigan, first in the form of PowerPoint files, second as an interactive website, and third as a mobile smartphone and computer tablet app. Identical in content, each format had its specific advantages and disadvantages, specifically the types of compatible devices, user features, and access limitations. After the conclusion of both courses, students were surveyed about their format choices and reasons for their selection, as well as usage patterns.
RESULTS:Â Only one out of 156 participating students (91.8% participation rate) reported not having used the SecondLookâą resource. The majority of students (>54%) tried only one resource format, with PowerPoint being the most popular final choice (58%). Although the interactive website and mobile app offered several user-friendly features not available with the PowerPoint files, they each only garnered around 20% final popularity. Ease of access and familiarity with the PowerPoint software were given most often as reasons for studentsâ format choice. The availability of specific devices and the flexibility of time and place to use the resource were also named as factors in the format selection. The access to a mobile learning tool enticed some students to use this resource in distractive environments which may interfere with learning. It also encouraged some students to forgo other learning opportunities, specifically textbooks, the course website, and other learning resources.
CONCLUSION:Â These results suggest that todayâs students are less motivated to actively seek out novel learning resources than commonly believed and instead often select familiar and convenient approaches and study tools.
Oral Award Nominee:
2 – Recorded Lectures: Who’s Watching, What’s Being Watched, and When?
Darren Hoffmann
PURPOSE:Â Lecture recording is used widely in medical schools to enable flexibility, efficient review of content, and distance learning. After implementing lecture recordings in all courses, lecture attendance dwindled over several years. We hypothesized that low lecture attendance was associated with higher usage of recorded lectures. Based on typical attendance patterns, we also hypothesized that recording use was associated with the type of content being presented or distance in time from the associated assessment.
METHODS:Â Detailed usage data from 64 lecture recordings were collected in an integrated biochemistry, cell biology and histology course in the first year of a combined Medical/PA program (n=177 students). Lectures were categorized based on attendance level (low, high), function in the course (primary content, review), and calendar distance from the exam (within one week, more than one week). Number of views was used as an indicator of usage.
RESULTS: Lecture recordings that featured primary content were used significantly more than lecture recordings of review sessions (180.78 ± 54.91 views vs. 91.28 ± 52.53 views, <0.0001). Due to this large difference, review lectures were excluded from subsequent analyses. Lectures that took place >1 week prior to an exam had significantly higher usage than those that occurred during the week of an exam (195.03 ± 55.89 views vs. 138.94 ± 32.19 views, p = 0.002).  Lectures with low attendance had more usage than those with high attendance (188.69 ± 53.81 views vs. 169.86 ± 55.85 views) but this difference was not significant (p = 0.24).
CONCLUSIONS:Â The results of this study indicate that lecture recordings are used more frequently for primary content than review material and for lectures that occur earlier in the sequence. It should not be assumed that lectures with low attendance have high online viewership.
3 – Cross-Year Peer Mentors in a Prematriculation Course Develop Competences Needed for Professional Success
Anna Wirta Kosobuski
PURPOSE:Â Prematriculation programs are implemented to support matriculant transition into medical school. The University of Minnesota Medical School, Duluth Campus (UM MSD) prematriculation program utilizes cross-year peer mentors to support students underrepresented in medicine (UIM), including Native American and rural students. Some consideration has been afforded to prematriculation program participant benefits (Wirta Kosobuski, Whitney, Skildum & Prunuske 2017); however, because not all programs utilize peer mentors, little attention has focused on how mentor roles affect peer mentors themselves. This study aimed to identify reported effects of peer mentorship on second-year medical school student mentors.
METHODS:Â Participants included 6 mentors across 3 years of a prematriculation program developed by the Center of American Indian and Minority Health, UM MSD. Mentors were second-year UIM medical students who were former prematriculation program participants. Primary analysis consisted of developing themes obtained from interviews with mentors subsequent to mentoring duties.
RESULTS:Â Cross-year peer mentors identified benefits and skills gained. Many of these aligned with the Accreditation Council for Graduate Medical Education core competencies: patient care; medical knowledge; practice-based learning and improvement; interpersonal and communication skills; professionalism; and systems-based practice. Challenges for mentors included the required time commitment and clear understanding of duties.
CONCLUSIONS:Â Results suggest that for UIM students, opportunities for involvement as cross-year peer mentors offer significant benefits, however, clearly defining mentor expectations duties is essential. Many skills acquired by near peer mentors are competencies required during residencies thus suggesting the need to consider implementation of cross-year mentoring more broadly throughout the medical school curriculum.
4 – A Peer-Assisted Learning Strategy for Evidence-Based Medicine Education in the Internal Medicine Clerkship
Michael R. Bubb
PURPOSE:Â An integrated curriculum at the University of Florida (UF) College of Medicine emphasizes a longitudinal evidence based medicine (EBM) thread building on work done in the first two years. The objectives of this study are to: (1) expand the structured teaching of EBM into 3rd year with a peer-assisted learning strategy; (2) focus on EBM critical appraisal skills and applicability issues; and (3) evaluate the impact of the addition of a formal EBM curriculum in 3rd year.
METHODS:Â Cooperative peer-assisted learning methodology was introduced with IRB approval. Students in groups of eight were paired, developed a PICO question, searched databases, retrieved one article, performed a critical appraisal and presented to their group once in a four-week block. Instructional videos were prepared to review key EBM concepts and peer-learning strategies. A faculty facilitator was present to answer questions. Pre- and post-testing of EBM knowledge employed the previously validated 15 point EBM Berlin Questionnaire. Results were compared with students completing the same rotation with the same available resources who submitted a written critical appraisal using the PICO format. The primary outcome was change in score.
RESULTS:Â 32 students completed the peer-assisted learning intervention and 32 students completed the individualized EBM activity. Average pretest scores were not different between the two groups (mean score of 7.32; SD 3.11). The change in score in the intervention group was 1.9 (95% CIz; 1.1 â 2.7) with a Cohenâs effect size of 0.58 or moderate to large effect. The change in score in the control group was 0.6 (95% CI: 0-1.2), and the difference between intervention and control did not achieve statistical significance.
CONCLUSIONS:Â In this preliminary study of the efficacy of peer-assisted EBM, an effect size was observed that was relatively large compared to other previously reported learning strategies.
Session 10 – TBL/PBL
1 – Evaluation of the Role of Incentive Structure on Student Participation and Performance in Active Learning Strategies: A Comparison of Case-Based and Team-Based Learning
Kathryn C. Behling
Purpose: Student participation is important for the success of active learning strategies, but participation is often linked to the level of preparation. At our institution, we use two types of active learning activities, a modified case-based learning exercise called active learning groups (ALG) and team-based learning (TBL). These strategies have different assessment and incentive structures for participation. Non-cognitive skills are assessed in ALG using a subjective 5-point Lickert scale. In TBL, assessment of individual student preparation is based on a multiple choice quiz conducted at the beginning of each session.
Methods: We studied first-year medical student participation and performance in ALG and TBL as well as performance on course final examinations. Student participation in TBL and ALG exercises occurring in concert with presentation of first year, medical school microbiology course content was recorded. Twenty-four students who made up 3 ALG groups (with 8 students per group) and 4 TBL teams (with 6 students per team) were consented to participate in this research. Student participation was quantified (%Time-TBL and %Time-ALG) and correlated with performance in ALG as well as on individual readiness assurance tests (iRAT), group readiness assurance tests (gRAT), and final examinations.
Results: Student performance in TBL, but not in ALG, was strongly correlated with final examination scores. Additionally, in students who performed in the upper 33rd percentile on the final examination, there was a positive correlation between final examination performance and participation in TBL and ALG. This correlation was not seen in students who performed in the lower 33rd percentile on the final examinations.
Conclusions: Our results suggest that assessments of medical knowledge during active learning exercises could supplement non-cognitive assessments and be good predictors of performance on summative examinations.
2 – Combination of Two Different Active Learning Strategies, Team-Based Learning and Hands-On Experimentation, Strengthens Student Acquisition of Course Content In Underrepresented in Medicine Undergraduate Students
Gonzalo A. Carrasco
Purpose:Â Diversity in medical providers has been shown to improve access to care in underserved communities and cultural competence, while also creating a richer learning environment for medical students. We have previously reported that combined team-based learning (TBL) exercises and hands-on experimentation in a molecular biology course for underrepresented in medicine (URM) senior undergraduate students resulted in improved student performance. In this two-year study, we also examined the correlation between student preparation, participation and performance in TBL exercises in this course.
Methods:Â The molecular biology course runs for six-weeks and consists of daily lectures, TBL exercises, and laboratory sessions. In the laboratory, students from four TBL groups, composed of 6 students each, worked together to amplify, sequence, and subsequently identify disease specific genes. Fifty-one students (31 female and 20 male) consented to participate in this study. Student participation in TBL was quantified (%Time-TBL) and correlated with performance on individual readiness assurance tests (iRAT), group readiness assurance tests (gRAT), and final grades (FG).
Results:Â Overall, there was a significant positive correlation between: iRAT and %Time-TBL, iRAT and FG (r=0.31,<0.01 and r=0.49,<0.01), and %Time-TBL and FG (r=0.28,<0.01). Subgroup analysis showed a significant positive correlation between iRAT and %Time-TBL in female and male students (r=0.26,<0.01 and r=0.42,<0.01 respectively). Interestingly, we only found a positive correlation between iRAT and FG (r=0.57,<0.01) and between %Time-TBL and FG in female (r=0.27,<0.01), but not male, students.
Conclusions:Â Prior preparation for TBL exercises, as demonstrated by higher iRAT scores, is correlated with participation in TBL and overall course performance. In female students, TBL preparation and participation was better correlated with course performance, which may be due to the immediate accountability that occurs during gRAT, which may positively reinforce confidence in female students both in TBL and throughout the course. Addition of the hands-on laboratory activities may also improve TBL team cohesion.
3 – Comparative Study of Online and Classroom-Based Team-Based Learning in Immunology
Robin A. Harvan
PURPOSE:Â Team-Based Learning (TBL) is an evidence based collaborative learning teaching strategy designed around units of instruction, known as âmodules,â that are taught in three-step phases: preparation, readiness assurance testing, and application-focused exercise. This educational research study investigated student achievement and student satisfaction related to TBL approach in a traditional face-to-face classroom setting and online format in an upper level undergraduate immunology course.
METHODS: In this study, 60 students were randomly assigned to 12 small groups and participated in four immunology classes/case study modules in-class or online. For each immunology class/case study, six assigned groups completed TBL case study lessons online and six groups completed two cases study lessons in class. In both comparative groups students completed pre-class readings and preparation.
This study compared the efficacy of student learning outcomes, preferences and satisfaction using the TBL approach in classroom-based and online formats in an upper level undergraduate immunology course. Data collection instruments included multiple-choice IRATs and GRATs administered as individual pre-test and group post-test after group discussion in flipped classroom settings online and face-to-face. Four multiple choice exams were also administered to both groups simultaneously in classroom settings with item analysis of TBL case specific questions. Questionnaires were administered to collect and analyze student preferences and satisfaction of TBL lessons conducted in class and online.
RESULTS:Â Results indicated that in all four modules students performed better on the GRAT than the IRAT. Comparisons of teaching and learning in-class versus online showed that students performed equally well on the IRATs, but had mixed performances on the GRATs. Not surprisingly, students preferred to learn in a traditional face-to-face classroom setting over the online format but would consider hybrid.
CONCLUSION:Â This comparative study further advanced our understanding of the benefits of team-based learning for health sciences and health professions students in classroom-based and online formats.
4 – Matchy-Matchy!: A Catchy TBL Application with Highly Kinetic Activities
Joseph Baker
PURPOSE: The application phase of a Team-Based Learning session (TBL) can involve discussions or activities that allow learners to apply, explore, and expand upon their existing knowledge. We have developed a series of highly kinetic activities for the application phase that differ greatly from the traditional group questioning and case resolution that is often employed. These highly kinetic activities involve a matching format that we call Matchy-Matchy! where learners have to match up elements of multiple cases to form the completed case scenario.
METHODS: The activities are designed for 2nd year medical students enrolled in systems courses. For example, one activity involves matching up cases of various endocrine disorders with their various attributes. The students, working in groups, are given an envelope of pre-cut cards, photos, EKGs, diagnostic images, and/or pathology slides to associate. When the exercise is completed, all groups reconvene for discussion. Student feedback was collected with our learning content management system and a Qualtrics survey software.  The impact on the exam was analyzed with ExamSoft software. This study was approved by ACOM IRB committee.
RESULTS: The analysis of student surveys indicates students found the activities to be helpful and relevant. The feedback comments submitted by the students were also very supportive. Â Analysis of the examination items relating to the content in the highly kinetic activity suggest a correlation can exist between the learning process of these events and success in the examination.
CONCLUSION: The highly kinetic TBL application has fostered a higher level of engagement, interest and fun. We believe that this will translate into measurable improvements over time. We plan to continue to develop these highly kinetic exercises for TBL application and monitor our studentsâ continuous improvement.