[The following notes were generated by Douglas McKell MS, MSc and Rebecca Rowe, PhD]

The Fall 2023 IAMSE WAS Seminar Series, “Brains, Bots, and Beyond: Exploring AI’s Impact on Medical Education,” began on September 7^th, 2023, and concluded on October 5^th, 2023. Over these five sessions, we will cover topics ranging from the basics of AI to its use in teaching and learning essential biomedical science content.

Dr. Cornelius James and Erkin Otles presented the second session in this series from the University of Michigan. Dr. James is a Clinical Assistant Professor in the Departments of Internal Medicine, Pediatrics, and Learning Health Sciences at the University of Michigan (U-M). He is a primary care physician, practicing as a general internist and a general pediatrician. Dr. James has completed the American Medical Association (AMA) Health Systems Science Scholars program and was one of ten inaugural 2021 National Academy of Medicine (NAM) Scholars in Diagnostic Excellence. As a NAM scholar, he began working on the Data Augmented Technology Assisted Medical Decision Making (DATA-MD) curriculum. The DATA-MD curriculum is designed to teach healthcare professionals to use artificial intelligence (AI) and machine learning (ML) in their diagnostic decision-making. Dr. James and his colleagues are also using DAYA-MD to develop a web-based AI-based curriculum for the AMA.

Mr. Erkin Otles is a seventh-year Medical Scientist Training Program Fellow (MD-PhD student) at the University of Michigan. His research interests are across the continuum of clinical reasoning and include creating digital reasoning tools at the intersection of AI and medicine, including informatics, clinical reasoning, and operation research. His doctoral research focused on creating AI tools for patients, physicians, and health systems. He has led work across the AI lifecycle with projects advancing from development to validation, technical integration, and workflow implementation. He is also interested in incorporating AI into Undergraduate Medical Education.

Dr. James began the session by reviewing the webinar objectives and the attendees’ response to AI’s impact on healthcare and medical education. Mr. Otles defined Artificial Intelligence AI as a lot of math and programming with nothing magical about its operation. He defined AI as intelligence that systems constructed by humans demonstrate.  He defined Machine Learning (ML) as a subfield of AI interested in developing methods that can learn and use data to improve performance on a specific task.  An example of this is a physician interested in analyzing pathology slides to see if there is evidence of cancer cells. AI or ML systems might use historical data (consisting of images and previous pathologist interpretations) to learn how to detect cancer evidence in slides it has never seen before, thereby demonstrating its learning and data recognition ability based on human programming.

As we learned in the first session of this series, AI and ML contain many sub-fields. Inside AI exists a technique known as natural language processing (NLP), which can process human-generated text or speech into data. There is also knowledge representation or reasoning (KRR), which is the ability to encode human knowledge in a way that is automatically processed.  Input from sensors and related biometric data is of this type. Even though AI and ML are used synonymously, many techniques are AI but not ML.

There are also subsets within ML, such as Deep Learning, which is very effective because it can identify patterns from large amounts of data. ML draws heavily from optimization theory, operations research, statistics analysis, human factors engineering, and game theory, to name a few contributing disciplines.

We encounter AI in our everyday lives, such as asking Siri to play music, route planning, spam detection, route planning, and topic word searching by Google.  Accomplishing these tasks often requires several AI tools working together to accomplish the seamless appearance of the final task presentation to a human. Mr. Otles emphasized that many industries depend heavily on AI embedded in their operations. For example, Airlines and e-commerce businesses use AI to optimize their operations workflow. In contrast, tech companies use AI to sort through large databases to create content that will keep their users engaged with ML, presenting this to each person based on prior user or purchasing habits.

Mr. Otles presented a brief review of ChatGPT that identified its two primary functions: 1. A Chatbot for human user interface, and 2. A Large Language Model to predict the best answer, using reinforcement learning training to identify the preferable word(s) response. He pointed out the inherent problem with this process that results in answers entirely dependent on whatever data it was trained on, which may not be accurate, relevant, or unpleasant. Understanding where the data came from allows us to understand how it can be used, including its limitations. The random sampling of the underlying data also contributes to its limitations in ensuring its accuracy and reliability.

Mr. Otles then switched to discussing how AI is used in health care. Not too long ago, it was unusual to see AI used in health care. Demonstrating the rapid growth in AI’s impact on healthcare, Mr. Otles presented a chart showing that the number of FDA-cleared AI devices used in healthcare increased from one device in 1995 to 30 devices in 2015 to 521 devices in 2022. He emphasized that not all AI devices need to be approved by the FDA before being employed in healthcare and that most are not. He gave an example of a specific AI system at the University of Michigan, approved for use by the FDA in 2012, that removes the bone images from a chest radiograph to provide the radiologist with a more accurate view of the lungs. Other examples of the use of AI in healthcare from the speaker’s research include prostate cancer outcome prediction in hospital infection and sepsis risks and deterioration risks. Mr. Otles cautioned that a number of the proprietary LLMs currently used that have not been subject to FDA scrutiny have not proven to be as accurate predictors as initially thought due to the developer’s lack of understanding of their use in actual clinical context it is being used in that was not the same as when they were trained in research-only environments (Singh, 2021; Wong, 2021). He also shared several feedback examples in medical education that identified the type and possible gaps, or biases, in feedback provided to medical and surgical trainees based on their level of training (Otles, 2021; Abbott, 2021; Solano, 2021).

Mr. Otles concluded his presentation by discussing why physicians should be trained in AI even though AI is not currently a part of most medical schools’ curricula. He emphasized that physicians should not just be users of AI in healthcare; they need to be actively involved in creating, evaluating, and improving AI in healthcare.  Healthcare users of AI need to understand how it works and be willing to form partnerships with engineers, cognitive scientists, clinicians, and others, as AI tools are not simply applications to be implemented without serious consideration of their impact. He stressed that healthcare data cannot be analyzed outside of its context. For example, a single lab value is meaningless without understanding the patient it came from, the reported value, the timing of the measurement, and so on. Mr. Otles pointed out that AI has a significant benefit for physicians to rapidly summarize information, predict outcomes, and learn over time which can ultimately benefit physicians. He suggested that these AI characteristics of using complicated data and workflows to reach an answer and subsequent action were the same type of decision processes that physicians themselves use and understand. As a result, Mr. Otles stressed that physicians need to be more than “users” of AI tools; they need to be involved in creating, evaluating, and improving AI tools (Otles, 2022).

Dr. James hosted the remainder of the Webinar.  Dr. James presented two questions to the audience: “Are you currently using AI for teaching, and Are you currently teaching about AI in health care”?  Dr. James then presented to the audience the latest recommendations from the National Academy of Medicine (NAM) related to AI and health care and highlighted their recommendation that we develop and deploy appropriate training and education programs related to AI (Lomis et al., 2021). These programs are not just for physicians but must include all healthcare professionals. He also discussed a recent poll published in the New England Journal of Medicine (NEJM), where 34% of the participants stated that the second most important topic medical schools should focus on was data science (including AI and ML) to prepare students to succeed in clinical practice (Mohta & Johnston, 2020).

Currently, AI has a minimal presence in the curriculum in most medical schools. If it is present, it is usually an elective, a part of the online curriculum, workshops, or certificate programs.  He stated that this piecemeal approach was insufficient to train physicians and other healthcare leaders to use AI effectively. As more and more medical schools start to include AI and ML in their curricula, Dr. James stressed that it is essential to set realistic goals for what AI instruction and medical education should look like.  Just as not all practicing physicians and other healthcare workers are actively engaged in clinical research, it should not be expected that all physicians or clinicians will develop a machine-learning tool. With this said, Dr. James stated that just as all physicians are required or should possess the skills necessary to use EBM in their practice, they should also be expected to be able to evaluate and apply the outputs of AI and ML in their clinical practice. Therefore, medical schools and other health professional schools need to begin training clinicians to use the AI vocabulary necessary to serve as patient advocates and ensure their patient data is protected and that the algorithms used for analysis are safe and not perpetuating existing biases.

Dr. James reviewed a paper by colleagues at Vanderbilt that provides a great place to start as a way to incorporate AI-related clinical competencies and how AI will impact current ACGME competencies, such as Problem-Based Learning and Improvement and Systems-Based Practice, which is part of the Biomedical Model of education used for the training of physicians (McCoy et al., 2020). Even though historically, the Biomedical model has been the predominant model for training physicians, he suggested that medical education start to think about transiting to a Biotechnomedical Model of educating clinicians or healthcare providers (Duffy, 2011). This model will consider the role that technology will play in preventing, diagnosing, and treating illness or disease.  He clearly stated that he does not mean that models like the bio-psycho-social-spiritual model be ignored. He stressed that he was suggesting that the Biotechnomedical model be considered complementary to the bio-psycho-social-spiritual model to get closer to the whole-person holistic care that we seek to provide. If medical education is going to be able to prepare our learners to be comfortable with these technologies successfully, then a paradigm shift is going to be necessary. He believes that within 1-2 years, we will see overlaps between AI and ML content in courses like Health System Science, Clinical Reasoning, Clinical Skills, and Evidence-Based Medicine. This is already occurring at the University of Michigan, where he teaches.

Dr. James feels strongly that Evidence-Based Medicine (EBM) is at least the first best home for AI and ML content.  We have all heard that the Basic and Clinical Sciences are the two pillars of medical education. Most of us have also heard of Health Systems Science, which is considered to be the third pillar of medical education. Dr. James anticipates that over the next five to ten years, AI will become the foundation for these three pillars of medical education as it will transform how we teach, assess, and apply knowledge in these three domains. He briefly reviewed this change in the University of Michigan undergraduate medical school curriculum.

Dr. James concluded his presentation with an in-depth discussion of the Data Augmented Technology Assisted Medical Decision Making (DATA-MD) program at his medical school. It is working on creating the foundation to design AI/ML curricula for all healthcare learners (James, 2021). His team has focused on the Diagnosis process using EBM, which will impact the Analysis process. Their work with the American Medica Association is supporting the creation of seven web-based modules using AI more broadly in medicine.

Dr. James concluded his presentation by stressing four necessary changes to rapidly and effectively incorporate AI and ML into training healthcare professionals. They are: 1. Review and Re-prioritize the existing Curriculum, 2. Identity AI/ML Champions, 3. Support Interprofessional Collaboration and Education, and 4. Invest in AI/ML Faculty Development. His final “Take Home “ points where AI/ML will continue to impact healthcare with or without physician involvement, AI/ML is already transforming the way medicine is practiced, AI/ML instruction is lacking in medical education, and Interprofessional collaboration is essential for healthcare professionals as key stakeholders.

As always, IAMSE Student Members can register for the series for FREE!

[The following notes were generated by Douglas McKell. MS, MSc and Rebecca Rowe, PhD]

The Fall 2023 IAMSE WAS Seminar Series, “Brains, Bots, and Beyond: Exploring AI’s Impact on Medical Education,” began on September 7, 2023, and concludes on October 5, 2023. Over these five sessions, we will cover topics including the foundational principles of Artificial Intelligence and Machine Learning, their multiple applications in health science education, and their use in teaching and learning essential biomedical science content.

The series began with the session “An Introduction to Artificial Intelligence and Machine Learning with Applications in Healthcare” by Dr. Homayoun Valafar. Dr. Valafar is Chair of the Department of Computer Science and Engineering, Director of SC INBRE Bioinformatics Core, and Director Big Data Health Science Center for Genomic Core at the University of South Carolina.

Dr. Valafar started the session by discussing the role of Artificial Intelligence (AI) and Machine Learning (ML) in scientific discovery, followed by defining the relationship between AI and ML. He then reviewed the process of training ML models for practical applications. He completed the session by providing two examples of AI and ML techniques where they are being applied in different healthcare environments.  

At the beginning, it is important to note that there is an overlap with data science with AI and ML. He briefly explained the traditional data management role of acquisition,  verification, storage, retrieval, and analysis by data scientists, pointing out that today, AI and ML play a central role in data management. ML and the application of AI offer several advantages over traditional data management models, especially in data storage and retrieval, as well as pattern recognition.

Dr. Valafar explained that while AI and ML tend to be used interchangeably, this is incorrect because they are distinctly different. He emphasized that while ML is a subset of AI, some types of AI do not involve ML.  A technique is considered AI when machines act intelligently because humans specifically program intelligent actions. Conversely, ML is a sub-branch of AI that occurs when machines learn on their own how to manage the data they are given. The machine itself will know what it is doing; however, the human may or may not know what the device is doing or how it is doing it.  The machine learns on its own the governing rules of its behavior to reach a solution. Dr. Valafar termed this difference as being “Black Boxed (ML) vs. White Boxed (AI)” or transparent vs. opaque. In the former, the ML produces excellent output, but you have no idea the rules it uses to accomplish this task. In the latter, you have programmed the rules, but in both cases, you still need to assess the accuracy of the output.

An example of AI is when a human tells “Alexis” to turn the lights on at 7:00 p.m. Here, you, the human, specify the rules that the machine is to follow. The machine has “learned” to interpret your gestures into actions it needs to take. Other examples included moving your computer cursor to change a letter or a word, multiple keystroke actions to “automatically” make changes in a document, and using voice recognition to specify locations, playlists, and songs.

He also shared examples of AI that are also examples of ML. They included providing the data to an ML program and asking it to find patterns indistinguishable to me as a human. For example, when given the data:

• find patterns that determine how a patient responds to a drug
• distinguish the distinguishing attributes of smoking behavior
• find common characteristics of patients with vascular disease.

ML can be divided into two branches: supervised learning and unsupervised learning. In supervised learning, the human is involved in machine learning model training. The human expert has reviewed, categorized, and labeled the data to discriminate between critical differences for investigation. e.g., a drug responder or a non-drug responder, or the presence or absence of calcification in the arterial system in a radiological image. This information is used to train the ML model. Because humans are involved, this technique is more common and reliable but tends to be more time and resource-consuming.  Supervised learning will also incorporate human error.  Unsupervised learning is when the machine will figure out the data independently, such as determining the clustering of data. This technique tends to be less prevalent and is used most in an academic setting.

This session focused on supervised learning, which includes collecting and aggregating relevant data, scrutinizing the data for accuracy and bias, such as sex, age, or race-based biases, removing biases, and dividing the data into training or testing sets. Failing to remove biases results in reliable but not valid data analysis by the ML model. The training set is used to train the model, and the testing set is not exposed to the ML network at all and is used to evaluate the trained model. The model’s training occurs when the internal parameters are adjusted until the correct input/output association is achieved. This occurs when the accuracy of performance increases while errors are decreased. The test set validates the accuracy of the training model. This process is central to the development of an Artificial Neural Network. The training process involved multiple iterations where you focus on increasing the accuracy of the output while minimizing the information loss.

Dr. Valafar emphasized that the ML process he outlined is based on the unsupervised approach and comes with an understanding that depending on what form of ML you employ, you may be able to extract the information out of the ML or it may just be embedded in the ML model itself and inaccessible you. In other words, you cannot explain nor replicate the process (rules) the ML model used to arrive at its output. ML techniques like this will usually depend on applying many well-known statistical methods such as Linear or Logistic Regression, Bayesian Classifiers, Decision Trees, Random Forest, and Artificial Neural Networks.

For the remainder of the webinar, Dr. Valafar provided several applications of how ML, incredibly biologically similar Deep Neural Networks, a sub-set of Artificial Neural Networks trained on large databases, can perform remarkably accurate predictive identification of critical characteristics of interest in new data, for example, a single patient expected response to a new drug over time. He reminded us that while this approach generally outperforms all other AI approaches, it still lacks explainability and reliability as a “Black Box” ML model. On the other hand, Decision Tree Models perform less well on complicated tasks but are explainable with information transparency. Examples included determining the cardiovascular factors given the set of individual patient factors such as age, weight, and smoking to categorize the patient as high risk or low risk. Human experts must provide the initial data categorization to help train the ML model.

ML-based AI examples included the Application of AI and ML to facilitate patient management in the emergency room during a mass casualty event by reducing the time-to-treatment for 300 simulated patients from 12 hours to 3 hours, identifying patients most likely to receive clinical benefit from Cardiac Resynchronization Therapy, determine an individualized predictive hydroxyurea drug response for Sickle Cell Anemia at 3, 6, and 9 months based on blood sample of target fetal hemoglobin above 12 % sing a Digital Twin graphing process, and using ML in vascular surgery where it can detect and quantify calcification or plaques and identify and track the health of the aorta and the peripheral arteries. The final application of AI and ML involves the patient wearing activity monitoring devices, such as watches, rings, necklaces, wrist or ankle bands that can monitor the number of steps, amount of sleep, sports activity, smoking cessation, whether the patient is eating and or taking their medications for example. In each of these examples, the individual’s physical actions are translated (interpreted by ML) into a pre-identified highly probable behavior with remarkable but not infallible accuracy, e.g., smoking and medication taking can be confused.

In conclusion, Dr. Valafar stated AI and ML have the potential to revolutionize healthcare and medical sciences. AI can be used to optimize patient outcomes, and ML can become an assistant to practitioners and tutors to apprentices by becoming a second set of eyes, by identifying subtle changes in a patient’s physiology, or by predicting an individual’s future risk factors for a disease progression or treatment response.  He ended by highlighting the two barriers that will prevent the full potential of AI in healthcare: 1. data sharing barriers need to be removed, and 2. the need for better collaboration between the health practitioner and the data sciences.

As always, IAMSE Student Members can register for the series for FREE!

Are you curious how Artificial Intelligence (AI) is transforming medical education, especially its impact on faculty teaching and student learning? Join the upcoming IAMSE Fall webinar series entitled “Brains, Bots, and Beyond: Exploring AI’s Impact on Medical Education” to learn about the intersection of AI and medical education. Over five sessions, we will cover topics ranging from the basics of AI to its use in teaching and learning essential biomedical science content.

The series begins on September 7 with a presentation by Homayoun Valafar to define AI and machine learning. The series will continue on September 14 with a discussion by Erkin Otles and Cornelius James on strategies to prepare our trainees to appropriately utilize AI in their future healthcare jobs. On September 21st, Michael Paul Cary and Sophia Bessias will present on critical ethical issues, including the potential for unintended bias and disparities arising from AI. Finally, Dina Kurzweil and Bill Hersh will wrap up the series on September 28th and October 5th, respectively, with practical tips for educators and learners alike to utilize AI to maximize teaching and learning. Don’t miss this exciting opportunity to join the conversation on the future of AI in medical education.

Transforming Healthcare Together: Empowering Health Professionals to Address Bias in the Rapidly Evolving AI-Driven Landscape

Presenters: Sophia Bessias, MPH, MSA and Michael Paul Cary, Jr., PhD, RN, FAAN
Session Date & Time: September 21, 2023 at 12pm Eastern
Session Description: As the interest in utilizing AI/machine learning in healthcare continues to grow, healthcare systems are adopting algorithms to enhance patient care, alleviate clinician burnout, and improve operational efficiency. However, while these applications may appear promising, they also carry certain risks, including the potential to automate and reinforce existing health disparities.

During this seminar, we will introduce the ABCDS Oversight framework developed at Duke Health. This comprehensive framework focuses on the governance, evaluation, and monitoring of clinical algorithms, providing participants with practical guidance to ensure the responsible implementation of AI/ML. Specifically, we will highlight how high-level principles can be translated into actionable steps for developers, allowing them to maximize patient benefit while minimizing potential risks.

Read the full session description here.

As always, IAMSE Student Members can register for the series for FREE!

Are you curious how Artificial Intelligence (AI) is transforming medical education, especially its impact on faculty teaching and student learning? Join the upcoming IAMSE Fall webinar series entitled “Brains, Bots, and Beyond: Exploring AI’s Impact on Medical Education” to learn about the intersection of AI and medical education. Over five sessions, we will cover topics ranging from the basics of AI to its use in teaching and learning essential biomedical science content.

The series begins on September 7 with a presentation by Homayoun Valafar to define AI and machine learning. The series will continue on September 14 with a discussion by Erkin Otles and Cornelius James on strategies to prepare our trainees to appropriately utilize AI in their future healthcare jobs. On September 21st, Michael Paul Cary and Sophia Bessias will present on critical ethical issues, including the potential for unintended bias and disparities arising from AI. Finally, Dina Kurzweil and Bill Hersh will wrap up the series on September 28th and October 5th, respectively, with practical tips for educators and learners alike to utilize AI to maximize teaching and learning. Don’t miss this exciting opportunity to join the conversation on the future of AI in medical education.

Artificial Intelligence: Preparing for the Next Paradigm Shift in Medical Education

Presenters: Cornelius James, MD and Erkin Otles, MD/PhD Student
Session Date & Time: September 14, 2023 at 12pm Eastern
Session Description: During this session, participants will learn about the impact that artificial intelligence (AI) and machine learning (ML) will have on the practice of medicine, and subsequently medical education. Participants will learn what AI and ML are, and about their current applications in healthcare. Finally, participants will be able to identify opportunities for incorporating AI/ML content into their curricula.

As always, IAMSE Student Members can register for the series for FREE!

Are you curious how Artificial Intelligence (AI) is transforming medical education, especially its impact on faculty teaching and student learning? Join the upcoming IAMSE Fall webinar series entitled “Brains, Bots, and Beyond: Exploring AI’s Impact on Medical Education” to learn about the intersection of AI and medical education. Over five sessions, we will cover topics ranging from the basics of AI to its use in teaching and learning essential biomedical science content.

The series begins on September 7 with a presentation by Homayoun Valafar to define AI and machine learning. The series will continue on September 14 with a discussion by Erkin Otles and Cornelius James on strategies to prepare our trainees to appropriately utilize AI in their future healthcare jobs. On September 21st, Michael Paul Cary and Sophia Bessias will present on critical ethical issues, including the potential for unintended bias and disparities arising from AI. Finally, Dina Kurzweil and Bill Hersh will wrap up the series on September 28th and October 5th, respectively, with practical tips for educators and learners alike to utilize AI to maximize teaching and learning. Don’t miss this exciting opportunity to join the conversation on the future of AI in medical education.

Homayoun Valafar, PhD

An Introduction to Artificial Intelligence and Machine Learning with Applications in Healthcare

Presenter: Homayoun Valafar, PhD
Session Date & Time: September 7, 2023 at 12pm Eastern
Session Description: In this session, Dr. Valafar will provide an introductory overview of the domain of Artificial Intelligence and Machine Learning. The session will include a brief overview of various techniques, an abstract view of training and testing AI models, some applications in the domain of Healthcare, and some of the challenges ahead.

As always, IAMSE Student Members can register for the series for FREE!

The International Association of Medical Science Educators (IAMSE) is pleased to announce that the archives for “The Struggle is Real: Breaking Barriers That Limit Student Success,” the 2022 Fall series of the Webcast Audio Seminar are now online!

The Webcast Audio Seminar archives are located on the IAMSE website under the Events heading as Web Seminars. Here, you will be able to search the archives or browse by year and series.

In addition, registration for the 2023 Fall series is now open! This series, entitled, “Brains, Bots, and Beyond: Exploring AI’s Impact on Medical Education”, will begin September 7. Click here to register! 

If you have any issues accessing the archives, please let us know at support@iamse.org.