A Web-based Module on Lymphoma for Senior Medical Students: Benefits for Learning

INTRODUCTION

Medical education is rapidly evolving under the influence of many forces, with one of the key contributors being the shift towards student-centred learning throughout tertiary education. With medical education taking a decisive step away from traditional didactic teaching, there is an increasing need for resources that will facilitate independent learning.

Many academics in the medical sciences view computer-assisted learning (CAL) resources as a means to overcome reduction in student contact hours (often as much a result of funding and staffing issues as educational principles).1 Geographical dispersal of clinical schools is also cited as impetus for CAL development.^1,2 However, although these issues may have stimulated the use of CAL resources, they have not necessarily driven effective implementation.

At the University of New South Wales (UNSW), the integrated Medicine program is structured around the development of core graduate capabilities, deemed vital for effective practice immediately following graduation and throughout a career in Medicine.³ The program is structured over three phases, with each successive phase increasing students’ independence. Phase 3 (years 5 and 6) is taught entirely in a clinical setting, mostly in tertiary referral hospitals, over 10 clinical rotations, each of eight weeks duration.

The iterative design of the program has driven the development of a novel biomedical sciences curriculum for students in Phase 3, built on previous learning. A major focus of the biomedical sciences curriculum, also emphasized in the assessment at the end of year 5, is the rational use and interpretation of diagnostic investigations. However, teaching of biomedical sciences is a challenge in the later phases of a Medicine program because students are focused on clinical learning. Further, there are issues with allocation of teaching time, and with multi-site implementation – in addition to four metropolitan Clinical Schools, UNSW has four Rural Clinical School (RCS) campuses, some of which are over 500 km away from the main campus.

It has been suggested that CAL is a potential solution for the above-mentioned challenges, particularly for topics perceived to include concepts that students find difficult.¹ Most of the evidence supporting the efficacy of CAL has come from studies describing a single application. Further, such evaluations are frequently qualitative, utilising staff and student feedback, rather than quantitatively comparing learning outcomes of different teaching modalities. Very few studies used randomized trials to compare learning outcomes between groups of students.

The present study employed a randomized controlled trial to evaluate the learning benefits of a web-based module on the topic of lymphoma for senior medical students. By reinforcing understanding of Pathology in a clinical context, the module was intended to support clinical experiences in the development of necessary skills to make appropriate use of diagnostic tools upon graduation.⁴

MATERIALS AND METHODS

A needs analysis survey was administered to final year medical students at UNSW. The survey asked students to identify those topics in the Pathology curriculum that they found most difficult, and what features they would find most helpful in a CAL module. From the survey, lymphoma and leukemia were topics with the highest perceived level of difficulty among students. Further, students emphasised that a case-based approach was most beneficial for their learning. Following discussion among the authors, lymphoma was chosen as the topic for this CAL module.

Development of the Web-based Lymphoma Module

Development of the module required a tool that could produce an interactive yet user-friendly learning resource, while providing ease of authoring. Adobe Captivate™ was chosen as the authoring program for its balance of design options and ease of use. Of particular note is the capacity of this software to import PowerPoint presentations, then to add rich interactivity to them.

Module Design

The aim of the module was to link clinical experiences to the basic sciences, particularly in relation to diagnostic protocols. This connection has been achieved in existing UNSW Pathology modules on CD-ROM, by addressing overarching principles in a Concepts and Causes section, followed by several sections, each of which contain exemplar case studies. A similar framework was used for the online Lymphoma module. The Concepts and Causes section began with brief revision of fundamentals of the lymphatic system, lymphoid cells, and their development. It then briefly outlined causes of lymphadenopathy before focusing on the classification, diagnosis, staging and prognosis of lymphoma. Each of the Case Studies then presented a case history, which was intended to be worked through in interactive steps. The cases represented a selection of the more common types of lymphoma, focusing on diagnosis and staging.

Interface and Navigation

The interface was kept simple, uncluttered and consistent throughout the module, from the introductory pages to the case studies and glossary (Figure 1) to minimize time needed to learn how to use the module, as well as to reduce the cognitive load on students. Text on each screen was kept to a minimum to avoid overwhelming students and creating the effect of an on-line textbook. Audio was used to supplement and expand on information provided in the text, to diversify the mode of presentation and to maintain interest.

Occasionally, a side branch from the main content path was utilized to provide additional or background information on a specific concept, for example, explaining the significance of light chain monoclonality. These side branches were kept to a maximum of two screens, and clearly returned the student to the screen from which they diverged.

Feedback and Interactive Features

In the context of CAL, interactivity and feedback are core features in promoting learning. Throughout the module, students were encouraged to interact with the concepts presented by answering questions and identifying features on images. Answers to the questions were provided immediately by audio or by a “roll over” link (Figure 2). For “roll over” answers, when the student held the mouse cursor over a green highlighted area on the screen, a text box appeared with the answer. For questions that required students to combine information from several screens, answers were often provided through audio commentary on the following screen.

Definitions of key terms were provided in a similar manner. Although a glossary was provided, linked from the main menu, students could also directly access definitions from the screen containing the term through orange “roll over” boxes. The colour coding was consistent throughout the module. The immediacy of access to information and feedback reduced the germane load (cognitive load required to process the interface) placed on students, allowing them to focus on the concepts presented.⁵

Figure 1

Figure 1 shows an annotated screenshot of the lymphoma module interface

Students were also asked to interact with histopathological and radiological images, either by listing visible features, or finding cell types of importance (e.g. Reed-Sternberg cells). Answers were again provided by “roll overs” (with arrows instead of text), or audio.

Web Access

The world-wide Web was used as a means of distribution. The module was converted to a Flash™ file then loaded onto a Web server, allowing students to view the module from home, urban and rural clinical schools, as well as on the main University campus. Access was password protected for the duration of the study. The module is now available to view (Flash player and speakers required) at <a href=”http://web.med.unsw.edu.au/Pathology/Lymphoma/lymphoma.htm” http://web.med.unsw.edu.au/Pathology/Lymphoma/lymphoma.htm

Study Design

Participants

All students in Year 5 (Phase 3) of the Medicine program at UNSW were invited via email to participate in a randomized controlled trial of a web-based lymphoma module, of which 22 responded (13 Female, 9 Male). It could be argued that this method of recruitment might have skewed the study in favour of participants who were comfortable with, and interested in, Web-based learning. However, all students in the Medicine program at UNSW utilise Web-based learning (e.g. virtual slides, online formative assessments) from the outset. Hence the vast majority of students are familiar with the format used for this module.

Participants were then matched for both previously demonstrated academic performance and gender, then randomized into either the Control group (no access to the module) or Study group (access to the module). Students were advised that individual results from a post-test on lymphoma and questionnaire at the end of the trial period would be kept confidential and would not impact on their academic standing. All participants were provided with access to the Web-based module after completion of the trial period and post-test. The study received ethics approval from the UNSW Human Research Ethics Committee (Ethics Approval No: 2007-7-43).

Figure 2

***Image2:center***
Figure 2 shows an annotated screenshot of an interactive “roll over” in the lymphoma module.

Participating students received instructions by email regarding the two-week study period. Both groups were provided with a short list of textbook references covering the topic of lymphoma. Emails to members of the Study group also included a link to the module, with each individual receiving a unique username and password for the duration of the study, which they were advised to keep confidential.

The post-test and questionnaire were both developed using Questionmark Perception™ (Questionmark, UK), a well-established suite of software for authoring Web-based assessments and surveys. The post-test was based around two case studies, which differed in both content and style from those presented in the module. The format of the post-test was similar to that experienced by all students in online formative assessments in earlier Phases of the program, and included standard multiple choice (select the single best answer), true/false, multiple response, and extended matching questions aligned with the case studies.⁶ The questions reflected the clinically-oriented style of learning in Phase 3 of the Medicine program, and were based on learning objectives publicised to all students via the Phase 3 Biomedical Sciences student manual. Students were presented with hematology and immunohistochemistry results, and histology and radiology images, as appropriate for each case, and asked to answer questions relating to diagnostic investigations, provisional diagnosis, and staging. In addition, all students were asked at the conclusion of the post-test to indicate how many hours they had spent studying the topic of lymphoma during the trial period.

The questionnaire obtained feedback from the Study group regarding their perceptions of the module as a learning resource. Students were asked to rate the module design and content, as well as providing free text responses for most valuable features, and suggestions for improvement.

Statistical Methods

A t-test was performed to compare previously demonstrated academic performance (as measured by Weighted Average Mark – WAM) between groups. Paired t-tests were performed to evaluate whether access to the Web-based lymphoma module affected the outcomes of the post-test. Linear regression analysis was employed to determine whether there was a relationship between time spent studying lymphoma and the post-test scores, as well as to determine if there was a relationship between the student WAM and post-test scores. The Mann-Whitney test was performed to compare the Study group’s ratings of the perceived difficulty of the topic of Lymphoma before and after access to the web-based module.

RESULTS

Of the 11 students allocated to the Control group, eight completed the post-test (73%), while 10 of the 11 students allocated to the Study group completed the post-test (91%). One post-test score in the Study group was excluded from the statistical analysis, because that student was unable to access the module during the period of the study due to a family bereavement, reducing the Study group to nine (completion rate of 82%). The measure of previously documented academic performance (WAM – Weighted Average Mark) for participants in the Control and Study groups was almost identical (Control: 72.96 ± 1.645; Study: 73.70 ± 2.013) (p=0.782).

The Study Group scored significantly higher on the post-test (mean 20.22 ± 1.246, the maximum score being 26) compared to the Control Group (mean 15.88 ± 1.199) (p=0.024) (Figure 3). Expressed in percentage terms, the mean scores for the two groups were 77.9% and 61% respectively. Several factors were examined to assess possible contributions to the results observed.

The Study group reported spending an average 1.1 hours more than the Control group studying the topic of lymphoma, but this difference was not significant (p=0.394). There was also no significant relationship between time spent studying lymphoma and the post-test scores for either group, determined by linear regression analysis (Control r2 = 0.099; Study r2 = 0.05). Differences in WAM accounted for 54% of variance in the Control group, but only 21% of variance in the Study group.

Participants in the Study group rated the perceived difficulty of the topic of lymphoma before and after the study on a 10-point scale (1 = least difficult, 10 = most difficult). There was a significant decrease in perceived level of difficulty from an average 8 to 6.3 (p=0.034, Mann-Whitney test). Student feedback regarding the module obtained via an online survey was overwhelmingly positive. The responses to the Likert scale items are summarised in Table 1.

Examples of features that students found most helpful included the case studies, questions with “roll over” answers, interactivity, definitions of key terms provided on the screen that such terms appeared, navigation tools that allowed return to previous screens, simple tables, histopathology slides, and the use of audio and visual stimuli, rather than plain text. Students also commented that the revision of basic science concepts in the Causes and Concepts section of the module was helpful. Below is a selection of comments from the participants:

“This was really fantastic and has made lymphoma a lot simpler in my head, and has motivated me to read up further on the topic. I am definitely a lot more confident about lymphoma after completing the module and quiz.”

“I also liked how it covered the aspects of pathology that were interesting and relevant to the clinical scenarios we will be faced with in the future.”

Students also made suggestions to improve the module. Some students found the length of the module made it difficult to maintain focus and interest throughout. Creating multiple shorter modules was suggested by several participants. While students appreciated the self-test questions throughout the module and agreed that these questions helped motivate them to think more carefully and purposefully about the content, they also saw room for development. Graded formative assessments were seen as a helpful tool, as they helped to “highlight mistakes more clearly”, whilst acknowledging this was “possibly because students are always obsessed with marks”.

Several students noted that the time taken for initial downloading of the module was quite lengthy, which is an important consideration with Web-based resources, particularly if the module is to be used in rural areas where bandwidth is often lower than in urban centres. However, the majority of students felt that a Web-based module increased accessibility, compared to modules delivered by CD-ROM.

Generally, students preferred less information to be provided by audio and suggested several improvements to the audio function, including addition of a visual display or indicator of the audio for each slide, so the user can judge what proportion of audio remains, and ensuring that the main points made in the audio recording are related to the screen displayed during play.

DISCUSSION

This randomized controlled trial has demonstrated that the use of a Web-based learning resource on lymphoma improved student objectively measured understanding of the topic, as well as decreasing perceptions of its difficulty. Considered in isolation, the trial is limited by the small number of participants in this study. However, because of the significant difference in scores between the Control and Study group, it was considered inappropriate to continue recruiting students to participate in the study, because the benefits for learning, particularly for a topic with a high perceived level of difficulty, were such that it would be unfair to continue to allocate students to the Control group. It should be emphasized that this study is one of very few randomized controlled trials of the learning benefits of CAL resources, thus the outcomes are of interest, despite the small cohort size.

***Image3:center***
Figure 3 is a scatter plot comparing post-test scores (maximum 26) for students in the Study and Control groups. *p<0.05, paired t-test.

The results of this study confirm the benefits for learning of appropriately designed CAL resources and reinforce our

earlier findings with regard to a CD-ROM-based module on glomerulonephritis.⁷ In contrast to our previous study, the present investigation employed relatively simple software tools (PowerPoint files imported into Adobe Captivate™) to build a Web-based module, so that time and attention could be focused on the content and educational value. Therefore in this respect it was a more effective approach for a content specialist seeking to develop and to disseminate a CAL module.

A common misconception relating to the implementation of CAL is that such modules can be used to completely replace existing face-to-face teaching sessions, thus alleviating staffing pressures. On the contrary, it has been demonstrated that rather than replacing current teaching strategies, CAL is highly beneficial when used in a supporting or complementary role.⁸ The focus of the online lymphoma module, and similar modules in Pathology at UNSW, has been to optimize student learning, particularly through improving the efficiency and value of student-teacher interactions. While this study did not directly examine this relationship, it has demonstrated that CAL modules used in private study can help to deepen student understanding of a topic and aid in connecting learning in basic sciences with clinical medicine. This foundation in turn serves to improve the focus and productivity of clinical experiences.

The lack of a significant relationship between previously demonstrated academic performance and post-test scores in the Study group was an unexpected finding. This is unlikely to be due to lack of content validity of the post-test, which was reviewed by two senior members of academic staff in Pathology, as well as a senior clinical hematologist. More likely, the significantly better scores observed in the Study group indicate that the lymphoma module was of assistance to both high performing and weaker students, and may even have been of greater assistance to weaker students. The potential value of CAL resources as remediation for weaker students is beginning to be explored. These results suggest that this could be a promising area for further development.

The length of time spent studying the topic of lymphoma was not a significant factor in the differences observed in the post-test scores, supporting claims that CAL leads to efficiency gains. Moreover, these data indicate that there were no significant differences in motivation to learn between the Study and Control groups. Interestingly, students in the Study group spent most of their time studying lymphoma in utilising the lymphoma module. One advantage of well-designed CAL resources is the ability to gather and present information from a wide range of sources, freeing student working memory to construct schemata for integrating concepts, rather than being occupied with searching for information.^5,9,10 In the Study group, the apparent preference for using the module over the traditional reference resources provided to both groups might be partly due to the capacity of the module to present salient information in a single interface that reduces cognitive load.

Interactive CAL modules are invaluable in motivating students to take ownership of their studies. Many design features promote this, including integration of learning objectives, self-tests, feedback and interactivity.⁵ In this module, students were given a degree of control over the amount of guidance they received with the option of labelled or unlabelled diagrams and histopathology slides, links to a glossary, and hints for working through cases. Students in the Study group commented that the module directed their studies on the topic of lymphoma. They also reported perceiving the topic as being less difficult after using the module, and commented that this further motivated and encouraged them as they continued to study.

Feedback and interactive learning are vital elements for mastering any topic, including Pathology.^4,11,12 Despite the recognised value of comprehensive feedback, its provision is often neglected because of the time-consuming, labour-intensive nature of individualizing responses, particularly with the surge in student numbers throughout higher education. Perhaps the most significant benefit of well-implemented CAL is the opportunity for automated, individualized feedback without the time or labour constraints of traditional methods, as previously demonstrated.^6,13 The flexibility of a CAL interface also allows concepts and explanations to be revisited and revised as required. Students in the Study group commented on this as being one of the most valuable features of the CAL module.

An important aspect of this study was to gauge the feasibility of, and student responses to, a Web-based module. The role of the Web in CAL is predominantly as a means of distribution. However, the additional features that can be utilised on a Web-based platform are attractive for developers of CAL resources. Hyperlinking to further learning resources is a common example. Keeping abreast of developments in scientific and medical knowledge is becoming increasingly taxing, as is maintaining relevant and accurate resources. In this context, the Web is an important platform for accessing and sharing information. This can have the unfortunate side-effect of increasing the burden on students due to the ease of access to copious amounts of unreliable information. Harnessing the Web to improve the efficiency of knowledge acquisition is an important target for educational resource development, particularly as reliance on technology increases in our society.

It was expected that students would comment on the lengthy initial downloading time of the module. However, a surprising response was the request for reduced audio content. In their review of the literature, Grunwald and Corsbie-Massay found that despite research showing improved long-term retention with audio presentation, students express a preference for written text.⁵ Student feedback in our study supports those findings, and further highlights the need for more research regarding the optimal integration of audio in CAL resources.

CONCLUSION

It has been reported that Web-based CAL is as effective as other teaching modalities when measuring knowledge gained by students.¹⁴ Considering the call of the World Federation for Medical Education to promote students’ ability to manage the information explosion (enhanced by current levels of Web accessibility) and ensuring competency with Information Technology, evidence for equal or greater efficacy compared to traditional methods of learning supports the rationale for further development and increased use of Web-based CAL resources.¹⁵ The results of this study are encouraging in that regard. This might be of particular relevance to medical curricula in the United States, where such resources could enhance the clinical curriculum with basic science content after the NBME changes the USMLE Step1/2 format. Larger studies of entire year entire cohorts would undoubtedly provide more robust data and possibly enhance the statistical validity of the conclusions, but the unambiguous evidence from this study has provided a sufficient basis for us to provide this module to all students.

ACKNOWLEDGEMENTS

The authors would like to thank all those who contributed to the production of the lymphoma module. In particular, we are grateful for the assistance of the following people:
• Faculty of Medicine, UNSW
– Dr Lindsay Hewson and Mr Colin Wood, Education Media Unit
• Haematology Department, Prince of Wales Hospital, Randwick
– Dr Gillian Rozenberg
– Mr Simon Cooper
• Department of Anatomical Pathology, Prince of Wales Hospital, Randwick
– Dr Kasim Ismail

REFERENCES

1. Greenhalgh, T. Computer assisted learning in undergraduate medical education. British Medical Journal. 2001; 322: 40-44.
2. Hays, R., Veitch, J., and Lam, A. Teaching clinical pathology by flexible delivery in rural sites. Australian Journal of Rural Health. 2005; 12: 232-235.
3. McNeil, H.P., Hughes, C.S., Toohey, S.M., and Dowton, S.B. An innovative outcomes-based medical education program built on adult learning principles. Medical Teacher. 2006; 28: 527-534.
4. Nash, J.R.G., West, K.P., and Foster, C.S. The teaching of anatomic pathology in England and Wales: a Transatlantic view. Human Pathology. 2001; 32: 1154-1156.
5. Grunwald, T., and Corsbie-Massay, C. Guidelines for cognitively efficient multimedia learning tools: educational strategies, cognitive load, and interface design. Academic Medicine. 2006; 81: 213-223.
6. Velan, G.M., Jones, P., McNeil, H.P., and Kumar, R.K. Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning. BMC Medical Education. 2008; 8: 52.
7. Velan, G.M., Dziegielewski, M., Killen, M.T., and Kumar, R.K. Development and evaluation of a computer-assisted learning module on glomerulonephritis for medical students. Medical Teacher. 2002; 24: 412-416.
8. Reid, W.A., Harvey, J., Watson, G.R., Luqmani, R., Harkin, P.J., and Arends, M.J. Medical student appraisal of interactive computer-assisted learning programs embedded in a general pathology course. Journal of Pathology. 2000; 191: 462-465.
9. Chalmers, P.A. User interface improvements in computer-assisted instruction, the challenge. Computers in Human Behaviour. 2000; 16: 507-517.
10. Kirschner, P.A., Sweller, J., and Clark, R.E. Why minimal guidance during instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist. 2006; 41: 75-86.
11. Marchevsky, A.M., Relan, A., and Baillie, S. Self-instructional “Virtual Pathology” laboratories using Web-based technology enhance medical school teaching of Pathology. Human Pathology. 2003; 34: 423-429.
12. Lacher, D. Using primary care Web sites in Pathology education. Pathology Education. 1999; 24:14-18.
13. Velan, G.M., Kumar, R.K., Dziegielewski, M., and Wakefield, D. Web-based self-assessments in pathology with Questionmark Perception. Pathology. 2002: 34: 282-284.
14. Chumley-Jones, H.S., Dobbie, A., and Alford, C.L. Web-based learning: Sound educational method or hype? A review of the evaluation literature. Academic Medicine. 2002; 77: S86-93.
15. World Federation for Medical Education (WFME). International standards in medical education: assessment and accreditation of medical schools’ educational programs. A WFME position paper. Medical Education. 1998; 32: 549-558.