The Impact of Online Lecture Recordings on Learning Outcomes in Pharmacology

INTRODUCTION

Interactive and/or active approaches to learning continue to be associated with better learning outcomes even through this era of e-Learning¹. Although the unfulfilled early promise of e-Learning is now being challenged in medical education² the convenience it offers to students, teachers, and institutional managers is overwhelming. For example, it is tempting to refer students to online recordings of lectures in cases where lecture theatres are overcrowded. Perhaps we should now be more discerning as to the role of online approaches so that they are tailored for student cohort characteristics (for example undergraduate vs graduate), course types (for example science vs medical/dental) as well as individual learner characteristics (including learning styles and cognitive approaches)^{3, 4}.

Lectopia was developed in the Arts Multimedia Centre, University of Western Australia (UWA) to provide flexibility in teaching and learning⁵ and allows students to access lecture recordings online at their convenience. Surveys conducted by the Arts Multimedia Centre report that students use Lectopia to access lecture recordings online for a number of reasons: to revise and clarify lecture material; to manage timetable clashes and missed lectures; to balance work and/or family commitments with study demands; to address difficulties in attending face-to-face lectures due to time taken to travel to and from university or residing in a regional area. Some students simply preferred to use Lectopia rather than attend face-to-face lectures⁶.

Online lecture recordings via Lectopia have been used in second year Pharmacology teaching at UWA since 2005. Thus, audio recordings are made automatically and streamed over the web via an enrolment-limited login access. In fact, Lectopia now supports learning and teaching university-wide including all Pharmacology teaching. From a teaching perspective, the intention in the Pharmacology programme was not to replace face-to-face lectures with Lectopia, but rather to provide an additional tool for students to access lecture material for revision and clarification. However, online lecture recordings might also be used to compensate for large class sizes and timetable clashes. The present study considers the impact of the availability of online lecture recordings on both the educational environment and learning outcomes in second year Pharmacology.

The use of Lectopia and other online lecture services has increased due to overwhelming support from students. Given their age, second year Pharmacology students are likely to embrace Lectopia technology as part of a learning strategy at university, without necessarily considering its potential pitfalls. Historically, no more than 3-4% of students failed these units. Since the introduction of Lectopia, the failure rate has increased, perhaps coincidentally, to approximately 10%. While many reasons might account for this, does the drop in lecture attendance and/or Lectopia availability contribute to poorer learning outcomes in Pharmacology? By choosing not to attend lectures, perhaps students forego networking opportunities, teacher/student engagement and the full face-to-face lecture experience which are an important component of learning and scholarship at university⁷.

METHODS

Course structure and assessment

In second year Pharmacology, there are two lectures a week and several laboratory sessions in each semester. Continuous summative assessments are held during the semester and the material covered in these assessments is directly linked to lecture material, which may be accessed online. Attendance at laboratory sessions is compulsory and students submit a report to obtain credit for each session. Laboratory sessions are not recorded using Lectopia. There is, however, some overlap between concepts covered in lectures and laboratories. The exam mainly covers material presented in lectures, although laboratory-based material may also be examined. Thus, assessment in second year Pharmacology includes laboratories, summative assessments, an exam and an aggregate final mark for the semester. With respect to student grades, 50 – 59% corresponds to a pass; 60 – 69% a credit; 70 – 79% a distinction and 80 – 100% a high distinction.

Student survey

Ethical approval for this study was obtained from the Human Research Ethics Committee, University of Western Australia. This project complies with the National Statement on Ethical Conduct in Human Research (2007) by the National Health and Research Council (NHMRC) of Australia⁸.

An online, voluntary survey (shown in Table 1) was used to evaluate the impact of Lectopia on learning outcomes among second year Pharmacology students. Students were informed that the survey was not anonymous since student feedback would be linked to their performance in the course. Only the Chief Investigator had access to the database containing student names, numbers and responses to this survey. Hence, the level of confidentiality with respect to this database was greater than that used for student marks where administrative, examination and other university officers also have access to student marks. Student names were removed prior to statistical analysis. Student numbers were linked to learning outcomes for the purpose of analysis. Data obtained in this study was only analysed following ratification of student marks by the Board of Examiners.

Table 2 shows the response rate and gender proportions of the survey population. The survey instrument (Table 1), was designed to capture the student face-to-face attendance, their use of online lecture recordings and the reasons for their declared activity.

Statistical analyses

Assessment outcomes for each semester included laboratory reports, continuous summative assessments, exam and the final mark. Students were categorised into those who attended 0, 1 or 2 lectures per week and assessment outcomes were compared among these groups using one-way analysis of variance and post-hoc pairwise comparisons with Bonferroni adjustment. Students were also grouped into those who used Lectopia instead of attending lectures 0, 1 or 2 times per week and assessment outcomes were compared among these three groups (one-way analysis of variance with post-hoc Bonferroni-adjusted pairwise comparisons).

Lecture attendance and the use of Lectopia instead of attending lectures were compared between genders using the Pearson Chi-square test of equal frequencies. Student marks obtained in various assessments were compared between genders using Students’ t-test.

Statistical analyses were conducted using SPSS 15.0.0 for Windows (SPSS Incorporated, Chicago, IL). The criterion for significance was p < 0.05 and comparisons yielding p < 0.1 are reported here as trends.

RESULTS

Impact of lecture attendance

Students were sorted into three groups, depending on whether they reported that they usually attended two, one or no lectures per week (Figure 1). The general trend observed for marks from laboratories, summative assessments, exam and the final mark was that the more lectures the student attended each week, the higher the mark. There was no significant difference between marks obtained in laboratories by the three groups of students. However, the trend seen with laboratory marks suggests that weaker-performing students might be less likely to attend lectures (p < 0.1). Non-attendance at lectures was associated with lower marks in continuous summative assessments held during the semester. Students who reported that they usually did not attend lectures had a significantly lower mark than students who attended both lectures (no lectures: 54.1 +/- 4.5%, both lectures: 69.8 +/- 0.9%; p < 0.003, Figure 1). There was also a significant difference between marks obtained by students who did not attend any lectures and those who usually attended one lecture per week (no lectures: 54.1 +/- 4.5%, one lecture: 68 +/- 2.1%; p < 0.03, Figure 1). Indeed, on average, students who attended both or one lecture(s) per week obtained a credit mark (69.8 +/- 0.9% and 68.0 +/- 2.1%, respectively), while students who did not attend lectures obtained a pass mark (54.1 +/- 4.5%) with respect to summative assessments. While there tended to be a trend for lower exam marks in the group who usually did not attend lectures, this failed to achieve significance (no lectures: 56 +/- 7.6%, both lectures: 67.3 +/- 1.0%; p < 0.1). Non-attendance at lectures was associated with a significantly lower final mark compared with the group who attended both lectures per week (58.9 +/- 4.8%; 70.1 +/- 0.8% respectively; p < 0.03). Thus, on average, students who attended both lectures obtained an overall distinction while students who did not attend lectures obtained a pass mark in the unit.

Impact of using Lectopia instead of attending lectures

Students were divided into three groups depending on how many times per week they usually used Lectopia instead of attending lectures (Figure 2). Whether or not Lectopia was used instead of attending lectures was not associated with any significant change in laboratory marks. Since laboratory sessions were not recorded using Lectopia, this suggests that these groups were similar with respect to their ability to write laboratory reports. However, there was a general trend observed for marks obtained in summative assessments, exam and the final mark such that the more often Lectopia was used instead of attending lectures, the lower the marks obtained. Importantly, using Lectopia instead of attending both lectures per week was associated with significantly lower marks in summative assessments, exam and the final mark compared with not using Lectopia instead of attending lectures (using Lectopia instead of attending both lectures: 61.4 + 2.7%, 58.6 + 3.9% and 63.0 + 2.7%, not using Lectopia instead of attending lectures: 69.5 + 1.1%, 67.7 + 1.2% and 70.3 + 0.9% respectively; p < 0.05, Figure 2).

Student comments regarding Lectopia use

Student comments regarding their use of Lectopia were categorised into: revision; clarification; timetable clashes; missed lecture; don’t use Lectopia; work/workload; prefer Lectopia (Figure 3). Some students listed more than one reason for using Lectopia, providing a total of 292 items. The major reasons reported for using Lectopia were for revision and clarification of lecture material. As expected, timetable clashes were an issue and often cited as a reason for using Lectopia. The issue of timetable clashes is currently a significant one at this university. Students who enrol in combined degrees as well as units of study at different year levels often have timetable clashes. The number and proportion of students who reported timetable clashes as an issue varied from one semester to another. Since recordings of their Pharmacology lecture were available online, students may therefore have been implicitly encouraged not to attend their conflicting Pharmacology lectures. Furthermore, where another unit had repeat lectures or practical sessions such that it would be possible to avoid timetable clashes, students may have selected time slots that clashed with Pharmacology lectures. Since Pharmacology lectures were recorded on Lectopia, there appears to have been a perception by such students that this would save time. Students also used Lectopia if they missed a lecture for other reasons. During the course of this study there seemed to be an increase in the proportion of students who reported using Lectopia for this reason. Students who may have become more familiar with and reliant on Lectopia may have been inclined to miss lectures and depend on Lectopia instead. Thus, Lectopia appears to have provided a convenient contingency such that missing a lecture was seen as less of an issue by students than if this service was unavailable. Some students reported that they did not use Lectopia. A small number of students reported using Lectopia due to work or workload issues or because they simply preferred Lectopia to attending face-to-face lectures.

Effect of Age and Gender

The majority of students who enroll at the University of Western Australia do so directly after completing high school in this state. For example, approximately half of the entire cohort of undergraduate students were 19 years and under as at December 31, 2006⁹. As such, most second year Pharmacology students are between 18 and 19 years of age.

Taken as a whole, females achieved significantly higher mean scores in laboratories, summative assessments, exam and the final mark (Table 3, p less than 0.02). Indeed, females obtained a grade higher than males in all assessment modes, regardless of lecture attendance and Lectopia use.

Overall, 78.5% of students reported that they attended both lectures each week, with the proportion of females (77.2%) and males (80.4%) not being significantly different (X²=0.346, p greater than0.05). Among the group who attended both lectures each week, females scored significantly higher than males on laboratories, exam and the final mark (Table 4, p less than 0.01). Indeed, females obtained a mean higher grade than males in laboratories and the final mark (high distinction and distinction versus distinction and credit, respectively). However, among those attending less than two lectures per week, there were no gender differences in any of the performance indicators. This may have been due to the small size of this sub-sample.

Furthermore, 36.4% of the sample used Lectopia rather than attending lectures at least once per week. The proportion of females (38.7%) and males (33.3%) who used Lectopia instead of attending lectures at least once a week, was not significantly different (X²=0.507, p greater than 0.05). Among those that did not use Lectopia instead of attending lectures, females scored higher than males in laboratories, summatives, exam and the final mark (Table 5, p less than 0.05). Again, females obtained a higher grade than males in these assessments (high distinctions and distinctions versus distinctions and credits, respectively). However, among those that used Lectopia instead of attending one or two lectures a week, there were no gender differences in any of the performance indicators.

***Image7:left***

DISCUSSION

As reported in Hamdi¹⁰, we have demonstrated that attending face-to-face lectures is important with respect to learning outcomes in Pharmacology. Similarly, Gatherer and Manning¹¹ reported that lecture attendance correlated well with examination performance in the biological sciences. In any science where explanations of mechanisms, using diagrams and figures and experiential engagement between teacher and student forms an important part of learning, simple online audio recordings may not be adequate. Indeed, we have shown that regular non-attendance at lectures was associated with lower marks in summative assessments and the final mark. These data also show a benefit of attending at least one lecture a week, perhaps by helping the student to keep in touch with curriculum content, teachers and peers through networking opportunities. This also allows students to gain the benefit of attending at least half the lectures delivered in Pharmacology. Similarly, the findings suggest that the regular use of Lectopia in lieu of attending lectures adversely affects learning. Using Lectopia to replace no more than one lecture a week appeared less damaging than replacing both lectures. In the present study, there was no evidence of a relationship between student ability (as measured by their weighted average mark) and lecture attendance (data not shown). This suggests that student ability may not be used to predict lecture attendance in Pharmacology. While the relationship between lecture attendance and student performance is still unclear, missed networking opportunities, diminished teacher/student engagement and the difference between online audio recordings and a face-to-face lecture experience offer possible contributing factors.

Online lecture recordings can play an important adjunct role for revision and clarification. If certain concepts are unclear, students can revisit and replay material over and over again until they “get it”. However in this study, students also reported using Lectopia due to timetable clashes which denied them the primary experiential “live” session.

That females performed better than males, when they usually attended both lectures per week and did not use Lectopia in replacement was intriguing. That this advantage was lost once Lectopia was used instead of attending lectures would support the engagement benefit of face-to-face lectures, although sample size may also contribute to the lost effect.

Here, students completed an online survey to report their own usual lecture attendance and use of Lectopia to replace lectures. Some students tended to over-estimate their lecture attendance, for example, students who were observed to be absent from lectures reported regular attendance in the survey. There was also a disparity between the number of students who reported usually missing both lectures (n=8) and those who used Lectopia instead of attending both lectures per week (n=18). Clearly, these discrepancies represent limitations of using self-reported data in the present study. It is possible that if lecture attendance had been independently logged, the differences between students who regularly attended lectures and those that did not might have been more marked, since non-attending students tended to perform poorly in assessments overall.

Attendance at laboratories is compulsory and almost all students comply with this. Lecture attendance in units of study within Business, Arts and Humanities is often compulsory and may be associated with course credit or penalty for non-attendance at a majority of lectures. In contrast, scientific disciplines at this university do not enforce lecture attendance due to potential equity issues such as timetable clashes and student disability. There are also practical considerations which make the collection of quality and accurate data on student attendance at each lecture potentially costly and time-consuming where large numbers of students are involved. While tempting, enforcing attendance at lectures in a university environment may even be problematic and counter-productive, with decreased student enrolment in Pharmacology as a result of these restrictions. An alternative might be to encourage attendance at lectures by making them more attractive to students. For example, a change from a didactic lecturing style to one that is more interactive and invites student participation has recently been shown to dramatically increase attendance at Pharmacology lectures. Indeed, one of the goals of a university education in the 21st century is to encourage critical and independent thinking, mastery of the subject, questioning and problem-solving⁷. In removing choice with respect to lecture attendance, we might deprive students of achieving these higher goals of a university education. Instead, Pharmacology students will be strongly encouraged to attend lectures, by providing them with the compelling data obtained in this study in support of face-to-face lectures.

REFERENCES

1.Romanov, K., and Nevgi A. Do medical students watch video clips in eLearning and do these facilitate learning? Medical Teacher. 2007; 29:490-494.
2.Harden, R.M. E-learning-caged bird or soaring eagle? Medical Teacher. 2008; 30:1-4.
3.Carbonaro, M., King, S., Taylor, E., Satzinger, F., Snart, F., and Drummond, J. Integration of e-learning technologies in an interprofessional health science course. Medical Teacher. 2008; 30:25-33
4.Maley, M.A.L., Harvey, J.R., de Boer, W.B., Scott, N.W., Arena, GE. Addressing current problems in teaching pathology to medical students: blended learning. Medical Teacher. 2008; 30:e1-e9.
5.Australia. Lectopia: http://www.lectopia.com.au [Access date: April 29, 2008]6.Arts Multimedia Centre. The iLecture system: guidelines and useful information. Crawley WA: The University of Western Australia; 2003.
7.Markwell, D. A large and liberal education: higher education for the 21st century. North Melbourne, VIC: Australian Scholarly Publishing. 2007.
8.National Health and Medical Research Council of Australia. National Statement on Ethical Conduct in Human Research (2007): http://www.nhmrc.gov.au/publications/synopses/e72syn.htm [Access date: 19 September, 2008]9.The University of Western Australia Statistics Office. Unistats. 2007: http://www.stats.uwa.edu.au/StatsOffice/unistats/2007 [Access date: April 3, 2008]10.Hamdi A. Effects of lecture absenteeism on Pharmacology course in medical students. Journal of the International Association of Medical Science Educators. 2006; 16:27-30.
11.Gatherer D., and Manning F.C.R. Correlation of examination performance with lecture attendance: a comparative study of first-year biological sciences undergraduates. Biochemical Education. 1998; 26:121-123.