LOOKING AT THE QUALITY OF CASE-BASED INTEGRATED THEORY ASSESSMENTS

Hettie Till*, and Rahim Karim, Undergraduate Education, Canadian Memorial Chiropractic College (CMCC), Toronto, Ontario M2H 3J1 CANADA

Purpose

A previous poster outlined the development of three case-based theory exams at the end of each module of Years I and II of our undergraduate program and combining the results of these three exams into an overall theory percent score.  The purpose of this poster is to describe the quality of these integrated assessments as experienced over the last two academic years.

Methods

Analyses of item difficulty and point biserials of all the extended matching questions were conducted and poorly discriminating questions referred to the relevant faculty for review and possible correction.  On the final number of questions in each paper, descriptive statistics as well as measures of internal consistency were calculated.  Construct validity was explored through correlations with other assessments in the same year, and through backwards stepwise linear regression with the modular results as dependent variables and the previous year’s assessment results as independent variables.

Results

Pearson’s correlations between modular theory results were all significant and between 0.75 and 0.87.   The modular theory results also correlated significantly with other assessments in the same year of study – strongest with the Biological Sciences OSPEs, Radiology OSPEs and the end of year OSCE.   Backward stepwise linear regressions all reached convergence in 7 iterations or less and between 69% to 74% of the variance in the final modular theory results was explained by the independent variables.

Conclusion

The fairly sound psychometric proportions of the integrated assessments support the high-stake decisions that we have to make on the basis of examination results.