Conducting quantitative research in medical education requires decisions to be made on several dimensions. First, a clear and detailed HYPOTHESIS needs to be formulated which identifies the key variables and the anticipated relationships among them.

Next, the STUDY DESIGN defines the sequence and timing of the interventions and outcomes measurement, as well as the allocation of subjects. All designs are compromises between the ideal and the constraints of time, resources, ethics, and the good-will of the participants. Some common problems in many medical education research designs are the lack of a comparison group and the difficulty in randomizing participants.

MEASUREMENT is one of the most complex aspects of quantitative medical education research. Although there are some commonly used measures of knowledge, attitude, and performance outcomes, much of medical education research centers around operationalizing the underlying constructs in which a researcher is interested. Not only must a researcher measure the dependent and independent variables, but must also measure variables and characteristics of the participants and the context that might influence the hypothesized relationships. All measurement must be concerned with issues of reliability and validity; these are not concepts relevant only to testing situations.

SAMPLING participants is another crucial aspect of research and care must be taken to ensure that the sample is representative of the target population. Sampling problems in medical education often stem from the tendency to use convenience samples and volunteers in our studies. Also, too many medical education research studies have an inadequate number of participants to provide adequate statistical power to detect a ‘meaningful’ relationship. Finally, sampling biases from non-random sampling, especially the use of volunteers can seriously limit the value of study results.

Once data are collected, they need to be analyzed. The complexities of selecting appropriate DATA ANALYSIS procedures can be boiled down to a few concepts: central tendency and variation, independent and dependent variables, and scale of measurement (nominal, ordinal, or continuous). All statistical procedures are analyses of central tendency and/or variation; the specific procedure depends on the number of independent and dependent variables and the scales of measurement these variables take. There are decision algorithms identified in the “Resources” section of this presentation that guide the selection of statistical procedures that are appropriate to your data.

When the results are finally disseminated to colleagues and the public, two recommendations are salient. One is to focus on EFFECT SIZE measures and CONFIDENCE INTERVALS rather than traditional statistical significance. Effect size measures quantify the magnitude of the relationship between or among variables. Confidence intervals define the precision of the estimate of these effects. Together, effect size measures and confidence intervals provide all the information contained in statistical significance and a great deal more. These two considerations are very valuable for making the results of quantitative research more useful in pragmatic decision making.

Finally, educational researchers should begin to include in their discussions and conclusions some attention to the COST-EFFECTIVENESS of educational interventions. When costs are factored into consideration, the meaning and practical significance of the results are typically enhanced.

View Slides from this Presentation