The purpose of this session was threefold: discuss common problems with clinical performance ratings (CPR), explain steps necessary to judiciously evaluate problem learners, and describe strategies for enhancing CPR.

The quality of performance ratings are determined by their accuracy, reproducibility, generalizability, and validity.  The main sources of errors in CPR systems include the raters, the performance rating SYSTEM, and the rating form itself.  Problems associated with raters vary, but sample problems involve raters:

 evaluating behaviors they didn’t observe, or don’t remember observing
 not using the full scale but rather being hawks (rare) or doves (most common)
 not wanting to record negatives
 rating a learner high or low in all categories rather than discriminating among the different categories

Clinical performance rating systems need to be administered with attention to detail.  The who, what, when, how, and so what questions associated with any system should be documented and implemented as such.  Examples of problems include:

 tardy forms or no forms completed
 lack of follow up when negative ratings or comments are submitted
 insufficient number of raters to truly generalize performance
 insufficient attention to due process guidelines

And lastly, examples of problems associated with the rating form include:

 too many items on the form
 no indication as to the extent of observation by the faculty member
 no global rating scale to capture “gestalt” judgment of faculty member

These lists are not exhaustive but represent many of the weaknesses in clinical performance rating systems.

Faculty should be educated on how to detect common symptoms among problem learners and how to effectively intervene.  An impaired learner can have psychological, substance abuse, or physical illness problems.  It is critical that faculty document noted problems and submit their written concerns to the clerkship or program director.  If communicated verbally, the education administrator should document the date and time of the conversation.  Preventative measures such as having a meaningful mentor/advisor system, a critical incident report system, and clearly documented expectations for the learners are helpful.  The key guidelines are to:

 Document changes in personality, performance, or physical appearance in a timely way
 Provide clear and consistent communication, both verbal and written
 Due process must be afforded
 Intervene early
 Protect the learner’s right to confidentiality
 Be aware of your institution’s policies for addressing problem learners.

Education administrators can enhance their clinical performance ratings by taking several steps.  These steps are nicely spelled out in a paper by Dr. Reed Williams and colleagues entitled “Cognitive, social and environment sources of bias in clinical performance ratings” published in Teaching and Learning in Medicine, 2003.   The authors offer a list of suggestions that should be considered when aiming to hone your clinical performance evaluation system.

It is a difficult but critical responsibility to evaluate our learners in the clinical environment.  There are challenges to implementing a fair and accurate performance evaluation system in the busy and complex hospital environment.  But we can hone our ability to judiciously and accurately evaluate our learners with adequate attention to 1) educating our faculty raters so to ensure adequate calibration and cooperation,  2) planning and documenting a sound performance evaluation system, and 3) having in place procedures for appropriately addressing problem learners.