The dilemma of practice-based learning: How do you measure it?

Last month, we discussed the challenge that medical staffs face in understanding practice-based learning, one of the six general competencies. One of the reasons that medical staffs struggle with this competency, as well as the other five, is that they were developed by the Accreditation Council for Graduate Medical Education and American Board of Medical Specialties for residency training programs and board diplomates. However, when The Joint Commission adopted the six competencies as best practice, it did not elaborate on just what it expected from hospitals and medical staffs. When working with medical staffs during the past three years, The Greeley Company has been able to define four expectations for practice-based learning that resonate with attending physicians:
 

1. Regularly review your individual provider and specialty performance data for all general competencies and use those data to self improve
2. When contacted about patient care concerns, respond in a constructive manner
3. Use hospital information technology to manage performance data and access online medical information
4. Facilitate the learning of students, trainees, and other healthcare professionals
    

Based on these expectations, we can begin to consider how practice-based learning can be measured and evaluated to improve the physician performance and, ultimately, patient care.

The first expectation (review your data) is based on the premise that physicians should receive a routine feedback report with clear targets to define excellent, acceptable, and below-standard performance. Medical staffs should design ongoing professional practice evaluation (OPPE) reports to be given to the physician, not just placed in their file for department chairs to review. With that premise, a medical staff should expect physicians to use that information for self improvement. We also advocate that each indicator have targets indicating acceptable and excellent performance. Think of a pass-fail course that you took in school. You never knew whether your performance in that course was considered average or excellent. Setting targets for acceptable and excellent performance helps physicians interpret the data. By looking at the percentage of indicators on the OPPE report in the excellent category or the percentage that have shown improvement, one can measure the practice-based learning competency.

The second expectation (respond in a constructive manner regarding concerns) is useful for leaders when addressing physicians who respond inappropriately to letters of inquiry for case review or aggregate performance data. Many peer review committees have been taken aback when physicians respond inappropriately. By creating a rule indicator and targets for inappropriate or non-responsiveness to medical staff inquiries, one can measure this expectation. An alternative is for the medical staff to not measure this expectation, but simply refer to it when faced with a non-cooperative physician.

The third expectation (use information technology) is becoming increasingly important with the nationwide push to implement electronic health records. In addition to providing electronic storage and transmission of patient care information, these systems also provide valuable prompts and clinical information for clinical decision making. At a minimum, the medical staff can measure whether physicians receive training on the use of these technology systems. Medical staffs may also want to provide physicians with feedback regarding how frequently they access online resources or and how many times their clinical recommendations are overridden based on evidence-based medicine.

The fourth expectation (facilitate learning) helps remind us that other medical professionals look to physicians to gain medical knowledge. Although we may think we are fulfilling this expectation, our performance in this area is best determined by others’ perceptions. Surveys are the best tools to evaluate those perceptions and quantify them. Residency and medical student training programs already have data in which students evaluate attending physicians, and that data can be used for OPPE to measure this expectation if your have such programs in your facility. If not, or if not all physicians in your facility are involved with formal teaching responsibilities, including a question regarding physicians’ willingness to foster education in a staff survey will provide data that is useful in measuring this expectation.

I hope that the above discussion provides you with the basis for evaluating practice-based learning.  I am sure some of you may have defined other expectations or measures for this competency. If so, please share them with me at rmarder@greeley.com; I would be delighted share them with others.

Robert J. Marder, MD, CMSL, is the vice president of The Greeley Company, a division of HCPro, Inc., in Marblehead, MA.