Using a scoring system for peer review

During the recent webcast, Peer Review: Seven Challenges and Realistic Solutions, speakers Robert J. Marder, MD; and Mark Smith, MD, MBA, FACS, answered a range of questions regarding data collection, peer review committee meetings, and process improvement. The following is their response to one of those questions:

Q: What are your thoughts on using a scoring system for peer review?

Dr. Marder: I believe in a three level scoring system: Care-appropriate, minor improvement opportunity, and significant improvement opportunity. I think you need to differentiate between whether everything’s OK, something’s a real big deal, or something’s in the middle but the physician can do better. In performance improvement culture, I believe that’s the best way to do it because you need to use this information eventually in the OPPE report, and if you only use a two-level system, you end up with people erring on the side of “no improvement opportunity” or “care appropriate” when there were some things could have been done better.

The scoreless system says it’s either appropriate or it gets sent to the department chair for further discussion and evaluation. It looks like it’s protecting peer review and it looks like it’s more collegial, but the reality is, it’s still a two-level system. If you roll things up into OPPE, you’re going to have only two items: either care appropriate or care not appropriate—and I think that’s patently less fair.
Having a scoring system where the committee is focused on three levels is fairer and it’s easier for the reviewers to do and focus on what the opportunities are. Just as important is deciding what the committee is going to ask the physician to do differently and better, and that’s where the improvement comes. I don’t think it comes from not from having a scoring system.

Dr. Smith: I have the same philosophy. On the flip side, in addition to not supporting a less than three-level system, I’m also not in favor of a more-than-three-level system. Some organizations have split the three levels, and they have subcategories, or five, six, or seven levels. The problem with that is you wind up having very log discussions that are splitting hairs about where [a situation] should fall, rather than focusing on the improvement opportunities that the particular situation presents.

I think the three-level system maximizes that opportunity terms of both recognizing it and allowing improvement to take place.