Can We Really Rate Doctors and Hospitals?

In the simple marketplace, in Union Square’s Greenmarket, for example, you can look at the produce and may a reasonable judgment regarding its quality; the veracity of the sellers’ claims, organic or free-range are a given. But in more complex markets, like healthcare information is more imbalanced. Physicians know more, not just about medical care but about the more elusive character of quality. Patients have readily observed measures, network participation, a well-curated web presence, word of mouth recommendations from both their family and friends and most importantly their doctor. Quality has many parameters, how many patients have their cholesterol or HgA1c checked, how long is the wait in the office, and so on; but determining quality is less observable it is just not one thing. But by any measure, patients know less, and this can unbalance the activities of the market. Shoddy care, like lemony used cars, can be disguised, repackage and sold for more than it is worth. According to classical economics, this is a market failure; the information necessary to make fair exchanges, information the market should produce has faltered. It may be due to truthfulness, but more often there is no agreed upon standards for the metric of quality or its reporting. Failure may also be due to patients receiving multiple conflicting signals based upon varying parameters. How is the patient to respond when their primary doctor makes one recommendation, the quality report suggests another and the payer a third? Ratings by an impartial 3rd party are felt to correct for this lack of information.

The failures to disclose ‘relevant information,' especially in important markets, draws the government’s regulatory attention. While specific legal underpinnings vary, most federal regulatory authority comes from Executive order 12866, requiring federal agencies identifying market failure to introduce regulatory correction when the benefits outweigh the costs. Our patients’ inability to make ‘sufficiently informed’ choices triggers regulatory relief, frequently in an aggregate labeling of ratings. How hard can that cost-benefit weighting be?

Costs seem pretty straightforward - the cost of developing, measuring and promulgating labels. Or is it? There are less obvious costs. There is the cost of actually attending to the new information. In this information age, our attention is a scarce, costly resource, hence the competition for ‘eye-balls’ and web page clicks. And even ignoring ratings requires a small aliquot of attention.

Certainly, benefits are apparent and quantifiable. To gain the benefit of ratings, they must be used and influence behavior. Patient response to ratings is exceedingly difficult to describe, let alone anticipate. A benefit is a summation of net influences. We recognize the benefit of diagnosing cancer in a patient, allowing treatment, possibly cure. But what about those first days for the patient, when they move from the land of the well to the land of the cancerous - it is tough to see the benefit in that situation, seems more like a cost. Or consider choosing between two physicians when the ‘better’ one is 60 minutes away, or worse yet, not on your insurance plan. Choosing the benefit of the better doctor now comes with quantifiable costs. It is unfair to dismiss these considerations as unimportant and insignificant, especially when regulators assume rather than ask.

Healthcare ratings have a salience issue, when should you note and apply the information? When you are feeling well, there is little reason to choose your physician or to spend lots of attention in selecting your insurance coverage (providing you have that opportunity at all) – no salience. Our cognitive bias of today over tomorrow, ‘present’ bias, drives a lot of our healthcare decision, unpleasantries are postponed. When you need a physician, when salience is greatest, ratings may have the least benefit in guiding behavior. You go to the nearest ED, not the highest rated in the city. How do you know choosing a ‘better’ doctor improves your outcome? Prior performance does not guarantee future success, at least completely.

Regulators designing ratings face these issues of defining and balancing costs and benefits, it is required by Executive order 12866.  In achieving this goal, there are several options each with their burden of “information-gathering demands.” Regulators could demonstrate economic savings or improved health care outcomes through research a difficult, time-consuming and expensive process. Moreover, healthcare outcomes are dynamic; a one-time analysis is insufficient. In the process of ‘monetizing’ benefits the opportunity to inflate benefits and make unwarranted assumptions, when necessary information is unavailable, is great. And aggregated numbers hide personal losses and gains. Regulators, assuming significant benefits will accrue, can apply a ‘breakeven analysis,’ asking whether benefits justify a pre-determined fixed cost. Even marketplace solutions, asking what price people are willing to pay for the information at the point of greatest salience and efficacy, is limited by present bias and payer constrained choices. Or regulators may simply confess an inability to gather the pertinent information, their decision while not quantified is their ‘best approximation.’ But this is a ‘stab in the dark’ perhaps informed with the light of expertise, but none the less uninformed.

The rating metric may not adequately serve our patients, but perhaps they provide a higher purpose. There is a moral argument to be made, that ratings by revealing the ‘better’ enhance equity of distribution, removing the ‘who you know’ aspect. Ratings could increase your universe of choices to include physicians previously unknown let alone acknowledged. But if distributional equity is the motivation, where is the proof that ratings provide this outcome. Ratings differentiate physicians that is their purpose. But can’t those ratings be used to tier services, used by payers to shift costs? We see it with generic drugs, how about generic doctors. If you want a brand name doctor, it simply costs more. Where is the equity and fairness in that?

Ratings are here to stay; they can provide useful information. But as physicians we have a dual responsibility, to understand and engage in the development and implementation of ratings but more importantly to do the due diligence, the research, to show whether this useful information is just that, useful. If we do not, then three and four-star doctors are only a few years away.

The impetus for this article was a recent paper in the University of Pennsylvania Law Review by Cass Sunstein on Mandatory Labeling. I am indebted to his ability to organize the topic in a studied, meaningful way. Any errors in presenting some of his thoughts rest solely with me.