Limitations of Performance Measures

Medicare is gradually moving providers to a new, quality driven model.The current fee for service model used for so many years rewards providers for doing more: more procedures, more prescriptions, more admissions. The new quality model is designed to reward success. For example, fewer hospitalizations, fewer complications, or fewer adverse drug events.

One tool being used are the Star Measures released by Medicare. Many of these star measures are things that pharmacists can impact both directly or indirectly. Some of these have made their way into the EQuIPP measures now being used to provide pharmacies feedback on their quality.

Currently, pharmacies are not being assigned a star rating. Medicare Prescription drug plans (PDPs), however, are being assigned ratings, and these ratings come from the pharmacies in their network. The implication is that pharmacies that are hurting a plan’s star rating could potentially be terminated from their network.

The Pharmacy Quality Alliance (PQA), who manage the EQuIPP scores, sets pharmacy goals for each measure. These goals are revised every quarter. Pharmacies exceeding the goals set by PQA are helping the plan achieve a better score.  PQA also creates a “Top 20%” metric, allowing a pharmacy to benchmark themselves against all other pharmacies. Achieving a top 20% in even a single measure is non-trivial for most pharmacies.

Limitations

Make no mistake: the existence of the EQuIPP measures is a positive step in the transition of pharmacy toward the goal of increased quality. But like any program, there are some innate limitations to the current implementation of these measures. And while I fully expect these to be addressed as time passes, it is important to understand the current limitations.

The Current Measures are Surrogates

Currently, the measures being collected are not directly measuring quality or outcomes. Three of the measures are related to patient compliance–the percentage of days covered or PDC. It is easy to assume that better compliance will result in better outcomes, but the indirect measure makes many assumptions. There are dozens of reasons compliance might appear poor yet the patient actually is meeting their therapeutic goals.

Other measures make therapeutic assumptions. Should a patient with a risk of one disease be on a medication simply because the have another disease. The answer is maybe, but each case must be evaluated individually. One size does not fit all. High risk medications are another measure, and are equally challenging. Just because a medication is high risk does not mean that the patient’s outcomes will be worse on the medication than off of it.

The measures take this into account by setting the goals lower than 100%, but as will become evident below, this creates additional problems.

Limited Populations

The current measures represent only a fraction of Medicare Part D patients. With any statistic, a low sample size means that any change (adding or dropping one patient) can have a large impact on the measurement of a score. A store with 3000 total active patients might have as few as 20 Medicare patients fall into a measure group. Each patient effects the measure by 5%. The high side of the equation for a pharmacy of this size might only be 200 patients in a measure group. Over time, we expect more plans to add their data, which will help create more meaning full numbers. PQA reportedly takes this into account by not reporting scores for measure with very low population counts, though the scores are still calculated. I have one rural store with only 7 diabetic Medicare Part D patients represented. Our Performance score jumps around like popcorn every time it is updated.

Moving Target

PQA revises the 5 star goal numbers quarterly. This creates additional variability in a pharmacy’s scores. Undobtedly, the target scores will continue to rise with time, but there is a finite ceiling on the target. The closer the target score gets to 100% the more unrealistic the goal becomes.

Saturation

As time goes on, we would expect most, or even all pharmacies to meet the goals set for each measure set by PQA. At that time, the measure is essentially saturated. Either additional measures need to be added (making it more difficult to manage), the goals need to be increased (see above), or measures will be retired. Increasing the goal becomes problematic, as there will always be outliers, and as one approaches 100%, the goal becomes unrealistic: success becomes luck-of-the-draw. The option of retiring a measure is also problematic, as the success observed with time will simply trend back to baseline levels as pharmacies put emphasis on the new measure(s).

The Curve Effect

Imagine being a very bright student, at the top of your class. If you participate in a honors class filled with similarly gifted students, one would expect all students to perform well. Now imagine that class was graded on a curve: only the top 20% get an A. If the material is sufficiently challenging that achieving a score of 100% is not possible, then there are going to be a number of outstanding students receiving less than top marks in the class.

This is exactly what is starting to occur with several of the current measures. As pharmacies improve their patients’  compliance using a variety of techniques, the more and more pharmacies are reaching the 5 star level. Rewards for performance, however, are generally being tied to achieving a top 20% status. The curve is starting to compress. Increasing a pharmacy’s score on a measure only a fraction of a percent can lead to large jumps up the ladder toward the top 20%. The better everyone does, the harder it will be to achieve a top 20% rating. Like the case described in saturation above, success will eventually become luck-of-the draw.

The Long View

The current system of measures are a good start. With time, however, Medicare is going to have to revise the measures to better reflect outcomes. Additionally, the use of a top 20% metric for rewarding providers should be retired. Simple thresholds are more realistic. With time, quality measures will help pharmacy transition from a focus on product to a focus on care.