Good Metrics Back on Track

Mark Jaben, MD

Jaben 2014For whatever reason, you want to reduce your door-to-doc times (D2D), currently an average of 25 minutes.  Perhaps you feel this is important for patient satisfaction, or the CEO wants to follow this metric, or your quality department uses this for reporting, or there is a billboard advertising this to the public. What will your first step be?

You can make your best guess, but if you are honest, you cannot really tell from this piece of data alone. We make a correlation- what we think is happening-and act on it, without knowing if it is the cause or not.  Our brain works by making a plausible explanation, looks for only the supporting data, provides what it thinks is the right context - whether that is actually happening or not- and off you go.

What we have confused is data with knowledge. Knowledge includes not just the data, but its context and understanding the implications of that context- the information that would enable us to make our best decision in the given circumstances. Data alone is not enough, and being data driven is not enough.

To get 'Good Metrics Gone Wrong' back on track, we need the data to be the 'right' data: capable to provide insight into the question at hand, and, for improvement, it must be believable: gathered acceptably and verifiable by those affected by the data. It is not about the data; it is about how we use the data.

A median D2D of 25 minutes is consistent with a D2D anywhere from 2 to 200 minutes (actual experience). Your D2D at 6pm likely tells a very different story than your D2D at 6 am. The median or average gathered over 24 hours or longer is not capable of provide any insight, yet the issues that determine D2D at each time are likely very different.

The range is much more insightful. Better yet, plot the D2D's for every patient over a 24 hour period with time in minutes on the x axis and the number of patients for each minute interval on the y axis. What you get is an asymmetric bell shaped curve with the majority of patients clustered around your 25 minute mark, but with the right end of the curve trailing off towards 200 minutes.  These patients had a very different experience. Learn what happened with these patients and you will begin to learn the cause of your D2D problems, and very likely, your patient satisfaction issues.

We may have to use median D2D time for reporting/marketing, but that does not mean it is the 'right' data for improvement. Data gives us the objective feedback we need to overcome the plausible explanation our brain jumps to, enabling us to discover the cause and adjust what we are doing, so we can learn, hopefully, to do it better.

Good metrics can only fulfill this role if they represent the 'right' data, and only if we use them credibly. It is not about the data, it is about how we use the data. And for successful improvement, how we decide on the data we use and our ability to successfully use it depends on the credibility of our relationship with those affected by that data. The loss of credibility and trust that results when we do otherwise often derails even the best intentioned improvement efforts.

Back to Newsletter

Feedback
Click here to
send us feedback