Join Section

Engagement Part 2- ED Metrics - Quality Improvement and Patient Safety Section Newsletter, September 2011

Engagement Part 2 - ED Metrics- An Effective Strategy to Kill Motivation
Mark Jaben, MD 

markjabenIn part one, I tried to advance the observation that RESULTS <-- LEARNING <--- ENGAGEMENT <-- RELATIONSHIPS. A relationship is based on mutually beneficial goals, attainable by understanding what each party believes they need to be successful in their work. This creates a foundation for engagement, which is essential to set the stage for the learning that leads to results.

Too often, we jump straight to the results without regard for the essential prerequisites and then wonder why people just don’t get it or won’t do it. In our zeal, we create metrics that seem appropriate, but rather than assist in promoting the engagement we need, these inadvertently do no more than enforce compliance, where people just do the bare minimum not to get punished and are not willing to invest themselves.

Here are some examples.

1) LOS (length of stay) - the anti Lean metric
What could be more Lean than lead time measurement, until you ask the question: lead time for what? How do I know where to start to address a 30 minute increase in LOS last month?

In emergency medicine, there are really three kinds of patients:  

a. Straightforward- few possible considerations, all of low risk; the question to be answered and the path for evaluation are clear and involve few possible tests. Such patients might have an ankle injury, laceration, urinary symptoms without abdominal pain, sore throat

b. Complicated- few possible considerations but some of higher risk and many more options of tests that might help guide care, but the question to be answered and the path for evaluation remain clear, such as patients with chest pain, SOB in a COPD patient, abrupt onset of focal neurologic symptoms, headache

c. Complex- the question to be answered and the path forward are unclear, with low or high risk potential, and requiring extensive testing and care to figure out the correct path, such as abdominal pain in a young woman or in the elderly, general weakness, dizziness.

Each category requires different resources, different staff, and different amounts of time to make decisions that are safe and appropriate, yet ED’s care for all three at the same time. Unless you look at lead time for each separately, how can you know where to act to achieve improvement? Moreover, when the measurement is reported days or weeks after the fact, the particular circumstances that led to the increase in LOS have been lost, making it even more difficult to figure out what to do. And if you don’t understand this measurement within the expected statistical variation that any measurement will experience, you might be misled into acting not only in ways that don’t address the real cause, you might act when there is no real reason to act.

2) Patient satisfaction-
A score of 95% would make most institutions pleased, but it really means 1/20 patients are not satisfied. If an ED sees 20,000 patients a year, roughly 60 patients a day, that means 3 people a day are dissatisfied! If your ED is busier, you can do the math yourself. Now 19/20 does seem pretty good, but often this means that a squeaky wheel gets all the attention. Once a prominent person complains, we focus there, not on the other 95% and miss the opportunity to ask what about our current care process is not working for everyone?

Patient satisfaction is important because it connects our work to what matters for patients. But, like average ED LOS, it is impacted by way too many people and processes to serve as a guide to what needs to be done for improvement. And like LOS, to be useful it must be broken down into the component pieces that can then be analyzed and acted upon. The real task here is to understand just what the component pieces are that must be in place to achieve patient satisfaction, a discussion that must begin with learning patient needs and perceptions and then involving everyone with a hand in the process to learn how best to educate patients to the realities of what can be done and what should be done, while reconciling with them how best to serve those needs and deciding together what will be done.

3) Triage in 15 min 80% of the time
Despite being an attempt to give people a break, this actually blames people. Why? Because it says we acknowledge there are circumstances out of your control, so we won’t hold you accountable. Quality is an inductive process- we look at an outlier event and try to draw a conclusion as to what led to this happening. Operations are deductive; we start from the principles of how to make things flow and then design the circumstances to conform with this. We should be asking how the process could accommodate all the circumstances all the time. 

‘Holding people accountable’ is really code speak for forcing someone to do what you want. This ignores the reality that your solution, in all likelihood, makes it harder for that person to be successful in their work, i.e., it just does not work for them. A problem is like a prism with each facet only visible to one person. To fully realize the true extent of the problem requires learning each of its facets. Crafting a solution that actually responds to the real issue and that can be acceptably implemented relies on this knowledge. ‘Holding people accountable,’ that is managing for compliance, ignores the reality that people decide how accountable and involved they want to be. Managing for engagement, not compliance, creates the atmosphere where people are willing to invest themselves in the creative, conceptual, decision making work required to provide high quality healthcare.

4) ‘The ED is not profitable’ and ‘Our ED is the doorway to the hospital’
We mislead ourselves when financial information is used to make operational decisions. So often, current accounting methods isolate areas financially and obscure the reality that those areas are quite interconnected operationally. Like LOS and patient satisfaction, financial results are a performance measurement, the consequence of how the operation is designed, and should be used to drive a search for how to improve operations. This is done by increasing capacity, not decreasing the ability to get the work done well. This increased capacity allows for decisions about how to best use capital, by either increasing resources to increase capacity further, leading to increasing volumes and margin, or accomplishing more with the same resources, resulting in cost savings. This takes a longer term view of what financial success means and rightly focuses attention on what it takes to get the work done well.

These are close cousins to ‘People are our greatest asset.’ If this is so, why are salaries for staff listed as an expense rather than a fixed asset? Reducing training and cutting staff is so often the default response to achieve short term cost saving. These are prime examples of using financial results to make operational decisions that decrease capacity and have the unintended consequence of making it harder to get work done well. Acknowledging the interplay between financial results and operational decisions is crucial for successful healthcare delivery.

5) Door to triage time; door to doc time- ‘better hustle to see those patients’
Contrary to popular wisdom, patients don’t come to the ED to ‘get triaged’ or to ‘see’ the doctor. They come to get advice and treatment about their problem. They come to have their uncertainties addressed. These measurements carry the risk of optimizing a part of the process at the expense of the entire process. These should be used as learning measures, not as performance measures, to help understand the obstacles in the downstream process, where the road block really exists.

6) Patients per hour/provider-
Although this seems a reasonable measure of productivity, it actually detracts from promoting the kind of collaboration needed for improvement to take hold and rewards thinking isolated to ‘me,’ rather than to ‘us.’ ‘Oh, I’m above average; I’m OK. Poor guy over there is always so slow; he makes my job harder.” Furthermore, people can hustle a little, but beyond a certain level, this only invites mistake, error and frustration, as people work beyond their comfortable capacity.

Measuring patients per hour by provider really measures the patients per hour for the department at that time. Some might say that the other variables are controlled by averaging across the number of shifts each provider works. It is very difficult to isolate these other variables, and although this assumption is convenient, this may be true, but it may not be true, as well. More importantly, what we need are people willing to critically look at their individual work flow and how this contributes to the system. If people are being held accountable for things out of their control, they will not be willing to engage in this critical refection and learning.

Until we design incentives and rewards that focus on this as the measure of success for the department and for individuals working in the department, it is unlikely people will be comfortable or willing to look critically at their contributions individually, how their work impacts those they work with, and how their work raises or lowers the comfortable capacity of the department. This will more likely happen when individuals are willing to assess and acknowledge the difficulties they are having, and the system commits itself to supporting their efforts to learn how to overcome these obstacles.

We don’t engage people by judging them on operational metrics. Judge the process by operational metrics, not people. Judge people by whether they can apply the skills, learning, and decision making to fulfill the parameters of success, those identified as necessary to do the work well. These become the standards against which performance can be judged. Self assessment of current performance against these criteria requires appropriate feedback. Current operational metrics include much that is out of the control of individuals, so they just won’t believe in these as reasonable feedback.

People know where they have difficulties. They just are rarely placed in situations where they can comfortably acknowledge these to themselves or to those in supervisory positions. Engaging people to judge their own performance takes advantage of everyone’s interest to be better at what they do. Training to the gaps and feedback make people your greatest asset.

In part 3, let’s consider how we could manage for engagement.

Click here to
send us feedback