Layering metrics

We have these quality systems with lots of levers, with interrelated components. And yet we select one or two metrics and realize that even if we meet them, we aren’t really measuring the right stuff nor are we driving continious improvement.

One solution is to create layered metrics, which basically means drill down your process and identify the metrics at each step.

Lots of ways to do this. An easy way to start is to use the 5-why process, a tool most folks are comfortable with.

So for example, CAPA. It is pretty much agreed upon that CAPAs should be completed in a timely manner. That makes this a top level goal. Unfortunately, in this hypothetical example, we are suffering a less than 100% closure goal (or whatever level is appropriate in your organization based on maturity)

Why 1	Why was CAPA closure not 100% Because CAPA tasks were not closed on time. Success factor needed for this step: CAPA tasks to be closed by due date. Metric for this step: CAPA closure task success rate
Why 2	Why were CAPA tasks not closed on time? Because individuals did not have appropriate time to complete CAPA tasks. Metric for this step: Planned versus Actual time commitment
Why 3	Why did individuals not have appropriate time to complete CAPA tasks? Because CAPA task due dates are guessed at. Metric for this step: CAPA task adherence to target dates based on activity (e.g. it takes 14 days to revise a document and another 14 days to train, the average document revision task should be 28 days)
Why 4	Why are CAPA task due dates guessed at? Because appropriate project planning is not completed. Metric for this step: Adherence to Process Confirmation
Why 5	Why is appropriate project planning not completed? Because CAPAs are always determined on the last day the deviation is due. Metric: Adherence to Root Cause Analysis process

I might on report on the top CAPA closure rate and 1 or 2 of these, and keep the others in my process owner toolkit. Maybe we jump right to the last one as what we report on. Depends on what needs to be influenced in my organization and it will change over time.

It helps to compare this output against the 12 system leverage points.

These metrics go from 3 “goals of the system” with completing CAPA tasks effectively and on time, to 4 “self organize” and 5 “rules of the system.” It also has nice feedback loops based on the process confirmations. I’d view them as potentially pretty successful. Of course, we would test these and tinker and basically experiment until we find the right set of metrics that improves our top-level goal.