The Pharmaceutical Inspection Convention Cooperation Scheme (PIC/S) on 01-Jan-2019 released a long-awaited guidance to help regulators harmonize the classification and reporting of good manufacturing practice (GMP) deficiency outcomes from inspections. The guidance is designed as a “tool to support the risk-based classification of GMP deficiencies from inspections and to establish consistency amongst inspectorates.”
| PI 040-1 “Guidance on Classification of GMP Deficiencies“ |
I recently had a discussion with one of the best root cause investigators and problem solvers I know, Thor McRockhammer. Thor had concerns about a case where the expected conditions were not met and there were indications that individuals engaged in troubleshooting and as a result not only made the problem worse but led to a set of issues that seem rather systematic.
Our conversation (which I do not want to go into too much detail on) was a great example of troubleshooting going wrong.
Troubleshooting is defined as “Reactive problem solving based upon quick responses to immediate symptoms. Provides relief and immediate problem mitigation. But may fail to get at the real cause, which can lead to prolonged cycles of firefighting.” Troubleshooting usually goes wrong one of a few ways:
Troubleshooting is a reactive process of fixing problems by rapid response and short-term corrective actions. It covers noticing the problem, stopping the damage and preventing spread of the problem.
So if our departure from expected conditions was a leaky gasket, then troubleshooting is to try to stop the leak. If our departure is a missing cart then troubleshooting usually involves finding the cart.
Troubleshooting puts things back into the expected condition without changing anything. It addresses the symptom and not the fundamental problems and their underlying causes. They are carried out directly by the people who experience the symptoms, relying upon thorough training, expertise and procedures designed explicitly for troubleshooting.
With out leaky gasket example, our operators are trained and have procedural guidance to tighten or even replace a gasket. They also know what not to do (for example don’t weld the pipe, don’t use a different gasket, etc). There is also a process for documenting the troubleshooting happened (work order, comment, etc).
To avoid the problems listed above troubleshooting needs a process that people can be thoroughly trained in. This process needs to cover what to do, how to communicate it, and where the boundaries are.
|
Step |
What we do |
Things to be aware of |
|
Concern |
· What do we known about the exact nature of the problem? |
· What do your standards say about how this concern should be documented? o For example, can be addressed as a comment or does it require a deviation or similar non-conformance · If the concern stems from a requirement it must be documented. |
|
Cause |
· What do you know about the apparent (or root) cause of the problem? |
· Troubleshooting is really good at dealing with superficial cause-and-effect relationships. As the cause deepens, fixing it requires deeper problem-solving. · The cause can be a deficiency or departure from a standard
|
|
Countermeasure |
· What immediate or temporary countermeasures can be taken to reduce or eliminate the problem? · Are follow-up or more permanent countermeasures required to prevent recurrence? o If so, do you need to investigate more deeply? |
· Countermeasures need to be evaluated against change management · Countermeasures cannot ignore, replace or go around standards · Apply good knowledge management |
|
Check results |
· Did the results of the action have any immediate effect on eliminating the concern or problem? · Does the problem repeat? o If so, do you need to investigate more deeply? |
· Recurrence should trigger deeper problem-solving and be recorded in the quality system. · Beware troubleshooting countermeasures becoming tribal knowledge and the new way of working |
Think of your standards as a box. This box defines what should happen per our qualified/validated state, our procedures, and the like. We can troubleshoot as much as we want within the box. We cannot change the box in any way, nor can we leave the box without triggering our deviation/nonconformance system (reactive) or entering change management (proactive).
Troubleshooting processes need a mechanism for letting supervisors happen. Troubleshooting that happens in the dark is going to cause a disaster.
Operators need to be trained how to document troubleshooting. Sometimes this is as simple as a notation or comment, other-times you trigger a corrective action process.
Engaging in troubleshooting, and not documenting it starts to look a like fraud and is a data integrity concern.
The change management process should be triggered as a result of troubleshooting. Operators should be trained to interpret it. This is often were concept of exact replacements and like-for-like come in.
It is trouble shooting to replace a part with an exact part. Anything else (including functional equivalency) is a higher order change management activity. It is important that everyone involved knows the difference.
| Covers | Is it troubleshooting? | |
| Like-for- Like | Spare parts that are identical replacements (has the same the same manufacturer, part number, material of construction, version) Existing contingency procedures (documented, verified, ideally part of qualification/validation) | Yes This should be built into procedures like corrective maintenance, spare parts, operations and even contingency procedures. |
| Functionally equivalent | Equivalent, for example, performance, specifications, physical characteristics, usability, maintainability, cleanability, safety | No Need to understand root cause. Need to undergo appropriate level of change management |
| New | Anything else | No Need to understand root cause. Need to undergo appropriate level of change management |
This applies to both administrative and technical controls.
ITIL Incident Management (or similar models) is just troubleshooting and has all the same hallmarks and concerns.
Trouble shooting is an integral process. And like all processes it should have a standard, be trained on, and go through continuous improvement.
Four Types of Problems: from reactive troubleshooting to creative innovation by Art Smalley (2018, Lean Enterprise Institute, Inc. ) is a fascinating book focusing on dividing problem solving and innovation into four major types:
Art Smalley is a well known Lean expert, and this book definitely grows out of the wisdom and is a pretty good read. He shares
the strengths and weakness of each problem solving technique providing many points of introspection, such as the questions at the end of each chapter and excellent illustrations.
This book provides s a framework, a mental model, to effectively approach and assess a situation in order to seek and bring the appropriate kind of thinking to calmly, confidently address the problem at hand.
In many ways this book was my favorite quality book of 2018. I think it could serve as a valuable primer and I’m contemplating how to use it for internal training this year.
The proposed technical correction regulation, the other two guidance documents, and the list deal with the transition of certain biological products from NDAs to BLAs. Starting with the simplest, the proposed (so-called) technical correction would amend the definition of “biological product” in 21 C.F.R. § 600.3(h) to conform to the definition implemented in the BPCIA and provide an interpretation of the statutory terms “protein” and “chemically synthesized polypeptide.” FDA calls it a “technical correction” in the proposed rule, but this isn’t really technical, nor is it a correction. Indeed, it reflects a significant change to the definition of biological product because the rule would replace the phrase “means any” with the phrase “means a” and would add the phrase “protein (except any chemically synthesized polypeptide)” to the definition of “biological product.” Consistent with the April 2015 Questions & Answers guidance, the proposed rule would amend 21 C.F.R. § 600.3(h) to further define protein as any alpha amino acid polymer with a specific defined sequence that is greater than 40 amino acids in size, and the term chemically synthesized polypeptide as any alpha amino acid polymer that: (1) is made entirely by chemical synthesis and (2) is greater than 40 amino acids but less than 100 amino acids in size. Given that that FDA has been using this definition since the publication of the April 2015 final version of this Question and Answer guidance, this proposed regulation is unlikely to catch industry by surprise. But this is just one of multiple steps FDA is taking to prepare industry for the March 2020 transition of certain biological products approved under NDAs to BLAs.
— Read on www.fdalawblog.net/2019/01/avalanche-or-roadblock-fda-publishes-flurries-of-biologic-and-biosimilar-materials/
We have these quality systems with lots of levers, with interrelated components. And yet we select one or two metrics and realize that even if we meet them, we aren’t really measuring the right stuff nor are we driving continious improvement.
One solution is to create layered metrics, which basically means drill down your process and identify the metrics at each step.
Lots of ways to do this. An easy way to start is to use the 5-why process, a tool most folks are comfortable with.
So for example, CAPA. It is pretty much agreed upon that CAPAs should be completed in a timely manner. That makes this a top level goal. Unfortunately, in this hypothetical example, we are suffering a less than 100% closure goal (or whatever level is appropriate in your organization based on maturity)
| Why 1 | Why was CAPA closure not 100% Because CAPA tasks were not closed on time. Success factor needed for this step: CAPA tasks to be closed by due date. Metric for this step: CAPA closure task success rate |
| Why 2 | Why were CAPA tasks not closed on time? Because individuals did not have appropriate time to complete CAPA tasks. Metric for this step: Planned versus Actual time commitment |
| Why 3 | Why did individuals not have appropriate time to complete CAPA tasks? Because CAPA task due dates are guessed at. Metric for this step: CAPA task adherence to target dates based on activity (e.g. it takes 14 days to revise a document and another 14 days to train, the average document revision task should be 28 days) |
| Why 4 | Why are CAPA task due dates guessed at? Because appropriate project planning is not completed. Metric for this step: Adherence to Process Confirmation |
| Why 5 | Why is appropriate project planning not completed? Because CAPAs are always determined on the last day the deviation is due. Metric: Adherence to Root Cause Analysis process |
I might on report on the top CAPA closure rate and 1 or 2 of these, and keep the others in my process owner toolkit. Maybe we jump right to the last one as what we report on. Depends on what needs to be influenced in my organization and it will change over time.
It helps to compare this output against the 12 system leverage points.
These metrics go from 3 “goals of the system” with completing CAPA tasks effectively and on time, to 4 “self organize” and 5 “rules of the system.” It also has nice feedback loops based on the process confirmations. I’d view them as potentially pretty successful. Of course, we would test these and tinker and basically experiment until we find the right set of metrics that improves our top-level goal.
