As we all try to figure out just exactly what Industry 4.0 and Quality 4.0 mean it is not an exaggeration to say “Data is your most valuable asset. Yet we all struggle to actually get a benefit from this data and data integrity is an area of intense regulatory concern.
To truly have value our data needs to be properly defined, relevant to the tasks at hand, structured such that it is easy to find and understand, and of high-enough quality that it can be trusted. Without that we just have noise.
Understand why data matters, how to pick the right metrics, and how to ask the right questions from data. Understand correlation vs. causation to be able to make decisions about when to act on analysis and when not to is critical.
2. How well do the sample data represent the population?
3. Does your data distribution include outliers? How did they affect the results?
4. What assumptions are behind your analysis? Might certain conditions render your assumptions and your model invalid?
5. Why did you decide on that particular analytical approach? What alternatives did you consider?
6. How likely is it that the independent variables are actually causing the changes in the dependent variable? Might other analyses establish causality more clearly?
Framing data, being able to ask the right questions, is critical to being able to use that data and make decisions. In the past it was adequate enough for a quality professional to have a familiarity with a few basic tools. Today it is critical to understand basic statistics. As Nate Silver advises in an interview with HBR. “The best training is almost always going to be hands on training,” he says. “Getting your hands dirty with the data set is, I think, far and away better than spending too much time doing reading and so forth.”
Understanding data is a key ability and is necessary to thrive. It is time to truly contemplate the data ecosystem as a system and stop treating it as a specialized area of the organization.
Effectiveness I recently had a bit of a wake-up call via Twitter. I asked the following question: “What’s the one thing /above all/ that makes for an effective organisation?” My thanks to all those who took the time to reply with their viewpoint. The wake-up call for me was the variety of these responses. All […]
In quality management systems, it is critical to look at effectiveness. If you do not measure, you do not know if the system is working the ways you expect and desire.
We often discuss lagging (output measurement) and leading (predictive) indicators, and this is a good way to start, but if we apply System Thinking and use Meadow’s twelve leverage points we can see that most metrics tend to be around 7-12, with the more effective levers being the least utilized.
I think there are a lot of value in finding metrics within these levers.
So for example, a few indicators on the effectiveness of lever 4 “The Power to Add, Change, Evolve, or Self-Organize System Structure”:
Lagging
Leading
Effective CAPAs to the System
Number of changes initiated by level of organization and scale of change
Risk Management is a key enabler of any quality by design, whether of product, facility/equipment or study. We do living risk assessments to understand the scope of our ongoing risk. Inevitably we either want to implement that new or improved design or we want to mitigate the ongoing risks in our operation. So we turn to change management. And as part of that change management we do a risk assessment. Our change management then informs ongoing risk review.
Risk Management Leads to Change Management
Design Implementation
Through the iterative design lifecycle there is a final design ready for introduction. Perhaps this is a totally new thing, perhaps it is a new set of equipment or processes, or just a modification.
All along through the iterative design lifecycle risk management has been applied to establish measurable, testable, unambiguous and traceable performance requirements. Now your process engages with change management to introduce the change.
And a new risk assessment is conducted.
This risk assessment is asking a different question. During the iterative design lifecycle the risk question is some form of “What are the risks from this design on the patient/process.” As part of risk management, the question is “What are the risks to SISPQ/GMP from introducing the change.”
This risk assessment is narrower, in that it looks at the process of implementing. Broader that it looks at the entirety of your operations: facility, supply chain, quality system, etc.
The design risk assessment and risk management activities informs the change management risk assessment, but it cannot replace them. They also can serve to lower the rigor of the change management risk assessment, allowing the use of a less formal tool.
Living Risk Reviews
In the third phase of risk management – risk review – we confirm that the risks identified and mitigated as planned and are functioning as intended. We also evaluate to see if any additional, previously unpredicted risks have appeared. Risk review is the living part of the lifecycle as we return to it on a periodic basis.
From this will come new mitigations, targeted to address the identified risks. These mitigations inevitably lead to change management.
We again do a new risk assessment focusing on the risk of implementing the change. Informed by the living risk assessment, we can often utilize a less formal tool to look at the full ramifications of introducing the mitigation (a change).
Each and every change requires a risk assessment to capture the risks of the change. This ICHQ10 requirement is the best way to determine if the change is acceptable.
This risk assessment evaluates the impact on the change on the facility, equipment, materials, supply chain, processes. testing, quality systems and everything else. It is one of the critical reasons it is crucial to involve the right experts.
From this risk assessment comes the appropriate actions before implementing the change, as well as appropriate follow-up activities and it can help define the effectiveness review.
Depends. Sometimes the risk management looks at the individual implementations. Othertimes you need to do separate ones. Many times the risk assessment lead you to breaking up one change control into many. Evaluate as follows:
Are the risks from the separate implementations appropriately captured
Are the risks from pauses between implementations appropriately captured
As the ripples appropriately understood
Change Management Leads back to Risk Management
Sometimes a change control requires a specific risk assessment to be updated, or requires specific risk management to happen.
What about HAACP?
Hazard Analysis Critical Control Point (HACCP) are great tools for risk assessments. They are often the catalyst for doing a change, they are often the artifact of a change. They should never be utilized for determining the impact of a change.
A hazard is any biological, chemical, or physical property that impacts human safety. The HAACP identifies and establishes critical limits. But a HAACP is not the tool to use to determine if a change should move forward and what actions to do. It is to static.
In Closing
Risk Management is an enabler for change, a tenet enshrined in the ICH guidances. We are engaging in risk management activities throughout our organizations. It is critical to understand how the various risk management activities fit together and how they should be separated.
There is little argument regarding the critical role that structured problem-solving plays in a lean transformation. Besides the business results associated with solving problems, developing problem-solving skills increases learning, drives the desired change in thinking, and helps people more clearly understand how lean works as a system. With this said, however, it is amazing how little effort many organizations put into developing effective problem-solving skills. It seems like more time is spent on things like 5S, value stream mapping, and other tools that are generally considered easier to apply and less likely to be met with resistance. As a result, transformation does not occur, improvements are not sustainable, and the big gains possible through lean thinking are never achieved.
Good discussion on the importance of rigorous, sustained problem-solving as part of Lean initiatives. I think many of us have experienced this in our own organizations.
Utilizing problem solving tools in a structured way helps us better understand what is happening, how it is happening and most importantly, why it is happening. Armed with this understanding we can then engage in those improvements. Problem solving is key to getting those improvements because it allows us to discover why a problem is actually happening and not to just treat symptoms.
Problem Solving needs to reach a level of detail that accurately identifies an actionable cause that can then be addressed.