I am looking forward to speaking at the GxP Cloud Compliance Summit in Boston in September on Implementing a Lifecycle Risk Management Approach to the Cloud. I’ll be discussing some of my favorite topics:
Best practices to harness a life cycle risk management approach to protect product quality and patient data
What does a living risk assessment look like when key parts of your IT infrastructure is maintained by cloud service providers
How does Q9 R1 impact functional and usage assessments around cloud applications
I am looking forward to meeting and discussing some of the critical questions in our heady embrace of the cloud.
Cloud based GxP systems have shifted in the last few years from “Something I guess we should figure out” to “Well guess we have it now” to “Well that is all I seem to have now.” And where 5 years ago it seemed we were obsessed about the fine details of Open vs Closed systems and what cloud-based applications are, we are now looking at much more mature questions around a risk based strategy that evaluates and ensures appropriate controls around Data Integrity, Privacy, and Security. Through a risk-based approach, we drive activities such as auditing, change control, qualification/validation, and oversight.
I am looking forward to having this discussion with my peers and sharing best practices and experiences. It is only though this type of event that we can grow as a professional.
Risk management is a crucial aspect of any organization or project. However, it is often subject to human errors in subjective risk judgments. This is because most risk assessment methods rely on subjective inputs from experts. Without certain precautions, experts can make consistent errors in judgment about uncertainty and risk.
There are methods that can correct the systemic errors that people make, but very few organizations implement them. As a result, there is often an almost universal understatement of risk. We need to keep in mind a few rules about experience and expertise.
Experience is a nonrandom, nonscientific sample of events throughout our lifetime.
Experience is memory-based and we are very selective regarding what we choose to remember,
What we conclude from our experience can be full of logical errors
Unless we get reliable feedback on past decisions, there is no reason to believe our experience will tell us much.
No matter how much experience we accumulate, we seem to be very inconsistent in its application.
Experts have unconscious heuristics and biases that impact their judgment, some important ones include:
Misconceptions of chance: If you flip a coin six times, which result is more likely (H= heads, T= tails): HHHTTT or HTHTTH? They are both equal, but many people assume that because the first series looks “less random” than the second, it must be less likely. This is an example of representativeness bias. We appear to judge odds based on what we assume to be representative scenarios. Human beings easily confuse patterns and randomness.
The conjunction fallacy: We often see specific events as more likely than broader categories of events.
Irrational belief in small samples
Disregarding variance in small samples. Small samples have more random variance that large samples is considered less than it should be.
Insensitivity to prior probabilities: People tend to ignore the past and focus on new information when making subjective estimates.
This is all about overconfidence as an expert, which will consistently underestimate risks.
What are some ways to overcome this? I recommend the following be built into your risk management system.
Pretend you are in the future looking back at failure. Start with the assumption that a major disaster did happen and describe how it happened.
Look to risks from others. Gather a list of related failures, for example, regulatory agency observations, and think of risks in relation to those.
Include Everyone. Your organization has numerous experts on all sorts of specific risks. Make the effort to survey representatives of just about every job level.
Do peer reviews. Check assumptions by showing them to peers who are not immersed in the assessment.
Implement metrics for performance. The Brier score is a way to evaluate the result of predictions both by how often the team was right and by the probability the estimated for getting a correct answer.
Further Reading
Here are some sources that discuss the topic of human errors and subjective judgments in risk management:
The pharmaceutical regulations call, repeatedly for business continuity plans. For example, the FDA calls for fairly significant requirements for Medically Necessary Products:
Medically necessary drug products and their components are manufactured all over the world. An emergency situation anywhere in the world thus might affect the availability of drug products in the United States and result in drug shortages. Emergency preparedness for situations that could result in high employee absenteeism is an important goal for manufacturers of drug products and their components. For example, in an influenza pandemic, widespread human outbreaks of illness would be expected in the United States and around the world, resulting in widespread high absenteeism that could hinder normal production activities and cause shortages in the supply of drug products, packaging materials, and drug components. It is therefore vital for industry to prepare before an emergency situation occurs and to develop plans to ensure continuity of operations during emergencies (including, for example, an influenza pandemic, natural disaster, or personnel issue) that would prevent a significant portion of the work force from reporting. It is especially important for manufacturers of finished drug products to be aware of their suppliers’ and contractors’ responses to personnel shortages and, when appropriate, work with them to ensure the availability of high quality materials and services that contribute to the manufacture of MNPs.
You can find less definitive requirements throughout the various health authorities’ regulations and guidances.
So what do we mean by business continuity?
Business continuity is the holistic management process that ensures operations continue and that products and services are delivered at predefined levels (e.g. no shortages, no halt to an ongoing clinical trial). This approach is aligned with ISO 22301 Business Continuity Management Systems.
Business continuity management is an ongoing process based on the plan-do-check-act methodology that is made up of 4 key elements:
Emergency Action and Response Plans
Disaster Recovery Plans
Crisis Management Plans
Business Continuity Plans
Emergency Action Plans
An emergency action plan is designed to respond to an emergency with mitigating procedures to protect, secure and evacuate people to safety. This is more an OSHA thing; chances are your average Quality unit doesn’t end up owning it. Unless you have no HS&E unit, and then you write one.
This plan includes procedures for detecting, warning, and responding to specific potential emergencies such as fire, severe weather, earthquake, medical emergencies, workplace violence, and other potential threats.
Disaster Recovery Plan
Disaster recovery plans are designed to recover from a disaster, usually related to equipment, infrastructure, and information technology. Something big goes boom, how do you restore this vital support system or equipment as soon as possible and minimize downtime and loss of data. Very important for computer system lifecycle, disaster recovery plans should include specific plans for recovery functions, resumption strategies, critical personnel, equipment, services, and external and internal communications.
Crisis Management Plans
Crisis management is all about planning and mitigating situations that have risk, and are usually a lot of management of communications internally and externally. This includes with regulators, health care providers, etc. When we implement SOPs for health authority notifications we are engaging in crisis management planning.
Business Continuity Plans
Business continuity planning identifies and plans for disasters to events that could negatively an organization’s business functions, objectives, income, reputation, and ultimate survival. This planning takes place in advance of the potential disasters or events that could harm an organization. It takes potential disasters and events into consideration with their effects on suppliers, vendors customers, and the organization’s other stakeholders.
In a GxP environment, we are looking at the potential impact of disasters on drug supply and clinical study outcomes (amongst other key activities).
The BCP is all about minimizing the effects of the disaster or event on the organization and returning to normal operations as soon as possible.
These Plans are Interrelated
All four plans are interrelated and should be coordinated. The plans can be combined, but as there are usually very different owners they are often separated.
Documented Plans
The business continuity planning process should result in formal, documented plans that serve as a reference guide in the event of a disaster or event. The existence of the business continuity plans should be well communicated, with individuals with responsibilities having ready access and additional training.
Applying the Risk Management Process
The Business Continuity process should leverage existing risk assessments and sit around it.
Select Team
The team should be multifunctional and very knowledgeable about the organization’s business and the risks it faces. This should be a permanent team, not ad hoc, as this is a living process. You can always bring in ad hoc members for specific questions.
Define Context, Purpose, Scope
At a minimum you are tackling the disruption to product supply and cessation of critical GxP data but there may be other business requirements to tackle. Make sure everyone agrees on these.
Define Terminology
Make sure everyone is on the same page with just what disaster, event, crisis, stakeholder, and business continuity plan (and other important concepts) are.
Agree on the scales for likelihood and severity.
Critical Function Assessment
Identify the business functions that are sensitive to downtime, fulfill regulatory obligations and are vital for maintaining product supply.
Threat Assessment
Identify the threats to the performance of the critical functions.
Utilize a risk matrix to assess the likelihood and severity of the identified hazards and risks.
Develop Business Continuity Plan(s)
After the hazards and risks have been identified, the impact understood and the risks assessed it is time to develop the business continuity plan (BCP). The BCP allows the organziation to survive the event or disaster with minimal disruption. The BCP focuses on mitigating the consequences of the event or disaster that could not be prevented. Recovery strategies for these cosnequences are determined, developed and become part of the BCP.
When many potential risks have been identified, use the risk score to prioritize.
BCPs cover management commitment, team ientification, team responsibilities, mitigation plans, recovery strategies, training, testing and evaluation and continious improvement. Basically the same thing any good plan does.
Mitigation plans are intended to lessen the negative effectis of an event or disaster.
Provide appropriate awareness training to everyone impacted, with more substantial trining to the BCP team.
Verify it periodically and ensure it is continues to be relevant.
Whenever relevant, procceduralize these BCP instructions.
I’ve recently seen a few audits that point out something along the line of “Recommendation to revise Quality Risk Management Process/Procedure to include detectability as a variable in determining Risk Priority Numbers (RPNs). The current process only includes the frequency and severity of impact in the calculation. However, ICH Q9 also recognizes the use of risk management tools which include the ability to detect harm (detectability) in the estimation of risk (refer to the section titled “Risk analysis”).”
So, first of all, that’s not what Q9 says. Q9 (R1) is actually pretty clear here, stating “Risk analysis is the estimation of the risk associated with the identified hazards. It is the qualitative or quantitative process of linking the likelihood of occurrence and severity of harms. In some risk management tools, the ability to detect the harm (detectability) also factors in the estimation of risk.”
Q9 later goes on to state “Quality risk management supports a scientific and practical approach to decision-making. It provides documented, transparent and reproducible methods to accomplish steps of the quality risk management process based on current knowledge about assessing the probability, severity and sometimes detectability of the risk.”
Q9 clearly recognizes that detectability is useful sometimes, with specific tools in specific cases. This is in alignment with risk management thinking in general, for example ISO 31000:2018 states that Risk analysis should consider factors such as:
— the likelihood of events and consequences; — the nature and magnitude of consequences; — complexity and connectivity; — time-related factors and volatility; — the effectiveness of existing controls; — sensitivity and confidence levels.
Detectability is then one of several methods to consider in risk analysis. The selection criteria for tools should take into account situations when detectability is desired and drive to use of those tools, for example, the FMEA which is built to determine how and when a failure can be detected. In other tools, detectability is usually built into the evaluation of current controls and is often captured in likelihood or somewhere else
When it comes to risk, avoid a one-size fits all. Think of what the intent is and use the right tool for the job.
Different Kinds of Unknowns, Source: Smithson (1989, p. 9); also in Bammer et al. (2008, p. 294).
An alternative way to look at uncertainty is offered by Klir, which adds discord to the mix.
Work-As-Prescribed can be a real avenue for all three of these uncertainties. But by using risk management to examine the possibilities of these uncertainties we can truly interrogate. This is one of the things we mean by risk management and knowledge management being bound at the hip as enablers.
To do this we need to make sure that:
There is the management of information quality. Management of information quality is crucial in risk management because uncertainty is prevalent. Uncertainty, as a state for which we lack information, means that uncertainty analysis should play an integral part in risk management to ensure that the uncertainty in the risk management process is kept at a feasible level.
There is explicit management of either existing knowledge that can be applied to improve the quality of the analyses or to improve the knowledge acquired in the process that can be used in the follow-up process. Knowledge management is pivotal to ensuring an effective risk management process by providing context and learning possibilities. In essence, risk management is not just about managing risks – the entire context surrounding the risks must be understood and managed effectively.
