Tag: risk assessment

Why the Shift to Hazard Identification in ICH Q9(r1) Matters

The revised ICH Q9 (R1) guideline shifts from “Risk Identification” to “Hazard Identification” to reflect a more precise approach to identifying potential sources of harm (hazards) rather than broadly identifying risks.

  1. Alignment with Risk Assessment Definition: The term “Hazard Identification” is more consistent with the established definition of Risk Assessment, which involves identifying hazards and analyzing and evaluating the associated risks.
  2. Clarity and Precision: By focusing on hazards, the guideline aims to improve the clarity and precision of the risk management process. This helps better understand and assess the potential harms associated with identified hazards, leading to more effective risk management.
  3. Improved Perception and Assessment: The change is expected to enhance how hazards are perceived and assessed, making the risk management process more robust and scientifically grounded. This is particularly important for ensuring patient safety and product quality.
  4. Consistency in Terminology: The revision aims to standardize the terminology used in quality risk management, reducing confusion and ensuring all stakeholders understand the terms and processes involved.
ICH Q9 (r1) Figure 1: Overview of a typical quality risk management process

This small change in terminology can lead to better risk-based decisions by highlighting the need to identify hazards and not risks during the first step of the risk assessment process to remove any distractions about risks that may interfere with the hazard identification activity. When a Risk Assessment team focuses only on identifying hazards, they do not have to think about any related probabilities of occurrence – they only have to consider the potential hazards concerning the risk question under consideration. This is also the case of the severity of harm during hazard identification. There is no need to work to estimate the severity of the harm that may be presented by a hazard that comes later after the hazards have been identified.  

Risk, Hazard and Harm

Expert Intuition and Risk Management

Saturday Morning Breakfast Cereal source http://smbc-comics.com/comic/horrible

Risk management is a crucial aspect of any organization or project. However, it is often subject to human errors in subjective risk judgments. This is because most risk assessment methods rely on subjective inputs from experts. Without certain precautions, experts can make consistent errors in judgment about uncertainty and risk.

There are methods that can correct the systemic errors that people make, but very few organizations implement them. As a result, there is often an almost universal understatement of risk. We need to keep in mind a few rules about experience and expertise.

  • Experience is a nonrandom, nonscientific sample of events throughout our lifetime.
  • Experience is memory-based and we are very selective regarding what we choose to remember,
  • What we conclude from our experience can be full of logical errors
  • Unless we get reliable feedback on past decisions, there is no reason to believe our experience will tell us much.

No matter how much experience we accumulate, we seem to be very inconsistent in its application.

Experts have unconscious heuristics and biases that impact their judgment, some important ones include:

  • Misconceptions of chance: If you flip a coin six times, which result is more likely (H= heads, T= tails): HHHTTT or HTHTTH? They are both equal, but many people assume that because the first series looks “less random” than the second, it must be less likely. This is an example of representativeness bias. We appear to judge odds based on what we assume to be representative scenarios. Human beings easily confuse patterns and randomness.
  • The conjunction fallacy: We often see specific events as more likely than broader categories of events.
  • Irrational belief in small samples
  • Disregarding variance in small samples. Small samples have more random variance that large samples is considered less than it should be.
  • Insensitivity to prior probabilities: People tend to ignore the past and focus on new information when making subjective estimates.

This is all about overconfidence as an expert, which will consistently underestimate risks.

What are some ways to overcome this? I recommend the following be built into your risk management system.

  • Pretend you are in the future looking back at failure. Start with the assumption that a major disaster did happen and describe how it happened.
  • Look to risks from others. Gather a list of related failures, for example, regulatory agency observations, and think of risks in relation to those.
  • Include Everyone. Your organization has numerous experts on all sorts of specific risks. Make the effort to survey representatives of just about every job level.
  • Do peer reviews. Check assumptions by showing them to peers who are not immersed in the assessment.
  • Implement metrics for performance. The Brier score is a way to evaluate the result of predictions both by how often the team was right and by the probability the estimated for getting a correct answer.

Further Reading

Here are some sources that discuss the topic of human errors and subjective judgments in risk management:

Bow-Tie Diagram

The bow-tie method is a powerful tool for visualizing and managing risks. Named after its distinctive shape, this tool is used to analyze the causes and consequences of potential risks.

At the center of the bow-tie diagram is the “top event,” which represents the risk being analyzed. On the left side of the diagram are the potential causes of the top event, while on the right side are the potential consequences. The diagram also includes barriers or controls that can be put in place to prevent or mitigate the risk.

To create a bow-tie diagram identify the “top event” representing the risk being analyzed. This is placed at the center of the diagram.

Next, you identify the potential causes of the top event and place them on the left side of the diagram. These causes can be further broken down into sub-causes if necessary.

On the right side of the diagram, you identify the potential consequences of the top event. These can also be further broken down into sub-consequences if necessary.

Once you have identified the causes and consequences of the top event, you can then add barriers or controls to the diagram. These are measures that can be put in place to prevent or mitigate the risk. Barriers can be placed between the causes and the top event to prevent it from occurring, while controls can be placed between the top event and its consequences to mitigate their impact.

The bow-tie method works by providing a clear and concise visual representation of a risk and its potential impacts. This allows stakeholders to better understand the risk and identify areas where additional controls may be needed.

This tool also works nicely with desirable consequences.

This picture showed up when I typed bow-tie on my computer. It’s relevant

Detectability in Risk Management is a “Sort of” “Sometimes” thing

I’ve recently seen a few audits that point out something along the line of “Recommendation to revise Quality Risk Management Process/Procedure to include detectability as a variable in determining Risk Priority Numbers (RPNs).  The current process only includes the frequency and severity of impact in the calculation.  However, ICH Q9 also recognizes the use of risk management tools which include the ability to detect harm (detectability) in the estimation of risk (refer to the section titled “Risk analysis”).”

So, first of all, that’s not what Q9 says. Q9 (R1) is actually pretty clear here, stating “Risk analysis is the estimation of the risk associated with the identified hazards. It is the qualitative or quantitative process of linking the likelihood of occurrence and severity of harms. In some risk management tools, the ability to detect the harm (detectability) also factors in the estimation of risk.”

Q9 later goes on to state “Quality risk management supports a scientific and practical approach to decision-making. It provides documented, transparent and reproducible methods to accomplish steps of the quality risk management process based on current knowledge about assessing the probability, severity and sometimes detectability of the risk.”

Q9 clearly recognizes that detectability is useful sometimes, with specific tools in specific cases. This is in alignment with risk management thinking in general, for example ISO 31000:2018 states that Risk analysis should consider factors such as:

— the likelihood of events and consequences;
— the nature and magnitude of consequences;
— complexity and connectivity;
— time-related factors and volatility;
— the effectiveness of existing controls;
— sensitivity and confidence levels.

Detectability is then one of several methods to consider in risk analysis. The selection criteria for tools should take into account situations when detectability is desired and drive to use of those tools, for example, the FMEA which is built to determine how and when a failure can be detected. In other tools, detectability is usually built into the evaluation of current controls and is often captured in likelihood or somewhere else

When it comes to risk, avoid a one-size fits all. Think of what the intent is and use the right tool for the job.

Computer Software Assurance Draft

The FDA published on 13-Sep-2022 the long-awaited draft of the guidance “Computer Software Assurance for Production and Quality System Software,” and you may, based on all the emails and posting be wondering just how radical a change this is.

It’s not. This guidance is just one big “calm down people” letter from the agency. They publish these sorts of guidance every now and then because we as an industry can sometimes learn the wrong lessons.

This guidance states:

  1. Determine intended use
  2. Perform a risk assessment
  3. Perform activities to the required level

I wrote about this approach in “Risk Based Data Integrity Assessment,” and it has existed in GAMP5 and other approaches for years.

So read the guidance, but don’t panic. You are either following it already or you just need to spend some time getting better at risk assessments and creating some matrix approaches.