“Advice for the government on communicating uncertainty” by Jessica Hullman on the blog Statistical Modeling, Causal Inference, and Social Science is a great read.
Great stuff to think about for those of us who constantly have to communicate uncertainty in our roles.
Q9 (r1) starts with all the same sections on scope and purpose. There are slight differences in ordering in scope, mainly because of the new sections below, but there isn’t much substantially different.
This is the first major change with added paragraphs on subjectivity, which basically admits that it exists and everyone should be aware of that. This is the first major change that should be addressed in the quality system “All participants involved with quality risk management activities should acknowledge, anticipate, and address the potential for subjectivity.”
Aligned with that requirement is a third bullet for decision-makers: “assure that subjectivity in quality risk management activities is controlled and minimised, to facilitate scientifically robust risk-based decision making.”
Solid additions, if a bit high level. A topic of some interest on this blog, recognizing the impact of subjectivity is critical to truly developing good risk management.
Expect to start getting questions on how you acknowledge, anticipate and address subjectivity. It will take a few years for this to work its way through the various inspectorates after approval, but it will. There are various ways to crack this, but it will require both training and tools to make it happen. It also reinforces the need for well-trained facilitators.
“The degree of rigor and formality of quality risk management should reflect available knowledge and be commensurate with the complexity and/ or criticality of the issue to be addressed.”
That statement in Q9 has long been a nugget of long debate, so it is good to see section 5.1 added to give guidance on how to implement it, utilizing 3 axis:
This section is important, the industry needs it as too many companies have primitive risk management approaches because they shoe-horn everything into a one size fits all level of formality and thus either go overboard or do not go far enough. But as written this draft of Q9 is a boon to consultants.
We then go on to get just how much effort should go into higher formality versus lower level of formality which boils down to higher formality is more stand alone and lower formality happens within another aspect of the quality system.
Another new section, definitely designed to align to ISO 9001-2015 thinking. Based on the level of formality we are given three types with the first two covering separate risk management activities and the third being rule-based in procedures.
Section 6 gets new subsection “The role of Quality Risk Management in addressing Product Availability Risks,” “Manufacturing Process Variation and State of Control (internal and external),” “Manufacturing Facilities,” “Oversight of Outsourced Activities and Suppliers.” These new subsections expand on what used to be solely a list of bullet points and provide some points to consider in their topic area. They are also good things to make sure risk management is built into if not already there.
The ICH members did exactly what they told us they were going to do, and pretty much nothing else. I do not think they dealt with the issues deeply and definitively enough, and have added a whole lot of ambiguity into the guidance. which is better than being silent on the topic, but I’m hoping for a lot more.
Subjectivity, uncertainty, and formality are critical topics. Hopefully your risk management program is already taking these into account.
I’m hoping we will also see a quick revision of the PIC/S “Assessment of Quality Risk Management Implementation” to align to these concepts.
An information gap is a known unknown, a question that one is aware of but for which one is uncertain of the answer. It is a disparity between what the decision maker knows and what could be known The attention paid to such an information gap depends on two key factors: salience, and importance.
Information gaps often dwell in the land of knightian uncertainty.
A wide range of reasons for information gaps exist:
Walker et al. (2010) developed a taxonomy of “levels of uncertainty”, ranging from Level 1 to Level 4,
which is useful in problem-solving:
Levels 3 and 4 uncertainties are described as representing “deep uncertainty”.
This hierarchy can be useful to help us think carefully about whether the uncertainty behind a problem can be defined in terms of a Level 1 prediction, with parameters for variation. Or, can it be resolved as group of Level 2 possibilities with probability estimates for each? Can the issue only be understood as a set of different Level 3 futures, each with a clear set of defined outcomes, or only by means of a Level 4 statement to the effect that we know only that there is something crucial that we don’t yet know?
There is often no clear or unanimous view of whether a particular uncertainty is set at a specific level. Uncertainty should always be considered at the deepest proposed level, unless or until those that propose this level can be convinced by an evidence-based argument that it should be otherwise.
An important part of innovation, risk management, change management, continuous improvement is overcoming the fear of the unknown. We humans are wired with an intense aversion to both risk and uncertainty. Research shows that both have separate neural reactions and that choices with ambiguous outcomes trigger a stronger fear response than do risky choices. Additional research shows that the risk itself isn’t so much the problem, but the uncertainty is: we are afraid primarily because we don’t know the outcome and less so because of the risk.
There are three types of uncertainty:
I wrote more on this in my post “Uncertainty and Subjectivity in Risk Management.” This post mostly stems from wanting an excuse to share a funny comic.
Investigations of a Dog 