Q9 (r1) Risk Management Draft

Q9 (r1) starts with all the same sections on scope and purpose. There are slight differences in ordering in scope, mainly because of the new sections below, but there isn’t much substantially different.

4.1 Responsibilities

This is the first major change with added paragraphs on subjectivity, which basically admits that it exists and everyone should be aware of that. This is the first major change that should be addressed in the quality system “All participants involved with quality risk management activities should acknowledge, anticipate, and address the potential for subjectivity.”

Aligned with that requirement is a third bullet for decision-makers: “assure that subjectivity in quality risk management activities is controlled and minimised, to facilitate scientifically robust risk-based decision making.”

Solid additions, if a bit high level. A topic of some interest on this blog, recognizing the impact of subjectivity is critical to truly developing good risk management.

Expect to start getting questions on how you acknowledge, anticipate and address subjectivity. It will take a few years for this to work its way through the various inspectorates after approval, but it will. There are various ways to crack this, but it will require both training and tools to make it happen. It also reinforces the need for well-trained facilitators.

5.1 Formality in Quality Risk Management

“The degree of rigor and formality of quality risk management should reflect available knowledge and be commensurate with the complexity and/ or criticality of the issue to be addressed.”

That statement in Q9 has long been a nugget of long debate, so it is good to see section 5.1 added to give guidance on how to implement it, utilizing 3 axis:

  • Uncertainty: This draft of Q9 utilizes a fairly simple definition of uncertainty and needs to be better aligned to ISO 31000. This is where I am going to definitely submit comments. Taking a straight knowledge management approach and defining uncertainty solely on lack of knowledge misses the other element of uncertainty that are important.
  • Importance: This was probably the critical determination folks applied to formality in the past.
  • Complexity: Not much said on complexity, which is worrisome because this is a tough one to truly analyze. It requires system thinking, and a ot of folks really get complicated and complex confused.

This section is important, the industry needs it as too many companies have primitive risk management approaches because they shoe-horn everything into a one size fits all level of formality and thus either go overboard or do not go far enough. But as written this draft of Q9 is a boon to consultants.

We then go on to get just how much effort should go into higher formality versus lower level of formality which boils down to higher formality is more stand alone and lower formality happens within another aspect of the quality system.

5.2 Risk-based Decision Making

Another new section, definitely designed to align to ISO 9001-2015 thinking. Based on the level of formality we are given three types with the first two covering separate risk management activities and the third being rule-based in procedures.

6. INTEGRATION OF QUALITY RISK MANAGEMENT INTO INDUSTRY AND REGULATORY OPERATIONS

Section 6 gets new subsection “The role of Quality Risk Management in addressing Product Availability Risks,” “Manufacturing Process Variation and State of Control (internal and external),” “Manufacturing Facilities,” “Oversight of Outsourced Activities and Suppliers.” These new subsections expand on what used to be solely a list of bullet points and provide some points to consider in their topic area. They are also good things to make sure risk management is built into if not already there.

Overall Thoughts

The ICH members did exactly what they told us they were going to do, and pretty much nothing else. I do not think they dealt with the issues deeply and definitively enough, and have added a whole lot of ambiguity into the guidance. which is better than being silent on the topic, but I’m hoping for a lot more.

Subjectivity, uncertainty, and formality are critical topics. Hopefully your risk management program is already taking these into account.

I’m hoping we will also see a quick revision of the PIC/S “Assessment of Quality Risk Management Implementation” to align to these concepts.

Information Gaps

An information gap is a known unknown, a question that one is aware of but for which one is uncertain of the answer. It is a disparity between what the decision maker knows and what could be known The attention paid to such an information gap depends on two key factors: salience, and importance.

  • The salience of a question indicates the degree to which contextual factors in a situation highlight it. Salience might depend, for example, on whether there is an obvious counterfactual in which the question can be definitively answered.
  • The importance of a question is a measure of how much one’s utility would depend on the actual answer. It is this factor—importance—which is influenced by actions like gambling on the answer or taking on risk that the information gap would be relevant for assessing.

Information gaps often dwell in the land of knightian uncertainty.

Communicating these Known Unknowns

Communicating around Known Unknowns and other forms of uncertainty

A wide range of reasons for information gaps exist:

  • variability within a sampled population or repeated measures leading to, for example, statistical margins-of-error
  • computational or systematic inadequacies of measurement
  • limited knowledge and ignorance about underlying processes
  • expert disagreement.

Levels of Uncertainty

Walker et al. (2010) developed a taxonomy of “levels of uncertainty”, ranging from Level 1 to Level 4,
which is useful in problem-solving:

  • Level 1 uncertainties are defined as relatively minor – as representing “a clear enough future” set within a “single system model” whereby outcomes can be estimated with reasonable accuracy;
  • Level 2 uncertainties display “alternative futures” but, again, within a single system in which probability estimates can be applied with confidence.

Levels 3 and 4 uncertainties are described as representing “deep uncertainty”.

  • Level 3 uncertainties are described as “a multiplicity of plausible futures”, in which multiple systems interact, but in which we can identify “a known range of outcomes”
  • Level 4 uncertainties lead us to an “unknown future” in which we don’t understand the system: we know only that there is something, or are some things, that we know we don’t know.

This hierarchy can be useful to help us think carefully about whether the uncertainty behind a problem can be defined in terms of a Level 1 prediction, with parameters for variation. Or, can it be resolved as group of Level 2 possibilities with probability estimates for each? Can the issue only be understood as a set of different Level 3 futures, each with a clear set of defined outcomes, or only by means of a Level 4 statement to the effect that we know only that there is something crucial that we don’t yet know?

There is often no clear or unanimous view of whether a particular uncertainty is set at a specific level. Uncertainty should always be considered at the deepest proposed level, unless or until those that propose this level can be convinced by an evidence-based argument that it should be otherwise.

Sources

  • Walker, W.E., Marchau, V.A.W.J. and Swanson, D. (2010) “Addressing Deep Uncertainty using Adaptive Policies: Introduction to Section 2”, Technological Forecasting & Social Change, 77: 917–23.

Types of Uncertainty

XKCD “Epistemic Uncertainty” https://xkcd.com/2440/

An important part of innovation, risk management, change management, continuous improvement is overcoming the fear of the unknown. We humans are wired with an intense aversion to both risk and uncertainty. Research shows that both have separate neural reactions and that choices with ambiguous outcomes trigger a stronger fear response than do risky choices. Additional research shows that the risk itself isn’t so much the problem, but the uncertainty is: we are afraid primarily because we don’t know the outcome and less so because of the risk.

There are three types of uncertainty:

  • Aleatoric Uncertainty: The uncertainty of quantifiable probabilities.
  • Epistemic Uncertainty: The uncertainty of knowledge. 
  • Knightian Uncertainty: The uncertainty of nonquantifiable risk.
A Two-Dimensional Framework for Characterizing Uncertainty from “Distinguishing Two Dimensions of Uncertainty” by Craig R. Fox and Gülden Ülkümen

I wrote more on this in my post “Uncertainty and Subjectivity in Risk Management.” This post mostly stems from wanting an excuse to share a funny comic.

Sensemaking, Foresight and Risk Management

I love the power of Karl Weick’s future-oriented sensemaking – thinking in the future perfect tense – for supplying us a framework to imagine the future as if it has already occurred. We do not spend enough time being forward-looking and shaping the interpretation of future events. But when you think about it quality is essentially all about using existing knowledge of the past to project a desired future.

This making sense of uncertainty – which should be a part of every manager’s daily routine – is another name for foresight. Foresight can be used as a discipline to help our organizations look into the future with the aim of understanding and analyzing possible future developments and challenges and supporting actors to actively shape the future.

Sensemaking is mostly used as a retrospective process – we look back at action that has already taken place, Weick himself acknowledged that people’s actions may be guided by future-oriented thoughts, he nevertheless asserted that the understanding that derives from sensemaking occurs only after the fact, foregrounding the retrospective quality of sensemaking even when imagining the future.

“When one imagines the steps in a history that will realize an outcome, then there is more likelihood that one or more of these steps will have been performed before and will evoke past experiences that are similar to the experience that is imagined in the future perfect tense.”

R.B. MacKay went further in a fascinating way by considering the role that counterfactual and prefactual processes play in future-oriented sensemaking processes. He finds that sensemaking processes can be prospective when they include prefactual “whatifs” about the past and the future. There is a whole line of thought stemming from this that looks at the meaning of the past as never static but always in a state of change.

Foresight concerns interpretation and understanding, while simultaneously being a process of thinking the future in order to improve preparedness. Though seeking to understand uncertainty, reduce unknown unknowns and drive a future state it is all about knowledge management fueling risk management.

Do Not Ignore Metaphor

A powerful tool in this reasoning, imagining and planning the future, is metaphor. Now I’m a huge fan of metaphor, though some may argue I make up horrible ones – I think my entire team is sick of the milk truck metaphor by now – but this underutilized tool can be incredibly powerful as we build stories of how it will be.

Think about phrases such as “had gone through”, “had been through” and “up to that point” as commonly used metaphors of emotional experiences as a physical movement or a journey from one point to another. And how much that set of journey metaphors shape much of our thinking about process improvement.

Entire careers have been built on questioning the heavy use of sport or war metaphors in business thought and how it shapes us. I don’t even watch sports and I find myself constantly using it as short hand.

To make sense of the future find a plausible answer to the question ‘what is the story?’, this brings a balance between thinking and acting, and allows us to see the future more clearly.

Bibliography

  • Cornelissen, J.P. (2012), “Sensemaking under pressure: the influence of professional roles and social accountability on the creation of sense”, Organization Science, Vol. 23 No. 1, pp. 118-137, doi: 10. 1287/orsc.1100.0640.
  • Greenberg, D. (1995), “Blue versus gray: a metaphor constraining sensemaking around a restructuring”, Group and Organization Management, Vol. 20 No. 2, pp. 183-209, available at: http://doi-org.esc-web.lib.cbs.dk:8443/10.1177/1059601195202007
  • Luscher, L.S. and Lewis, M.W. (2008), “Organizational change and managerial sensemaking: working through paradox”, Academy of Management Journal, Vol. 51 No. 2, pp. 221-240, doi: 10.2307/20159506.
  • MacKay, R.B. (2009), “Strategic foresight: counterfactual and prospective sensemaking in enacted environments”, in Costanzo, L.A. and MacKay, R.B. (Eds), Handbook of Research on Strategy and Foresight, Edward Elgar, Cheltenham, pp. 90-112, doi: 10.4337/9781848447271.00011
  • Tapinos, E. and Pyper, N. (2018), “Forward looking analysis: investigating how individuals “do” foresight and make sense of the future”, Technological Forecasting and Social Change, Vol. 126 No. 1, pp. 292-302, doi: 10.1016/j.techfore.2017.04.025.
  • Weick, K.E. (1979), The Social Psychology of Organizing, McGraw-Hill, New York, NY.
  • Weick, K.E. (1995), Sensemaking in Organizations, Sage, Thousand Oaks, CA.