Information Gaps

An information gap is a known unknown, a question that one is aware of but for which one is uncertain of the answer. It is a disparity between what the decision maker knows and what could be known The attention paid to such an information gap depends on two key factors: salience, and importance.

  • The salience of a question indicates the degree to which contextual factors in a situation highlight it. Salience might depend, for example, on whether there is an obvious counterfactual in which the question can be definitively answered.
  • The importance of a question is a measure of how much one’s utility would depend on the actual answer. It is this factor—importance—which is influenced by actions like gambling on the answer or taking on risk that the information gap would be relevant for assessing.

Information gaps often dwell in the land of knightian uncertainty.

Communicating these Known Unknowns

Communicating around Known Unknowns and other forms of uncertainty

A wide range of reasons for information gaps exist:

  • variability within a sampled population or repeated measures leading to, for example, statistical margins-of-error
  • computational or systematic inadequacies of measurement
  • limited knowledge and ignorance about underlying processes
  • expert disagreement.

Levels of Uncertainty

Walker et al. (2010) developed a taxonomy of “levels of uncertainty”, ranging from Level 1 to Level 4,
which is useful in problem-solving:

  • Level 1 uncertainties are defined as relatively minor – as representing “a clear enough future” set within a “single system model” whereby outcomes can be estimated with reasonable accuracy;
  • Level 2 uncertainties display “alternative futures” but, again, within a single system in which probability estimates can be applied with confidence.

Levels 3 and 4 uncertainties are described as representing “deep uncertainty”.

  • Level 3 uncertainties are described as “a multiplicity of plausible futures”, in which multiple systems interact, but in which we can identify “a known range of outcomes”
  • Level 4 uncertainties lead us to an “unknown future” in which we don’t understand the system: we know only that there is something, or are some things, that we know we don’t know.

This hierarchy can be useful to help us think carefully about whether the uncertainty behind a problem can be defined in terms of a Level 1 prediction, with parameters for variation. Or, can it be resolved as group of Level 2 possibilities with probability estimates for each? Can the issue only be understood as a set of different Level 3 futures, each with a clear set of defined outcomes, or only by means of a Level 4 statement to the effect that we know only that there is something crucial that we don’t yet know?

There is often no clear or unanimous view of whether a particular uncertainty is set at a specific level. Uncertainty should always be considered at the deepest proposed level, unless or until those that propose this level can be convinced by an evidence-based argument that it should be otherwise.

Sources

  • Walker, W.E., Marchau, V.A.W.J. and Swanson, D. (2010) “Addressing Deep Uncertainty using Adaptive Policies: Introduction to Section 2”, Technological Forecasting & Social Change, 77: 917–23.

Types of Uncertainty

XKCD “Epistemic Uncertainty” https://xkcd.com/2440/

An important part of innovation, risk management, change management, continuous improvement is overcoming the fear of the unknown. We humans are wired with an intense aversion to both risk and uncertainty. Research shows that both have separate neural reactions and that choices with ambiguous outcomes trigger a stronger fear response than do risky choices. Additional research shows that the risk itself isn’t so much the problem, but the uncertainty is: we are afraid primarily because we don’t know the outcome and less so because of the risk.

There are three types of uncertainty:

  • Aleatoric Uncertainty: The uncertainty of quantifiable probabilities.
  • Epistemic Uncertainty: The uncertainty of knowledge. 
  • Knightian Uncertainty: The uncertainty of nonquantifiable risk.
A Two-Dimensional Framework for Characterizing Uncertainty from “Distinguishing Two Dimensions of Uncertainty” by Craig R. Fox and Gülden Ülkümen

I wrote more on this in my post “Uncertainty and Subjectivity in Risk Management.” This post mostly stems from wanting an excuse to share a funny comic.

Sensemaking, Foresight and Risk Management

I love the power of Karl Weick’s future-oriented sensemaking – thinking in the future perfect tense – for supplying us a framework to imagine the future as if it has already occurred. We do not spend enough time being forward-looking and shaping the interpretation of future events. But when you think about it quality is essentially all about using existing knowledge of the past to project a desired future.

This making sense of uncertainty – which should be a part of every manager’s daily routine – is another name for foresight. Foresight can be used as a discipline to help our organizations look into the future with the aim of understanding and analyzing possible future developments and challenges and supporting actors to actively shape the future.

Sensemaking is mostly used as a retrospective process – we look back at action that has already taken place, Weick himself acknowledged that people’s actions may be guided by future-oriented thoughts, he nevertheless asserted that the understanding that derives from sensemaking occurs only after the fact, foregrounding the retrospective quality of sensemaking even when imagining the future.

“When one imagines the steps in a history that will realize an outcome, then there is more likelihood that one or more of these steps will have been performed before and will evoke past experiences that are similar to the experience that is imagined in the future perfect tense.”

R.B. MacKay went further in a fascinating way by considering the role that counterfactual and prefactual processes play in future-oriented sensemaking processes. He finds that sensemaking processes can be prospective when they include prefactual “whatifs” about the past and the future. There is a whole line of thought stemming from this that looks at the meaning of the past as never static but always in a state of change.

Foresight concerns interpretation and understanding, while simultaneously being a process of thinking the future in order to improve preparedness. Though seeking to understand uncertainty, reduce unknown unknowns and drive a future state it is all about knowledge management fueling risk management.

Do Not Ignore Metaphor

A powerful tool in this reasoning, imagining and planning the future, is metaphor. Now I’m a huge fan of metaphor, though some may argue I make up horrible ones – I think my entire team is sick of the milk truck metaphor by now – but this underutilized tool can be incredibly powerful as we build stories of how it will be.

Think about phrases such as “had gone through”, “had been through” and “up to that point” as commonly used metaphors of emotional experiences as a physical movement or a journey from one point to another. And how much that set of journey metaphors shape much of our thinking about process improvement.

Entire careers have been built on questioning the heavy use of sport or war metaphors in business thought and how it shapes us. I don’t even watch sports and I find myself constantly using it as short hand.

To make sense of the future find a plausible answer to the question ‘what is the story?’, this brings a balance between thinking and acting, and allows us to see the future more clearly.

Bibliography

  • Cornelissen, J.P. (2012), “Sensemaking under pressure: the influence of professional roles and social accountability on the creation of sense”, Organization Science, Vol. 23 No. 1, pp. 118-137, doi: 10. 1287/orsc.1100.0640.
  • Greenberg, D. (1995), “Blue versus gray: a metaphor constraining sensemaking around a restructuring”, Group and Organization Management, Vol. 20 No. 2, pp. 183-209, available at: http://doi-org.esc-web.lib.cbs.dk:8443/10.1177/1059601195202007
  • Luscher, L.S. and Lewis, M.W. (2008), “Organizational change and managerial sensemaking: working through paradox”, Academy of Management Journal, Vol. 51 No. 2, pp. 221-240, doi: 10.2307/20159506.
  • MacKay, R.B. (2009), “Strategic foresight: counterfactual and prospective sensemaking in enacted environments”, in Costanzo, L.A. and MacKay, R.B. (Eds), Handbook of Research on Strategy and Foresight, Edward Elgar, Cheltenham, pp. 90-112, doi: 10.4337/9781848447271.00011
  • Tapinos, E. and Pyper, N. (2018), “Forward looking analysis: investigating how individuals “do” foresight and make sense of the future”, Technological Forecasting and Social Change, Vol. 126 No. 1, pp. 292-302, doi: 10.1016/j.techfore.2017.04.025.
  • Weick, K.E. (1979), The Social Psychology of Organizing, McGraw-Hill, New York, NY.
  • Weick, K.E. (1995), Sensemaking in Organizations, Sage, Thousand Oaks, CA.

Identifying Waste in Risk Management

Risk Management often devolves into a check-the-box, non-valued activity in an organization. While many organizations ensure they have the right processes in place, they still end up not protecting themselves against risk effectively. A lot of our organizations struggle to understand risk and apply this mindset in productive ways.

As quality professionals we should be applying the same improvement tools to our risk management processes as we do anything else.

To improve a process, we first need to understand the value from the process. Risk management is the identification, evaluation, and prioritization of risks (defined in ISO 31000 as the effect of uncertainty on objectives) followed by coordinated and economical application of resources to minimize, monitor, and control the probability or impact of unfortunate events or to maximize the realization of opportunities.

Risk management then is an application of decision quality to reduce uncertainty on objectives. We can represent the process this way:

The risk evaluation is the step where the knowledge base is evaluated, and a summary judgment is reached on the risks and uncertainties involved in the case under investigation. This evaluation must take the values of the decision-makers into account and a careful understanding has to be had on just what the practical burden of proof is in the particular decision.

Does Risk Management then create value for those perceived by the stakeholders? Can we apply a value stream approach and look to reduce wastes?  Some common ones include:

Waste in Risk ManagementExampleReflects
Defective Information“The things that hurts you is never in a risk matrix”  “You have to deliver a risk matrix, but how you got there doesn’t matter”Missing stakeholder viewpoints, poor Risk Management process, lack of considering multiple sources of uncertainty, poor input data, lack of sharing information
Overproduction“if it is just a checklist sitting somewhere, then people don’t use it, and it becomes a wasted effort”Missing standardization, serial processing and creation of similar documents, reports are not used after creation
Stockpiling Information“we’re uncertain what are the effect of the risk as this early stage, I think it would make more sense to do after”Documented risk lay around unutilized during a project, change or operations
Unnecessary movement of people“It can be time consuming walking around to get information about risk”Lack of documentation, risks only retrievable by going around asking employees
Rework“Time spend in risk identification is always little in the beginning of a project because everybody wants to start and then do the first part as quickly as possible.”Low quality initial work, ‘tick the-box’ risk management
Information rot“Risk reports are always out of date”The documents were supposed to be updated and re-evaluated, but was not, thus becoming partially obsolete over time
Common wastes in Risk Management

Once we understand waste in risk management we can identify when it happens and engage in improvement activities. We should do this based on the principles of decision quality and very aware of the role uncertainty applies.

References

  • Anjum, Rani Lill, and Elena Rocca. “From Ideal to Real Risk: Philosophy of Causation Meets Risk Analysis.” Risk Analysis, vol. 39, no. 3, 19 Sept. 2018, pp. 729–740, 10.1111/risa.13187.
  • Hansson, Sven Ove, and Terje Aven. “Is Risk Analysis Scientific?” Risk Analysis, vol. 34, no. 7, 11 June 2014, pp. 1173–1183, 10.1111/risa.12230
  • Walker, Warren E., et al. “Deep Uncertainty.” Encyclopedia of Operations Research and Management Science, 2013, pp. 395–402, 10.1007/978-1-4419-1153-7_1140
  • Willumsen, Pelle, et al. “Value Creation through Project Risk Management.” International Journal of Project Management, Feb. 2019, 10.1016/j.ijproman.2019.01.007