Identifying Waste in Risk Management

Risk Management often devolves into a check-the-box, non-valued activity in an organization. While many organizations ensure they have the right processes in place, they still end up not protecting themselves against risk effectively. A lot of our organizations struggle to understand risk and apply this mindset in productive ways.

As quality professionals we should be applying the same improvement tools to our risk management processes as we do anything else.

To improve a process, we first need to understand the value from the process. Risk management is the identification, evaluation, and prioritization of risks (defined in ISO 31000 as the effect of uncertainty on objectives) followed by coordinated and economical application of resources to minimize, monitor, and control the probability or impact of unfortunate events or to maximize the realization of opportunities.

Risk management then is an application of decision quality to reduce uncertainty on objectives. We can represent the process this way:

The risk evaluation is the step where the knowledge base is evaluated, and a summary judgment is reached on the risks and uncertainties involved in the case under investigation. This evaluation must take the values of the decision-makers into account and a careful understanding has to be had on just what the practical burden of proof is in the particular decision.

Does Risk Management then create value for those perceived by the stakeholders? Can we apply a value stream approach and look to reduce wastes?  Some common ones include:

Waste in Risk ManagementExampleReflects
Defective Information“The things that hurts you is never in a risk matrix”  “You have to deliver a risk matrix, but how you got there doesn’t matter”Missing stakeholder viewpoints, poor Risk Management process, lack of considering multiple sources of uncertainty, poor input data, lack of sharing information
Overproduction“if it is just a checklist sitting somewhere, then people don’t use it, and it becomes a wasted effort”Missing standardization, serial processing and creation of similar documents, reports are not used after creation
Stockpiling Information“we’re uncertain what are the effect of the risk as this early stage, I think it would make more sense to do after”Documented risk lay around unutilized during a project, change or operations
Unnecessary movement of people“It can be time consuming walking around to get information about risk”Lack of documentation, risks only retrievable by going around asking employees
Rework“Time spend in risk identification is always little in the beginning of a project because everybody wants to start and then do the first part as quickly as possible.”Low quality initial work, ‘tick the-box’ risk management
Information rot“Risk reports are always out of date”The documents were supposed to be updated and re-evaluated, but was not, thus becoming partially obsolete over time
Common wastes in Risk Management

Once we understand waste in risk management we can identify when it happens and engage in improvement activities. We should do this based on the principles of decision quality and very aware of the role uncertainty applies.

References

  • Anjum, Rani Lill, and Elena Rocca. “From Ideal to Real Risk: Philosophy of Causation Meets Risk Analysis.” Risk Analysis, vol. 39, no. 3, 19 Sept. 2018, pp. 729–740, 10.1111/risa.13187.
  • Hansson, Sven Ove, and Terje Aven. “Is Risk Analysis Scientific?” Risk Analysis, vol. 34, no. 7, 11 June 2014, pp. 1173–1183, 10.1111/risa.12230
  • Walker, Warren E., et al. “Deep Uncertainty.” Encyclopedia of Operations Research and Management Science, 2013, pp. 395–402, 10.1007/978-1-4419-1153-7_1140
  • Willumsen, Pelle, et al. “Value Creation through Project Risk Management.” International Journal of Project Management, Feb. 2019, 10.1016/j.ijproman.2019.01.007

VUCA – Accented Just Right It is a Profanity

Talk about strategy, risk management or change and it is inevitable that the acronym VUCA — short for volatility, uncertainty, complexity, and ambiguity—will come up. VUCA is basically a catchall for “Hey, it’s crazy out there!” And like many catch-all’s it is misleading, VUCA conflates four distinct types of challenges that demand four distinct types of responses. VUCA can quickly become a crutch, a way to throw off the hard work of strategy and planning—after all, you can’t prepare for a VUCA world, right?

The mistake folks often make here is treating these four traits as a single idea, which leads to poorer decision making.

VUCA really isn’t a tool. It’s a checklist of four things that hopefully your system is paying attention to. All four represent distinct elements that make our environment and organization harder to grasp and control. 

Probing Unknown Unknowns

In the post “Risk Management is about reducing uncertainty,” I discussed ignorance and surprise, covering the idea of “unknown unknowns”, those things that we don’t even know that we don’t know.

Our goal should always be to reduce ignorance. Many unknown unknowns are just things no one has bothered to find out. What we need to do is ensure our processes and systems are constructed so that they recognize unknowns.

There are six factors that need to be explored to find the unknown unknowns.

  1. Complexity: A complex process/system/project contains many interacting elements that increase the variety of its possible behaviors and results. Complexity increases with the number, variety, and lack of robustness of the elements of the process, system or project.
  2. Complicatedness: A complicated process/system/project involves many points of failure, the ease of finding necessary elements and identifying cause-and-effect relationships; and the experts/participants aptitudes and experiences.
  3. Dynamism: The volatility or the propensity of elements and relationships to change.
  4. Equivocality: Knowledge management is a critical enabler of product and project life cycle management. If the information is not crisp and specific, then the people who receive it will be equivocal and won’t be able to make firm decisions. Although imprecise information itself can be a known unknown, equivocality increases both complexity and complicatedness. 
  5. Perceptive barriers: Mindlessness. This factor includes a lot of our biases, including an over-reliance on past experiences and traditions, the inability to detect weak signals and ignoring input that is inconvenient or unappealing.
  6. Organizational pathologies: Organizations have problems, culture can have weaknesses. These structural weaknesses allow unknown unknowns to remain hidden.
Interrogating Knowable Unknown Unknowns

The way to address these six factors is to evaluate and challenge by using the following approaches:

Interviewing

Interviews with stakeholders, subject matter experts and other participants can be effective tools for uncovering lurking problems and issues. Interviewers need to be careful not to be too enthusiastic about the projects they’re examining and not asking “yes or no” questions. The best interviews probe deep and wide.

Build Knowledge by Decomposing the System/Process/Project

Standard root cause analysis tools apply here, break it down and interrogate all the subs.

  1. Identifying the goals, context, activities and cause-effect relationships
  2. Breaking the domains into smaller elements — such as processes, tasks and stakeholders
  3. Examining the complexity and uncertainty of each element to identify the major risks (known unknowns) that needed managing and the knowledge gaps that pointed to areas of potential unknown unknowns.

Analyze Scenarios

Construct several different future outlooks and test them out (mock exercises are great). This approach accepts uncertainty, tries to understand it and builds it into the your knowledge base and reasoning. Rather than being predictions, scenarios are coherent and credible alternative futures built on dynamic events and conditions that are subject to change.

Communicate Frequently and Effectively

Regularly and systematically reviewing decision-making and communication processes, including the assumptions that are factored into the processes, and seeking to remove information asymmetries, can help to anticipate and uncover known unknowns. Management Review is part of this, but not the only component. Effective and frequent communication is essential for adaptability and agility. However, this doesn’t necessarily mean communicating large volumes of information, which can cause information overload. Rather, the key is knowing how to reach the right people at the right times. Some important aspects include:

  • Candor: Timely and honest communication of missteps, anomalies and missing competencies. Offer incentives for candor to show people that there are advantages to owning up to errors or mistakes in time for management to take action. It is imperative to eliminate any perverse incentives that induce people to ignore emerging risks.
  • Cultivate an Alert Culture: A core part of a quality culture should be an alert culture made up of people who strive to illuminate rather than hide potential problems. Alertness is built by: 1) emphasizing systems thinking; 2) seek to include and build a wide range of experiential expertise — intuitions, subtle understandings and finely honed reflexes gained through years of intimate interaction with a particular natural, social or technological system; and 3) learn from surprising outcomes.

By working to evaluate and challenge, to truly understand our systems and processes, our risk management activities will be more effective and truly serve to make our systems resilient.

Recommended Reading

Uncertainty and Subjectivity in Risk Management

The July-2019 monthly gift to members of the ASQ is a lot of material on Failure Mode and Effect Analysis (FMEA). Reading through the material got me to thinking of subjectivity in risk management.

Risk assessments have a core of the subjective to them, frequently including assumptions about the nature of the hazard, possible exposure pathways, and judgments for the likelihood that alternative risk scenarios might occur. Gaps in the data and information about hazards, uncertainty about the most likely projection of risk, and incomplete understanding of possible scenarios contribute to uncertainties in risk assessment and risk management. You can go even further and say that risk is socially constructed, and that risk is at once both objectively verifiable and what we perceive or feel it to be. Then again, the same can be said of most of science.

Risk is a future chance of loss given exposure to a hazard. Risk estimates, or qualitative ratings of risk, are necessarily projections of future consequences. Thus, the true probability of the risk event and its consequences cannot be known in advance. This creates a need for subjective judgments to fill-in information about an uncertain future. In this way risk management is rightly seen as a form of decision analysis, a form of making decisions against uncertainty.

Everyone has a mental picture of risk, but the formal mathematics of risk analysis are inaccessible to most, relying on probability theory with two major schools of thought: the frequency school and the subjective probability school. The frequency school says probability is based on a count of the number of successes divided by total number of trials. Uncertainty that is ready characterized using frequentist probability methods is “aleatory” – due to randomness (or random sampling in practice). Frequentist methods give an estimate of “measured” uncertainty; however, it is arguably trapped in the past because it does not lend itself to easily to predicting future successes.

In risk management we tend to measure uncertainty with a combination of frequentist and subjectivist probability distributions. For example, a manufacturing process risk assessment might begin with classical statistical control data and analyses. But projecting the risks from a process change might call for expert judgments of e.g. possible failure modes and the probability that failures might occur during a defined period. The risk assessor(s) bring prior expert knowledge and, if we are lucky, some prior data, and start to focus the target of the risk decision using subjective judgments of probabilities.

Some have argued that a failure to formally control subjectivity — in relation to probability judgments – is the failure of risk management. This was an argument that some made during WCQI, for example. Subjectivity cannot be eliminated nor is it an inherent limitation. Rather, the “problem with subjectivity” more precisely concerns two elements:

  1. A failure to recognize where and when subjectivity enters and might create problems in risk assessment and risk-based decision making; and
  2. A failure to implement controls on subjectivity where it is known to occur.

Risk is about the chance of adverse outcomes of events that are yet to occur, subjective judgments of one form or another will always be required in both risk assessment and risk management decision-making.

We control subjectivity in risk management by:

  • Raising awareness of where/when subjective judgments of probability occur in risk assessment and risk management
  • Identifying heuristics and biases where they occur
  • Improving the understanding of probability among the team and individual experts
  • Calibrating experts individually
  • Applying knowledge from formal expert elicitation
  • Use expert group facilitation when group probability judgments are sought

Each one of these is it’s own, future, post.