Expert Intuition and Risk Management

Risk management is a crucial aspect of any organization or project. However, it is often subject to human errors in subjective risk judgments. This is because most risk assessment methods rely on subjective inputs from experts. Without certain precautions, experts can make consistent errors in judgment about uncertainty and risk.

There are methods that can correct the systemic errors that people make, but very few organizations implement them. As a result, there is often an almost universal understatement of risk. We need to keep in mind a few rules about experience and expertise.

• Experience is a nonrandom, nonscientific sample of events throughout our lifetime.
• Experience is memory-based and we are very selective regarding what we choose to remember,
• What we conclude from our experience can be full of logical errors
• Unless we get reliable feedback on past decisions, there is no reason to believe our experience will tell us much.

No matter how much experience we accumulate, we seem to be very inconsistent in its application.

Experts have unconscious heuristics and biases that impact their judgment, some important ones include:

• Misconceptions of chance: If you flip a coin six times, which result is more likely (H= heads, T= tails): HHHTTT or HTHTTH? They are both equal, but many people assume that because the first series looks “less random” than the second, it must be less likely. This is an example of representativeness bias. We appear to judge odds based on what we assume to be representative scenarios. Human beings easily confuse patterns and randomness.
• The conjunction fallacy: We often see specific events as more likely than broader categories of events.
• Irrational belief in small samples
• Disregarding variance in small samples. Small samples have more random variance that large samples is considered less than it should be.
• Insensitivity to prior probabilities: People tend to ignore the past and focus on new information when making subjective estimates.

This is all about overconfidence as an expert, which will consistently underestimate risks.

What are some ways to overcome this? I recommend the following be built into your risk management system.

• Pretend you are in the future looking back at failure. Start with the assumption that a major disaster did happen and describe how it happened.
• Look to risks from others. Gather a list of related failures, for example, regulatory agency observations, and think of risks in relation to those.
• Include Everyone. Your organization has numerous experts on all sorts of specific risks. Make the effort to survey representatives of just about every job level.
• Do peer reviews. Check assumptions by showing them to peers who are not immersed in the assessment.
• Implement metrics for performance. The Brier score is a way to evaluate the result of predictions both by how often the team was right and by the probability the estimated for getting a correct answer.

Further Reading

Here are some sources that discuss the topic of human errors and subjective judgments in risk management:

• “The Practical Aspect: The Human Elements of Risk” by Vasant Raval and Rajesh Sharma discusses the impact of human behavior on risk scenarios and the need for effective human reliability assessment.
• “How to Minimize Your Biases When Making Decisions” by Robert F. Wolf discusses the impact of behavioral biases on decision-making and provides suggestions for minimizing their impact.
• “Identifying the Biases Behind Your Bad Decisions” by John Beshears and Francesca Gino discusses the impact of behavioral biases on decision-making and provides suggestions for minimizing their impact.
• “Exploring the link between Human Error and Risk Management” by Marsh Commercial discusses the link between human error and risk management and provides insights into the prevention of accidents in the workplace.
• “Human Error and Human Factors” by HIROC discuss the need for risk managers and incident reviewers to understand the nature and types of human errors in order to prevent or mitigate future events.
• “Performance Management and the Human Error Factor: a New Perspective” by IRMI discusses the need to dig deeper into the inner workings of an organization to understand the reasons for human errors.