Probing Unknown Unknowns

In the post “Risk Management is about reducing uncertainty,” I discussed ignorance and surprise, covering the idea of “unknown unknowns”, those things that we don’t even know that we don’t know.

Our goal should always be to reduce ignorance. Many unknown unknowns are just things no one has bothered to find out. What we need to do is ensure our processes and systems are constructed so that they recognize unknowns.

There are six factors that need to be explored to find the unknown unknowns.

1. Complexity: A complex process/system/project contains many interacting elements that increase the variety of its possible behaviors and results. Complexity increases with the number, variety, and lack of robustness of the elements of the process, system or project.
2. Complicatedness: A complicated process/system/project involves many points of failure, the ease of finding necessary elements and identifying cause-and-effect relationships; and the experts/participants aptitudes and experiences.
3. Dynamism: The volatility or the propensity of elements and relationships to change.
4. Equivocality: Knowledge management is a critical enabler of product and project life cycle management. If the information is not crisp and specific, then the people who receive it will be equivocal and won’t be able to make firm decisions. Although imprecise information itself can be a known unknown, equivocality increases both complexity and complicatedness. 
5. Perceptive barriers: Mindlessness. This factor includes a lot of our biases, including an over-reliance on past experiences and traditions, the inability to detect weak signals and ignoring input that is inconvenient or unappealing.
6. Organizational pathologies: Organizations have problems, culture can have weaknesses. These structural weaknesses allow unknown unknowns to remain hidden.

The way to address these six factors is to evaluate and challenge by using the following approaches:

Interviewing

Interviews with stakeholders, subject matter experts and other participants can be effective tools for uncovering lurking problems and issues. Interviewers need to be careful not to be too enthusiastic about the projects they’re examining and not asking “yes or no” questions. The best interviews probe deep and wide.

Build Knowledge by Decomposing the System/Process/Project

Standard root cause analysis tools apply here, break it down and interrogate all the subs.

1. Identifying the goals, context, activities and cause-effect relationships
2. Breaking the domains into smaller elements — such as processes, tasks and stakeholders
3. Examining the complexity and uncertainty of each element to identify the major risks (known unknowns) that needed managing and the knowledge gaps that pointed to areas of potential unknown unknowns.

Analyze Scenarios

Construct several different future outlooks and test them out (mock exercises are great). This approach accepts uncertainty, tries to understand it and builds it into the your knowledge base and reasoning. Rather than being predictions, scenarios are coherent and credible alternative futures built on dynamic events and conditions that are subject to change.

Communicate Frequently and Effectively

Regularly and systematically reviewing decision-making and communication processes, including the assumptions that are factored into the processes, and seeking to remove information asymmetries, can help to anticipate and uncover known unknowns. Management Review is part of this, but not the only component. Effective and frequent communication is essential for adaptability and agility. However, this doesn’t necessarily mean communicating large volumes of information, which can cause information overload. Rather, the key is knowing how to reach the right people at the right times. Some important aspects include:

• Candor: Timely and honest communication of missteps, anomalies and missing competencies. Offer incentives for candor to show people that there are advantages to owning up to errors or mistakes in time for management to take action. It is imperative to eliminate any perverse incentives that induce people to ignore emerging risks.
• Cultivate an Alert Culture: A core part of a quality culture should be an alert culture made up of people who strive to illuminate rather than hide potential problems. Alertness is built by: 1) emphasizing systems thinking; 2) seek to include and build a wide range of experiential expertise — intuitions, subtle understandings and finely honed reflexes gained through years of intimate interaction with a particular natural, social or technological system; and 3) learn from surprising outcomes.

By working to evaluate and challenge, to truly understand our systems and processes, our risk management activities will be more effective and truly serve to make our systems resilient.

Recommended Reading

• E. Lamarre and M. Pergler, “Risk: Seeing Around the Corners,” McKinsey Quarterly, October 2009: 102-106.
• S. Leleur, “Systemic Planning: Dealing With Complexity by a Wider Approach to Planning,” Emergence: Complexity & Organization 9, no. 1-2 (2007): 2-10.
• C.H. Loch, M.E. Solt and E.M. Bailey, “Diagnosing Unforeseeable Uncertainty in a New Venture,” Journal of Product Innovation Management 25, no. 1 (January 2008): 28-46.
• P.J.H. Schoemaker, “Scenario Planning: A Tool for Strategic Thinking,” Sloan Management Review 36, no. 2 (winter 1995): 25-40.
• V. Tang and V. Salminen, “Towards a Theory of Complicatedness: Framework for Complex Systems Analysis and Design” (paper presented at the 13th International Conference on Engineering Design, Glasgow, Scotland, August 2001).
• K.E. Weick and K.M. Sutcliffe, “Mindfulness and the Quality of Organizational Attention,” Organization Science 17, no. 4 (July-August 2006): 514-524.