Build Key Risk Indicators

We perform risk assessments; execute risk mitigations; and we end up with four types of inherent risks (parenthesis is opportunities) in our risk register:

Mitigated (or enhanced)
Avoided (or exploited)
Transferred (or shared)
Accepted

We’ve built a set of risk response plans to ensure we are continuing to treat these risks. And now we need to monitor the effectiveness of our risk plan and to ensure that the risks are behaving in the manner anticipated during risk treatment.

The living risk assessment is designed to conduct reassessment of risks after treatment and continuously throughout the life cycle. However, not all systems and risks need to be reassessed continually, and the organization should prioritize which systems should be reassessed based on a schedule.

Identify indicators that inform the organization about the status of the risk without having to conduct a full risk assessment every time. The trending status of these indicators can act as a flag for investigations, which may result in complete risk assessments.

This risk indicator is then a metric that indicates the state of the level of risk. It is important to note that not all indicators show the exact level of risk exposure, instead providing a trend of drivers, causes or intermediary effects of risk.

The most important risks can be categorized as key risks and the indicators for these key risks are known as key risk indicators (KRIs) which can be defined as: A metric that provides a leading or lagging indicator of the current state of risk exposure on key objectives. KRIs can be used to continually assess current and predict potential risk exposures.

These KRIs need to have a strong relationship with the key performance indicators of the organization.

KRIs are monitored through Quality Management Review.

A good rule of thumb is as you identify the key performance indicators to assess the performance of a specific process, product, system or function you then identify the risks and the KRIs for that objective.

Strive to have leading indicators that measure the elements that influences the risk performance. Lagging indicators will measure they actual performance of the risk controls.

These KRIs qualitatively or quantitatively present the risk exposure by having a strong relationship qirh the risk, its intermediate output or its drivers.

Let’s think in terms of a pharmaceutical supply chain. We’ve done our risk assessments and end up with a top level view like this:

For the risk column we should have some good probabilities and impacts and mitigations in place. We can then chose some KRIs to monitor, such as

Nonconformance rate
Supplier score card
Lab error rate
Product Complaints

As we develop, our KRIs can get more specific and focused. A good KRI is:

Quantifiable
Measurable (accurately and precisely)
Can be validated (have a high level of confidence)
Relevant (measuring the right thing associated with decisions)

In developing a KRI to serve as a leading indicator for potential future occurrences of a risk, it can be helpful to think through the chain of events that led to the event so that management can uncover the ultimate driver (i.e., root cause(s)) of the risk event. When KRIs for root cause events and intermediate events are monitored, we are in an enviable position to identify early mitigation strategies that can begin to reduce or eliminate the impact associated with an emerging risk event.

These KRIs will help us monitor and quantify our risk exposure. They help our organizations compare business objectives and strategy to actual performance to isolate changes, measure the effectiveness of processes or projects, and demonstrate changes in the frequency or impact of a specific risk event.

Effective KRIs can provide value to the organization in a variety of ways. Potential value may be derived from each of the following contributions:

Risk Appetite – KRIs require the determination of appropriate thresholds for action at different levels within the organization. By mapping KRI measures to identified risk appetite and tolerance levels, KRIs can be a useful tool for better articulating the risk appetite that best represents the organizational mindset.
Risk and Opportunity Identification – KRIs can be designed to alert management to trends that may adversely affect the achievement of organizational objectives or may indicate the presence of new opportunities.
Risk Treatment – KRIs can initiate action to mitigate developing risks by serving as triggering mechanisms. KRIs can serve as controls by defining limits to certain actions.

The Risk Register

Every organization should ask themselves seven questions about the health of their risk management program.

Do you have a risk management plan?
Have you identified and captured your risks in a risk register?
How have you evaluated and prioritized your risks?
Have you engaged the appropriate stakeholders in the risk identification and evaluation processes?
What about risk owners? Does each risk have a risk owner?
Have the risk owners developed risk response plans for the highest risks?
Are you facilitating a review of your risks periodically, resulting in updates to the risk register and effective risk responses?

At the heart of this program sits the Risk Register, which brings together information about risks to inform those exposed to risks and those who have responsibility for their management. A risk register is used to record and track information about individual risks and how they are being controlled. It can be used to communicate information about risks to stakeholders and highlight particularly important risks. While it can be used at any level of the organization where there are a large number of risks, controls and treatments that need to be tracked, a risk register really shines as a central component of a quality management review. The risk register includes:

List of risks, failure modes or hazards and expected outcomes
A statement about the probability of consequences occurring
Sources or causes of the risk
Priority or risk levels
What is currently being done to control the risk
Risk owner
Actual outcome, if and when available

Risks are generally listed individually as separate events but interdependencies should be flagged.

In recording information about risks, the distinction between risks (the potential effects of what might happen) and risk sources (how or why it might happen) and controls that might fail should be explicit. It can also be useful to indicate the early warning signs that an event might be about to occur.

Many risk registers also include some rating of the significance of a risk, an indication of whether a risk is considered to be acceptable or tolerable, or whether further treatment is needed and the reasons for this decision. Where a significance rating is applied to a risk based on consequences and their likelihood, this should take account of the possibility that controls will fail. A level of risk should not be allocated for the failure of a control as if it were an independent risk.

A risk register is used as the basis for tracking implementation of proposed treatments, so it should contain information about treatments and how they will be implemented, or make reference to other documents or data bases with this information. (Such information can include risk owners, actions, action owners, action business case summaries, budgets and timelines, etc.). This living document can usually roll (or even serve as) the Quality Plan.

Strengths of risk registers include the following.

Information about risks is brought together in a form where actions required can be identified and tracked.
Information about different risks is presented in a comparable format, which can be used to indicate priorities and is relatively easy to interrogate.
The construction of a risk register usually involves many people and raises general awareness of the need to manage risk.

By doing this, the risk register serves as a central underpining for the organization as it builds a risk culture, driving transparency and accountability.

Building Risk Based Thinking in the Organization requires a strong governance structure

Pay attention the the following limitations:

Risks captured in risk registers are typically based on events, which can make it difficult to accurately characterize some forms of risk
The apparent ease of use can give misplaced confidence in the information because it can be difficult to describe risks consistently and sources of risk, risks, and weaknesses in controls for risk are often confused.
There are many different ways to describe a risk and any priority allocated will depend on the way the risk is described and the level of disaggregation of the issue.
Considerable effort is required to keep a risk register up to date (for example, all proposed treatments should be listed as current controls once they are implemented, new risks should be continually added and those that no longer exist removed).
Risks are typically captured in risk registers individually. This can make it difficult to consolidate information to develop an overall treatment program.

Artifacts, like the risk register, both demonstrate and channel culture. Invest the time in your organization’s register, and you will reap dividends towards developing a risk friendly culture.

Structured What-If Technique as a Risk Assessment Tool

The structured what-if technique, SWIFT, is a high-level and less formal risk identification technique that can be used independently, or as part of a staged approach to make bottom-up methods such as FMEA more efficient. SWIFT uses structured brainstorming in a facilitated workshop where a predetermined set of guidewords (timing, amount, etc.) are combined with prompts elicited from participants that often begin with phrases such as “what if?” or “how could?”.

At the heart of a SWIFT is a list of guidewords to enable a comprehensive review of risks or sources of risk. At the start of the workshop the context, scope and purpose of the SWIFT is discussed and criteria for success articulated. Using the guidewords and “what if?” prompts, the facilitator asks the participants to raise and discuss issues such as:

known risks
risk sources and drivers
previous experience, successes and incidents
known and existing controls
regulatory requirements and constraints

The list of guidewords is utilized by the facilitator to monitor the discussion and to suggest additional issues and scenarios for the team to discuss. The team considers whether controls are adequate and if not considers potential treatments. During this discussion, further “what if?” questions are posed.

Often the list of risks generated can be used to fuel a qualitative or semi-quantitative risk assessment method, such as an FMEA is.

A SWIFT Analysis allows participants to look at the system response to problems rather than just examining the consequences of component failure. As such, it can be used to identify opportunities for improvement of processes and systems and generally can be used to identify actions that lead to and enhance their probabilities of success.

What-If Analysis

What–If Analysis is a structured brainstorming method of determining what things can go wrong and judging the likelihood and consequences of those situations occurring. The answers to these questions form the basis for making judgments regarding the acceptability of those risks and determining a recommended course of action for those risks judged to be unacceptable. An experienced review team can effectively and productively discern major issues concerning a process or system. Lead by an energetic and focused facilitator, each member of the review team participates in assessing what can go wrong based on their past experiences and knowledge of similar situations.

What If?	Answer	Likelihood	Severity	Recommendations
What could go wrong?	What would happen if it did?	How likely?	Consequences	What will we do about them Again – prevent and monitor

What-If Analysis

Steps in a SWIFT Analysis

Prepare the guide words: The facilitator should select a set of guide words to be used in the SWIFT.
Assemble the team: Select participants for the SWIFT workshop based on their knowledge of the system/process being assessed and the degree to which they represent the full range of stakeholder groups.
Background: Describe the trigger for the SWIFT (e.g., a regulatory change, an adverse event, etc.).
Articulate the purpose: Clearly explain the purpose to be served by the SWIFT (e.g., to improve effectiveness of the process).
Define the requirements: Articulate the criteria for success
Describe the system: Provide appropriate-level textual and graphical descriptions of the system or process to be risk assessed. A clear understanding is necessary and can be is established through interviews, gathering a multifunctional team and through the study of documents, plans and other records. Normally the
Identify the risks/hazards: This is where the structured what-if technique is applied. Use the guide words/headings with each system, high-level subsystem, or process step in turn. Participants should use prompts starting with the phrases like “What if…” or “How could…” to elicit potential risks/hazards associated with the guide word. For instance, if the process is “Receipt of samples,” and the guide word is “time, timing or speed,” prompts might include: “What if the sample is delivered at a shift change” (wrong time) or “How could the sample be left waiting too long in ambient conditions?” (wrong timing).
Assess the risks: With the use of either a generic approach or a supporting risk analysis technique, estimate the risk associated with the identified hazards. In light of existing controls, assess the likelihood that they could lead to harm and the severity of harm they might cause. Evaluate the acceptability of these risk levels, and identify any aspects of the system that may require more detailed risk identification and analysis.
Propose actions: Propose risk control action plans to reduce the identified risks to an acceptable level.
Review the process: Determine whether the SWIFT met its objectives, or whether a more detailed risk assessment is required for some parts of the system.
Document: Produce an overview document to communicate the results of the SWIFT.
Additional risk assessment: Conduct additional risk assessments using more detailed or quantitative techniques, if required. The SWIFT Analysis is really effective as a filtering mechanism to focus effort on the most valuable areas.

Guideword Examples

The facilitator and process owner can choose any guide words that seem appropriate. Guidewords usually stem around:

Wrong: Person or people
Wrong: Place, location, site, or environment
Wrong: Thing or things
Wrong: Idea, information, or understanding
Wrong: Time, timing, or speed
Wrong: Process
Wrong: Amount
Failure: Control or Detection
Failure: Equipment

If your organization has invested time to create root cause categories and sub-categories, the guidewords can easily start there.

Information Gaps

An information gap is a known unknown, a question that one is aware of but for which one is uncertain of the answer. It is a disparity between what the decision maker knows and what could be known The attention paid to such an information gap depends on two key factors: salience, and importance.

The salience of a question indicates the degree to which contextual factors in a situation highlight it. Salience might depend, for example, on whether there is an obvious counterfactual in which the question can be definitively answered.
The importance of a question is a measure of how much one’s utility would depend on the actual answer. It is this factor—importance—which is influenced by actions like gambling on the answer or taking on risk that the information gap would be relevant for assessing.

Information gaps often dwell in the land of knightian uncertainty.

Communicating these Known Unknowns

Communicating around Known Unknowns and other forms of uncertainty

A wide range of reasons for information gaps exist:

variability within a sampled population or repeated measures leading to, for example, statistical margins-of-error
computational or systematic inadequacies of measurement
limited knowledge and ignorance about underlying processes
expert disagreement.

Ambiguity

Ambiguity is present in virtually all real-life situations and are those ‘situations in which we do not have sufficient information to quantify the stochastic nature of the problem. It is a lack of knowledge as
to the ‘basic rules of the game’ where cause-and-effect are not understood and there is no precedent for
making predictions as to what to expect

Ambiguity is often used, especially in the context of VUCA, to cover situations in situations that have:

Doubt about the nature of cause and effect
Little to no historical information to predict the outcome
Difficult to forecast or plan for

It is important to answer whether there are risks of lack of experience and predictability that might affect the situation, and interrogate our unknown unknowns.

People are ambiguity averse in that they prefer situations in which probabilities are perfectly known to situations in which they are unknown.

Ambiguity is best resolved by experimentation.