Risk assessment is a pillar of the quality system because it gives us the ability to anticipate in a consistent manner. It is built on some fundamental criteria:
When I teach an introductory risk management class, I usually use an icebreaker of “What is the riskiest activity you can think of doing. Inevitably you will get some version of skydiving, swimming with sharks, jumping off bridges. This activity is great because it starts all conversations around likelihood and severity. At heart, the question brings out the concept of risk important activities and the nature of controls.
The things people think of, such as skydiving, are great examples of activities that are surrounded by activities that control risk. The very activity is based on accepting reducing risk as low as possible and then proceeding in the safest possible pathway. These risk important activities are the mechanism just before a critical step that:
Ensure the appropriate transfer of information and skill
Ensure the appropriate number of actions to reduce risk
Influence the presence or effectiveness of barriers
Influence the ability to maintain positive control of the moderation of hazards
Risk important activities is a concept important to safety-thought and are at the center of a lot of human error reduction tools and practices. Risk important activities are all about thinking through the right set of controls, building them into the procedure, and successfully executing them before reaching the critical step of no return. Checklists are a great example of this mindset at work, but there are a ton of ways of doing them.
In the hospital they use a great thought process, “Five rights of Safe Medication Practices” that are: 1) right patient, 2) right drug, 3) right dose, 4) right route, and 5) right time. Next time you are getting medication in the doctor’s office or hospital evaluate just what your caregiver is doing and how it fits into that process. Those are examples of risk important activities.
Assessing controls during risk assessment
Risk is affected by the overall effectiveness of any controls that are in place.
The key aspects of controls are:
the mechanism by which the controls are intended to modify risk
whether the controls are in place, are capable of operating as intended, and are achieving the expected results
whether there are shortcomings in the design of controls or the way they are applied
whether there are gaps in controls
whether controls function independently, or if they need to function collectively to be effective
whether there are factors, conditions, vulnerabilities or circumstances that can reduce or eliminate control effectiveness including common cause failures
A risk can have more than one control and controls can affect more than one risk.
We always want to distinguish between controls that change likelihood, consequences or both, and controls that change how the burden of risk is shared between stakeholders
Any assumptions made during risk analysis about the actual effect and reliability of controls should be validated where possible, with a particular emphasis on individual or combinations of controls that are assumed to have a substantial modifying effect. This should take into account information gained through routine monitoring and review of controls.
Risk Important Activities, Critical Steps and Process
Critical steps are the way we meet our critical-to-quality requirements. The activities that ensure our product/service meets the needs of the organization.
These critical steps are the points of no-return, the point where the work-product is transformed into something else. Risk important activities are what we do to remove the danger of executing that critical step.
Beyond that critical step, you have rejection or rework. When I am cooking there is a lot of prep work which can be a mixture of critical steps, from which there is no return. I break the egg wrong and get eggshells in my batter, there is a degree of rework necessary. This is true for all our processes.
The risk-based approach to the process is to understand the critical steps and mitigate controls.
We are thinking through the following:
Critical Step: The action that triggers irreversibility. Think in terms of critical-to-quality attributes.
Output: The desired result (positive) or the possible difficulty (negative)
Preconditions: Technical conditions that must exist before the critical step
Resources: What is needed for the critical step to be completed
Local factors: Things that could influence the critical step. When human beings are involved, this is usually what can influence the performer’s thinking and actions before and during the critical step
Good risk management requires a mindset that includes the following attributes:
Expect to be surprised: Our processes are usually underspecified and there is a lot of hidden knowledge. Risk management serves to interrogate the unknowns
Possess a chronic sense of unease: There is no such thing as perfect processes, procedures, training, design, planning. Past performance is not a guarantee of future success.
Bend, not break: Everything is dynamic, especially risk. Quality comes from adaptability.
One cannot control risk, or even successfully identify it unless a system is able flexibly to monitor both its own performance (what happens inside the system’s boundary) and what happens in the environment (outside the system’s boundary). Monitoring improves the ability to cope with possible risks
When performing the risk assessment, challenge existing monitoring and ensure that the right indicators are in place. But remember, monitoring itself is a low-effectivity control.
Ensure that there are leading indicators, which can be used as valid precursors for changes and events that are about to happen.
For each monitoring control, as yourself the following:
How have the indicators been defined? (By analysis, by tradition, by industry consensus, by the regulator, by international standards, etc.)
Relevance
When was the list created? How often is it revised? On which basis is it revised? Who is responsible for maintaining the list?
Type
How many of the indicators are of the ‘leading,’ type and how many are of the lagging? Do indicators refer to single or aggregated measurements?
Validity
How is the validity of an indicator established (regardless of whether it is leading or lagging)? Do indicators refer to an articulated process model, or just to ‘common sense’?
Delay
For lagging indicators, how long is the typical lag? Is it acceptable?
Measurement type
What is the nature of the measurements? Qualitative or quantitative? (If quantitative, what kind of scaling is used?)
Measurement frequency
How often are the measurements made? (Continuously, regularly, every now and then?)
Analysis
What is the delay between measurement and analysis/interpretation? How many of the measurements are directly meaningful and how many require analysis of some kind? How are the results communicated and used?
Stability
Are the measured effects transient or permanent?
Organization Support
Is there a regular inspection scheme or -schedule? Is it properly resourced? Where does this measurement fit into the management review?
We perform risk assessments; execute risk mitigations; and we end up with four types of inherent risks (parenthesis is opportunities) in our risk register:
Mitigated (or enhanced)
Avoided (or exploited)
Transferred (or shared)
Accepted
We’ve built a set of risk response plans to ensure we are continuing to treat these risks. And now we need to monitor the effectiveness of our risk plan and to ensure that the risks are behaving in the manner anticipated during risk treatment.
The living risk assessment is designed to conduct reassessment of risks after treatment and continuously throughout the life cycle. However, not all systems and risks need to be reassessed continually, and the organization should prioritize which systems should be reassessed based on a schedule.
Identify indicators that inform the organization about the status of the risk without having to conduct a full risk assessment every time. The trending status of these indicators can act as a flag for investigations, which may result in complete risk assessments.
This risk indicator is then a metric that indicates the state of the level of risk. It is important to note that not all indicators show the exact level of risk exposure, instead providing a trend of drivers, causes or intermediary effects of risk.
The most important risks can be categorized as key risks and the indicators for these key risks are known as key risk indicators (KRIs) which can be defined as: A metric that provides a leading or lagging indicator of the current state of risk exposure on key objectives. KRIs can be used to continually assess current and predict potential risk exposures.
These KRIs need to have a strong relationship with the key performance indicators of the organization.
A good rule of thumb is as you identify the key performance indicators to assess the performance of a specific process, product, system or function you then identify the risks and the KRIs for that objective.
Strive to have leading indicators that measure the elements that influences the risk performance. Lagging indicators will measure they actual performance of the risk controls.
These KRIs qualitatively or quantitatively present the risk exposure by having a strong relationship qirh the risk, its intermediate output or its drivers.
Let’s think in terms of a pharmaceutical supply chain. We’ve done our risk assessments and end up with a top level view like this:
For the risk column we should have some good probabilities and impacts and mitigations in place. We can then chose some KRIs to monitor, such as
Nonconformance rate
Supplier score card
Lab error rate
Product Complaints
As we develop, our KRIs can get more specific and focused. A good KRI is:
Quantifiable
Measurable (accurately and precisely)
Can be validated (have a high level of confidence)
Relevant (measuring the right thing associated with decisions)
In developing a KRI to serve as a leading indicator for potential future occurrences of a risk, it can be helpful to think through the chain of events that led to the event so that management can uncover the ultimate driver (i.e., root cause(s)) of the risk event. When KRIs for root cause events and intermediate events are monitored, we are in an enviable position to identify early mitigation strategies that can begin to reduce or eliminate the impact associated with an emerging risk event.
These KRIs will help us monitor and quantify our risk exposure. They help our organizations compare business objectives and strategy to actual performance to isolate changes, measure the effectiveness of processes or projects, and demonstrate changes in the frequency or impact of a specific risk event.
Effective KRIs can provide value to the organization in a variety of ways. Potential value may be derived from each of the following contributions:
Risk Appetite – KRIs require the determination of appropriate thresholds for action at different levels within the organization. By mapping KRI measures to identified risk appetite and tolerance levels, KRIs can be a useful tool for better articulating the risk appetite that best represents the organizational mindset.
Risk and Opportunity Identification – KRIs can be designed to alert management to trends that may adversely affect the achievement of organizational objectives or may indicate the presence of new opportunities.
Risk Treatment – KRIs can initiate action to mitigate developing risks by serving as triggering mechanisms. KRIs can serve as controls by defining limits to certain actions.
Have you identified and captured your risks in a risk register?
How have you evaluated and prioritized your risks?
Have you engaged the appropriate stakeholders in the risk identification and evaluation processes?
What about risk owners? Does each risk have a risk owner?
Have the risk owners developed risk response plans for the highest risks?
Are you facilitating a review of your risks periodically, resulting in updates to the risk register and effective risk responses?
At the heart of this program sits the Risk Register, which brings together information about risks to inform those exposed to risks and those who have responsibility for their management. A risk register is used to record and track information about individual risks and how they are being controlled. It can be used to communicate information about risks to stakeholders and highlight particularly important risks. While it can be used at any level of the organization where there are a large number of risks, controls and treatments that need to be tracked, a risk register really shines as a central component of a quality management review. The risk register includes:
List of risks, failure modes or hazards and expected outcomes
A statement about the probability of consequences occurring
Sources or causes of the risk
Priority or risk levels
What is currently being done to control the risk
Risk owner
Actual outcome, if and when available
Risks are generally listed individually as separate events but interdependencies should be flagged.
In recording information about risks, the distinction between risks (the potential effects of what might happen) and risk sources (how or why it might happen) and controls that might fail should be explicit. It can also be useful to indicate the early warning signs that an event might be about to occur.
Many risk registers also include some rating of the significance of a risk, an indication of whether a risk is considered to be acceptable or tolerable, or whether further treatment is needed and the reasons for this decision. Where a significance rating is applied to a risk based on consequences and their likelihood, this should take account of the possibility that controls will fail. A level of risk should not be allocated for the failure of a control as if it were an independent risk.
A risk register is used as the basis for tracking implementation of proposed treatments, so it should contain information about treatments and how they will be implemented, or make reference to other documents or data bases with this information. (Such information can include risk owners, actions, action owners, action business case summaries, budgets and timelines, etc.). This living document can usually roll (or even serve as) the Quality Plan.
Strengths of risk registers include the following.
Information about risks is brought together in a form where actions required can be identified and tracked.
Information about different risks is presented in a comparable format, which can be used to indicate priorities and is relatively easy to interrogate.
The construction of a risk register usually involves many people and raises general awareness of the need to manage risk.
By doing this, the risk register serves as a central underpining for the organization as it builds a risk culture, driving transparency and accountability.
Building Risk Based Thinking in the Organization requires a strong governance structure
Pay attention the the following limitations:
Risks captured in risk registers are typically based on events, which can make it difficult to accurately characterize some forms of risk
The apparent ease of use can give misplaced confidence in the information because it can be difficult to describe risks consistently and sources of risk, risks, and weaknesses in controls for risk are often confused.
There are many different ways to describe a risk and any priority allocated will depend on the way the risk is described and the level of disaggregation of the issue.
Considerable effort is required to keep a risk register up to date (for example, all proposed treatments should be listed as current controls once they are implemented, new risks should be continually added and those that no longer exist removed).
Risks are typically captured in risk registers individually. This can make it difficult to consolidate information to develop an overall treatment program.
Artifacts, like the risk register, both demonstrate and channel culture. Invest the time in your organization’s register, and you will reap dividends towards developing a risk friendly culture.
The structured what-if technique, SWIFT, is a high-level and less formal risk identification technique that can be used independently, or as part of a staged approach to make bottom-up methods such as FMEA more efficient. SWIFT uses structured brainstorming in a facilitated workshop where a predetermined set of guidewords (timing, amount, etc.) are combined with prompts elicited from participants that often begin with phrases such as “what if?” or “how could?”.
At the heart of a SWIFT is a list of guidewords to enable a comprehensive review of risks or sources of risk. At the start of the workshop the context, scope and purpose of the SWIFT is discussed and criteria for success articulated. Using the guidewords and “what if?” prompts, the facilitator asks the participants to raise and discuss issues such as:
known risks
risk sources and drivers
previous experience, successes and incidents
known and existing controls
regulatory requirements and constraints
The list of guidewords is utilized by the facilitator to monitor the discussion and to suggest additional issues and scenarios for the team to discuss. The team considers whether controls are adequate and if not considers potential treatments. During this discussion, further “what if?” questions are posed.
Often the list of risks generated can be used to fuel a qualitative or semi-quantitative risk assessment method, such as an FMEA is.
A SWIFT Analysis allows participants to look at the system response to problems rather than just examining the consequences of component failure. As such, it can be used to identify opportunities for improvement of processes and systems and generally can be used to identify actions that lead to and enhance their probabilities of success.
What-If Analysis
What–If Analysis is a structured brainstorming method of determining what things can go wrong and judging the likelihood and consequences of those situations occurring. The answers to these questions form the basis for making judgments regarding the acceptability of those risks and determining a recommended course of action for those risks judged to be unacceptable. An experienced review team can effectively and productively discern major issues concerning a process or system. Lead by an energetic and focused facilitator, each member of the review team participates in assessing what can go wrong based on their past experiences and knowledge of similar situations.
What If?
Answer
Likelihood
Severity
Recommendations
What could go wrong?
What would happen if it did?
How likely?
Consequences
What will we do about them Again – prevent and monitor
What-If Analysis
Steps in a SWIFT Analysis
SWIFT Risk Assessment
Prepare the guide words: The facilitator should select a set of guide words to be used in the SWIFT.
Assemble the team: Select participants for the SWIFT workshop based on their knowledge of the system/process being assessed and the degree to which they represent the full range of stakeholder groups.
Background: Describe the trigger for the SWIFT (e.g., a regulatory change, an adverse event, etc.).
Articulate the purpose: Clearly explain the purpose to be served by the SWIFT (e.g., to improve effectiveness of the process).
Define the requirements: Articulate the criteria for success
Describe the system: Provide appropriate-level textual and graphical descriptions of the system or process to be risk assessed. A clear understanding is necessary and can be is established through interviews, gathering a multifunctional team and through the study of documents, plans and other records. Normally the
Identify the risks/hazards: This is where the structured what-if technique is applied. Use the guide words/headings with each system, high-level subsystem, or process step in turn. Participants should use prompts starting with the phrases like “What if…” or “How could…” to elicit potential risks/hazards associated with the guide word. For instance, if the process is “Receipt of samples,” and the guide word is “time, timing or speed,” prompts might include: “What if the sample is delivered at a shift change” (wrong time) or “How could the sample be left waiting too long in ambient conditions?” (wrong timing).
Assess the risks: With the use of either a generic approach or a supporting risk analysis technique, estimate the risk associated with the identified hazards. In light of existing controls, assess the likelihood that they could lead to harm and the severity of harm they might cause. Evaluate the acceptability of these risk levels, and identify any aspects of the system that may require more detailed risk identification and analysis.
Propose actions: Propose risk control action plans to reduce the identified risks to an acceptable level.
Review the process: Determine whether the SWIFT met its objectives, or whether a more detailed risk assessment is required for some parts of the system.
Document: Produce an overview document to communicate the results of the SWIFT.
Additional risk assessment: Conduct additional risk assessments using more detailed or quantitative techniques, if required. The SWIFT Analysis is really effective as a filtering mechanism to focus effort on the most valuable areas.
Guideword Examples
The facilitator and process owner can choose any guide words that seem appropriate. Guidewords usually stem around:
Wrong: Person or people
Wrong: Place, location, site, or environment
Wrong: Thing or things
Wrong: Idea, information, or understanding
Wrong: Time, timing, or speed
Wrong: Process
Wrong: Amount
Failure: Control or Detection
Failure: Equipment
If your organization has invested time to create root cause categories and sub-categories, the guidewords can easily start there.
Investigations of a Dog 