Evaluating Controls as Part of Risk Management

When I teach an introductory risk management class, I usually use an icebreaker of “What is the riskiest activity you can think of doing. Inevitably you will get some version of skydiving, swimming with sharks, jumping off bridges. This activity is great because it starts all conversations around likelihood and severity. At heart, the question brings out the concept of risk important activities and the nature of controls.

The things people think of, such as skydiving, are great examples of activities that are surrounded by activities that control risk. The very activity is based on accepting reducing risk as low as possible and then proceeding in the safest possible pathway. These risk important activities are the mechanism just before a critical step that:

Ensure the appropriate transfer of information and skill
Ensure the appropriate number of actions to reduce risk
Influence the presence or effectiveness of barriers
Influence the ability to maintain positive control of the moderation of hazards

Risk important activities is a concept important to safety-thought and are at the center of a lot of human error reduction tools and practices. Risk important activities are all about thinking through the right set of controls, building them into the procedure, and successfully executing them before reaching the critical step of no return. Checklists are a great example of this mindset at work, but there are a ton of ways of doing them.

In the hospital they use a great thought process, “Five rights of Safe Medication Practices” that are: 1) right patient, 2) right drug, 3) right dose, 4) right route, and 5) right time. Next time you are getting medication in the doctor’s office or hospital evaluate just what your caregiver is doing and how it fits into that process. Those are examples of risk important activities.

Assessing controls during risk assessment

Risk is affected by the overall effectiveness of any controls that are in place.

The key aspects of controls are:

the mechanism by which the controls are intended to modify risk
whether the controls are in place, are capable of operating as intended, and are achieving the expected results
whether there are shortcomings in the design of controls or the way they are applied
whether there are gaps in controls
whether controls function independently, or if they need to function collectively to be effective
whether there are factors, conditions, vulnerabilities or circumstances that can reduce or eliminate control effectiveness including common cause failures
whether controls themselves introduce additional risks.

A risk can have more than one control and controls can affect more than one risk.

We always want to distinguish between controls that change likelihood, consequences or both, and controls that change how the burden of risk is shared between stakeholders

Any assumptions made during risk analysis about the actual effect and reliability of controls should be validated where possible, with a particular emphasis on individual or combinations of controls that are assumed to have a substantial modifying effect. This should take into account information gained through routine monitoring and review of controls.

Risk Important Activities, Critical Steps and Process

Critical steps are the way we meet our critical-to-quality requirements. The activities that ensure our product/service meets the needs of the organization.

These critical steps are the points of no-return, the point where the work-product is transformed into something else. Risk important activities are what we do to remove the danger of executing that critical step.

Beyond that critical step, you have rejection or rework. When I am cooking there is a lot of prep work which can be a mixture of critical steps, from which there is no return. I break the egg wrong and get eggshells in my batter, there is a degree of rework necessary. This is true for all our processes.

The risk-based approach to the process is to understand the critical steps and mitigate controls.

We are thinking through the following:

Critical Step: The action that triggers irreversibility. Think in terms of critical-to-quality attributes.
Input: What came before in the process
Output: The desired result (positive) or the possible difficulty (negative)
Preconditions: Technical conditions that must exist before the critical step
Resources: What is needed for the critical step to be completed
Local factors: Things that could influence the critical step. When human beings are involved, this is usually what can influence the performer’s thinking and actions before and during the critical step
Defenses: Controls, barriers and safeguards

Risk Management Mindset

Good risk management requires a mindset that includes the following attributes:

Expect to be surprised: Our processes are usually underspecified and there is a lot of hidden knowledge. Risk management serves to interrogate the unknowns
Possess a chronic sense of unease: There is no such thing as perfect processes, procedures, training, design, planning. Past performance is not a guarantee of future success.
Bend, not break: Everything is dynamic, especially risk. Quality comes from adaptability.
Learn: Learn from what goes well, from mistakes, have a learning culture
Embrace humility: No one knows everything, bring those in who know what you do not.
Acknowledge differences between work-as-imagined and work-as-done: Work to reduce the differences.
Value collaboration: Diversity of input
Drive out subjectivity: Understand how opinions are formed and decisions are made.
Systems Thinking: Performance emerges from complex, interconnected and interdependent systems and their components

The Role of Monitoring

One cannot control risk, or even successfully identify it unless a system is able flexibly to monitor both its own performance (what happens inside the system’s boundary) and what happens in the environment (outside the system’s boundary). Monitoring improves the ability to cope with possible risks

When performing the risk assessment, challenge existing monitoring and ensure that the right indicators are in place. But remember, monitoring itself is a low-effectivity control.

Ensure that there are leading indicators, which can be used as valid precursors for changes and events that are about to happen.

For each monitoring control, as yourself the following:

Indicator	How have the indicators been defined? (By analysis, by tradition, by industry consensus, by the regulator, by international standards, etc.)
Relevance	When was the list created? How often is it revised? On which basis is it revised? Who is responsible for maintaining the list?
Type	How many of the indicators are of the ‘leading,’ type and how many are of the lagging? Do indicators refer to single or aggregated measurements?
Validity	How is the validity of an indicator established (regardless of whether it is leading or lagging)? Do indicators refer to an articulated process model, or just to ‘common sense’?
Delay	For lagging indicators, how long is the typical lag? Is it acceptable?
Measurement type	What is the nature of the measurements? Qualitative or quantitative? (If quantitative, what kind of scaling is used?)
Measurement frequency	How often are the measurements made? (Continuously, regularly, every now and then?)
Analysis	What is the delay between measurement and analysis/interpretation? How many of the measurements are directly meaningful and how many require analysis of some kind? How are the results communicated and used?
Stability	Are the measured effects transient or permanent?
Organization Support	Is there a regular inspection scheme or -schedule? Is it properly resourced? Where does this measurement fit into the management review?

Key risk indicators come into play here.

Hierarchy of Controls

Not every control is the same. This principle applies to both current control and planning future controls.

Drive Out Fear on International Workers Day

Happy International Workers Day. Let’s celebrate by Driving Out Fear!

Thirty-five years ago Deming wrote that “no one can put in his best performance unless he feels secure.” Unfortunately, today we still live in a corporate world where fear and management by fear is ubiquitous. That fear is growing after more than a year of a global pandemic. As quality professionals we must deal with it at every opportunity.

Fear undermines quality, productivity, and innovation. The existence of fear leads to a vicious downward spiral.

Some sources of fear include:

Competition: Many managers use competition to instill fear. Competition is about winners and losers. Success cannot exist without failure. Managers deem the anxiety generated by competition between co-workers a good thing as they compete for scarce resources, power and status. Therefore, management encourage competition between individuals, between groups and departments and between business units.
“Us and Them” Culture: The “us and them” culture that predominates in so many organizations proliferated by silos. Includes barriers between staff and supervisors.
Blame Culture: Fear predominates in a blame culture. Blame culture can often center around enshrining the idea of human error.

We drive out fear by building a culture centered on employee well-being. This is based on seven factors.

Factor	Means	Obtained by
Responsibility	Well defined responsibilities and ownership	The opportunity an employee has to provide input into decision making in his department An individual employees’ own readiness to set high personal standards An individual employee’s interest in challenging work assignments The opportunity an employee has to improve skills and capabilities Excellent career advancement opportunities The organization’s encouragement of problem-solving and innovative thinking
Management Competence	Managers trained with skills that lend themselves to contributing to the work of their team ensures that they will be looked to for help. Managers need to be able to guide.	Direct Supervisor/Manager Leadership Abilities Management is engaged and leads by example (Gemba walks) Management by Facts
Consideration	When managers act as if employees have no feelings and just expect them to do their work as if they are robots, it can make employees uneasy. Such behavior makes them feel detached and merely a tool to carry out an end. In such environments, many times the only times employees hear from the manager is when something goes well or really bad. In either case, the perception could be that the manager has mood swings and that also adds to the employee’s insecurity. They may feel reluctant to talk to their manager for fear he is in one of his bad moods.	Senior Management’s sincere interest in employee well-being An individual employee’s relationship with their supervisor Open and effective communication Trust in management and co-workers
Cooperation	The feeling that every person is on their own to look out for their interest is a sad state to be in. Yet when everyone has a fear that the other workers will take advantage of them or make them look bad at the first opportunity, a selfish and insecure environment will result. Employees should be able to work together for the benefit of the company. They should focus on group goals in addition to their personal goals, recognizing that individually there will be failures, but that the whole is more important than the individual parts.	Trust Well trained employees Collaboration as a process Organizational culture (psychological safety) Hire and promote the right behaviors & traits to match the culture
Feedback	Information that is given back to the employee regarding their performance on the job.	Know what is expected of them (clear job descriptions) Effective processes for timely feedback Recognition Know their opinion matters
Information	Transparency is critical. When employees know nothing about how a company is doing in terms of where they should be, it is a source of uneasiness. Without that knowledge, for all they know the company could be doing very poorly and that could be a bad thing for everyone. When they have a better sense of where the company is in the scheme of their objectives set by management, it helps them feel more secure. That is not to say it is the news being good or bad that affects their security, but rather the fact that they actually have the news.	Strategy and Mission — especially the freedom and autonomy to succeed and contribute to an organization’s success Organizational Culture and Core/Shared Values Feel that their job is important
Stability	Employees feel more secure when their role does not change frequently and they understand what tomorrow will mean.	Job Content — the ability to do what I do best Availability of Resources to Perform the Job Effectively Career development – opportunities to learn and grow

Levels of Problems in Culture

When thinking about root cause analysis it is useful to think of whether the problem is stemming from a cultural level or when it may be coming from an operational. We can think of these problems as hazards stemming from three areas:

Culture/philosophy is the over-arching view of how the organization conducts business from top-level decision-makers on through the corporate culture of an organization.
Policies are the broad specifications of the manner in which operations are performed. This includes the end-to-end processes.
Policies lead to the development of process and procedures, which are specifications for a task or series of tasks to accomplish a predetermined goal leading to a high degree of consistency and uniformity in performance.

Hazards unrecognized (risks not known or correctly appraised)	Hazards forseen (risks anticipated but response not adequate)
Culture/Philosophy Quality not source of corporate pride Regulatory standards seen as maxima	Culture/Philosophy Quality seen as source of corporate pride Regulatory standards seen as minima
Policy Internal monitoring schemes inadequate (e.g. employee concerns not communicated upwards) Insufficient resources allocated to quality Managers insufficiently trained or equipped Reliance on other organization’s criteria (e.g. equipment manufacturer)	Policy Known deficiencies (e.g. equipment, maintenance) not addressed Defenses not adequately monitored Defenses compromised by other policies (e.g. adversarial employee relations, incentive systems, performance monitoring)
Procedures No written procedures	Procedures Documentation inadequate Inadequate, or Loop-hole in, controls Procedures conflict with one another or with organizational policy

This approach on problems avoids a focus on the individuals involved and avoids a blame culture, which will optimize learning culture. Blaming the individuals risks creating an unsafe culture and creates difficulties for speaking up which should be an espoused quality value. Focus on deficiencies in the system to truly address the problem.

Doorway Effect

Vivian Bringslimark recently shared an article “It’s not just doorways that make us forget what we came for in the next room“

This definitely brings to mind how movement is one the traditional Lean wastes.

It also gets me thinking of the first principle of human performance – People Make Mistakes

Research like this really drives home the importance of reducing reliance on memory in our processes.

What prevents us from improving systems?

Improvement is a process and sometimes it can feel like it is a one-step-forward-two-steps-back sort of shuffle. And just like any dance, knowing the steps to avoid can be critical. Here are some important ones to consider. In many ways they can be considered an onion, we systematically can address a problem layer and then work our way to the next.

Human-error-as-cause

The vague, ambiguous and poorly defined bucket concept called human error is just a mess. Human error is never the root cause; it is a category, an output that needs to be understood. Why did the human error occur? Was it because the technology was difficult to use or that the procedure was confusing? Those answers are things that are “actionable”—you can address them with a corrective action.

The only action you can take when you say “human error” is to get rid of the people. As an explanation the concept it widely misused and abused.

Human performance instead of human error

Attribute	Person Approach	System Approach
Focus	Errors and violations	Humans are fallible; errors are to be expected
Presumed Cause	Forgetfulness, inattention, carelessness, negligence	“Upstream” failures, error traps; organizational failures that contribute to these
Countermeasure to apply	Fear, more/longer procedures, retraining, disciplinary measures, shaming	Establish system defenses and barriers

Options to avoid human error

Human error has been a focus for a long time, and many companies have been building programmatic approaches to avoiding this pitfall. But we still have others to grapple with.

Causal Chains

We like to build our domino cascades that imply a linear ordering of cause-and-effect – look no further than the ubiquitous presence of the 5-Whys. Causal chains force people to think of complex systems by reducing them when we often need to grapple with systems for their tendency towards non-linearity, temporariness of influence, and emergence.

This is where taking risk into consideration and having robust problem-solving with adaptive techniques is critical. Approach everything like a simple problem and nothing will ever get fixed. Similarly, if every problem is considered to need a full-on approach you are paralyzed. As we mature we need to have the mindset of types of problems and the ability to easily differentiate and move between them.

Root cause(s)

We remove human error, stop overly relying on causal chains – the next layer of the onion is to take a hard look at the concept of a root cause. The idea of a root cause “that, if removed, prevents recurrence” is pretty nonsensical. Novice practitioners of root cause analysis usually go right to the problem when they ask “How do I know I reached the root cause.” To which the oft-used stopping point “that management can control” is quite frankly fairly absurd. The concept encourages the idea of a single root cause, ignoring multiple, jointly necessary, contributory causes let alone causal loops, emergent, synergistic or holistic effects. The idea of a root cause is just an efficiency-thoroughness trade-off, and we are better off understanding that and applying risk thinking to deciding between efficiency and resource constraints.

In conclusion

Our problem solving needs to strive to drive out monolithic explanations, which act as proxies for real understanding, in the form of big ideas wrapped in simple labels. The labels are ill-defined and come in and out of fashion – poor/lack of quality culture, lack of process, human error – that tend to give some reassurance and allow the problem to be passed on and ‘managed’, for instance via training or “transformations”. And yes, maybe there is some irony in that I tend to think of the problems of problem solving in light of these ways of problem solving.