A concept map of Churchman’s general systems approach.
Great overview of Churchman’s approach. Highly recommended reading.
A concept map of Churchman’s general systems approach.
Great overview of Churchman’s approach. Highly recommended reading.
Risk assessments have a core of the subjective to them, frequently including assumptions about the nature of the hazard, possible exposure pathways, and judgments for the likelihood that alternative risk scenarios might occur. Gaps in the data and information about hazards, uncertainty about the most likely projection of risk, and incomplete understanding of possible scenarios contribute to uncertainties in risk assessment and risk management. You can go even further and say that risk is socially constructed, and that risk is at once both objectively verifiable and what we perceive or feel it to be. Then again, the same can be said of most of science.
Risk is a future chance of loss given exposure to a hazard. Risk estimates, or qualitative ratings of risk, are necessarily projections of future consequences. Thus, the true probability of the risk event and its consequences cannot be known in advance. This creates a need for subjective judgments to fill-in information about an uncertain future. In this way risk management is rightly seen as a form of decision analysis, a form of making decisions against uncertainty.
Everyone has a mental picture of risk, but the formal mathematics of risk analysis are inaccessible to most, relying on probability theory with two major schools of thought: the frequency school and the subjective probability school. The frequency school says probability is based on a count of the number of successes divided by total number of trials. Uncertainty that is ready characterized using frequentist probability methods is “aleatory” – due to randomness (or random sampling in practice). Frequentist methods give an estimate of “measured” uncertainty; however, it is arguably trapped in the past because it does not lend itself to easily to predicting future successes.
In risk management we tend to measure uncertainty with a combination of frequentist and subjectivist probability distributions. For example, a manufacturing process risk assessment might begin with classical statistical control data and analyses. But projecting the risks from a process change might call for expert judgments of e.g. possible failure modes and the probability that failures might occur during a defined period. The risk assessor(s) bring prior expert knowledge and, if we are lucky, some prior data, and start to focus the target of the risk decision using subjective judgments of probabilities.
Some have argued that a failure to formally control subjectivity — in relation to probability judgments – is the failure of risk management. This was an argument that some made during WCQI, for example. Subjectivity cannot be eliminated nor is it an inherent limitation. Rather, the “problem with subjectivity” more precisely concerns two elements:
Risk is about the chance of adverse outcomes of events that are yet to occur, subjective judgments of one form or another will always be required in both risk assessment and risk management decision-making.
We control subjectivity in risk management by:
Each one of these is it’s own, future, post.
Risk Management is all about eliminating surprise. So to truly start to understand our risks, we need to understand uncertainty, we need to understand the unknowns. Borrowing from Andreas Schamanek’s Taxonomies of the unknown, let’s explore a few of the various taxonomies of what is not known.
I’m pretty sure Ann Kerwin first gave us the “known unknowns” and the “unknown knowns” that people still find a source of amusement about former defense secretary Rumsfield.
|Known||Known knowns||Known unknowns (conscious ignorance)|
|Unknown||Unknown knowns (tacit knowledge)||Unknown unknowns (meta-ignorance)|
Risk management is then a way of teasing out the unknowns and allowing us to take action:
Smithson distinguishes between passive and active ignorance. Passive ignorance involves areas that we are ignorant of, whereas active ignorance refers to areas we ignore. He uses the term ‘error’ for the unknowns encompassed by passive ignorance and ‘irrelevance’ for active ignorance.
Taboo is fascinating because it gets to the heart of our cultural blindness, those parts of our organization that are closed to scrutiny.
Smithson can help us understand why risk assessments are both a qualitative and a quantitative endeavor. While dealing with the unknown is the bread and butter of statistics, only a small part of the terrain of uncertainty is covered. Under Smithson’s typology, statistics primarily operates in the area of incompleteness, across probability and some kinds of vagueness. In terms of its considerations of sampling bias, statistics also has some overlap with inaccuracy. But, as the typology shows, there is much more to unknowns than the areas statistics deals with. This is another reason that subject matter experts, and different ways of thinking is a must.
Ensuring wide and appropriate expert participation gives additional perspectives on unknowns. There is also synergies by finding unrecognized similarities between disciplines and stakeholders in the unknowns they deal with and there may be great benefit from combining forces. It is important to use these concerns to enrich thinking about unknowns, rather than ruling them out as irrelevant.
Risk management is all about managing surprise. It helps to break surprise down to three types: risk, uncertainty and ignorance.
Effective use of the methodology moves ideally from ignorance to eventually risk.
|Description||Methods of Mitigation|
|Closed Ignorance|| |
Information is available but SMEs are unwilling or unable to consider that some outcomes are unknown to them.
Self-audit process, regular third-party audits, and open and transparent system with global participation
|Open Ignorance|| |
Information is available and SMEs are willing to recognize and consider that some outcomes are unknown.
Surprise occurs because an individual SME lacks knowledge or awareness of the available information.
effective teams xxplore multiple perspectives by including a diverse set of individuals and data sources for data gathering and analysis.
Transparency in process.
Surprise occurs because a group of SMEs has only similar viewpoints represented or may be less willing to consider views outside the community.
|Diversity of viewpoints and sue of tools to overcome group-think and “tribal” knowledge|
Surprise occurs because the SMEs are unable to anticipate and prepare for external shocks or internal changes in preferences, technologies, and institutions.
Simulating impacts and gaming alternative outcomes of various potentials under different conditions
(Blue Team/Read Team exercises)
Surprise occurs when inadequate forecasting tools are used to analyze the available data, resulting in inter-relationships, hidden dependencies, feedback loops, and other negative factors that lead to inadequate or incomplete understanding of the data.
Track changes and interrelationships of various systems to discover potential macro-effect force changes
Risk Management is all about understanding surprise and working to reduce uncertainty and ignorance in order to reduce, eliminate and sometimes accept. As a methodology it is effective at avoiding surrender and denial. With innovation we can even contemplate exploitation. As organizations mature, it is important to understand these concepts and utilize them.
Cultivating expertise, in short learning, is critical to building a quality culture. Yet, the urgency of work easily trumps learning. It can be difficult to carve out time for learning in the inexorable flow of daily tasks. We are all experienced with the way learning ends up being in the lowest box on the 2×2 Eisenhower matrix, or however you like to prioritize your tasks.
For learning to really happen, it must fit around and align itself to our working days. We need to build our systems so that learning is an inevitable result of doing work. There are also things we as individuals can practice to make learning happen.
Practice mindfulness. As you go about your daily job be present and aware, using it as an opportunity to ability to learn and develop. Don’t just sit in on that audit; notice and learn the auditor’s tactics and techniques as you engage with her. Ask product managers about product features; ask experts about industry trends; ask peers for feedback on your presentation skills. These kinds of inquiries are learning experiences and most peers love to tell you what they know.
Keep a to-learn list. Keep a list of concepts, thoughts, practices, and vocabulary you want to explore and then later later explore them when you have a few moments to reflect. Try to work a few off the list, maybe during your commute or at other times when you have space to reflect.
Build learning into your calendar. Many of us schedule email time, time for project updates, time to do administrative work. Make sure you dedicate time for learning.
Share meaningfully. Share with others, but just don’t spread links. Discuss why you are sharing it, what you learned and why you think it is important. This blog is a good example of that.
Make sure our learning and knowledge management systems are built into everything we do. Make them easy to use. Ensure content is shared internally and leads to continuous improvement.
Ensure learning is valued.
Plan for short-term wins. There is no nirvana, no perfect state. Ensure you have lots of little victories and shareable moments. Plan for this as part of your schedules and cycles.
Learning is a very effective lever for system improvement. At the very least it gives us the power to “add, change, evolve or self-organize system structure” (lever 4) and can also start giving us ways to change the paradigm (lever 2) and eventually even transcend paradigms (lever 1).
One of the hallmarks of a quality culture is learning from our past experiences, to eliminate repeat mistakes and to reproduce success. The more times you do an activity, the more you learn, and the better you get (within limits for simple activities). Knowledge management is an enabler of quality systems, in part, to focus on learning and thus accelerate learning across the organization as a whole, and not just one person or a team.
This is where the” lessons learned” process comes in. There are a lot of definitions of lessons learned out there, but the definition I keep returning to is that a lessons learned is a change in personal or organizational behavior as a result from learning from experience. Ideally, this is a permanent, institutionalized change, and this is often where our quality systems can really drive continuous improvement.
The lessons learned process is an application of knowledge management.
Lessons identified is generate, assess, and share.
Updated processes (and documents) is contextualize, apply and update.
Identifying lessons needs to be done regularly, the closer to actual change management and control activities the better. The formality of this exercise depends on the scale of the change. There are basically a few major forms:
The chosen formality should be based on the level of change. A healthy organization will be utilizing all of these.
|Level of Change||Form of Lesson Learned|
After action (when things go wrong)
After action (weekly, daily as needed)
After action (daily)
Successful lessons learned:
In practice there are a lot of similarities between the techniques to facilitate a good lessons learned and a root cause analysis. Start with a good core of questions, starting with the what:
From these what questions, we can continue to narrow in on the learnings by asking why and how questions. Ask open questions, and utilize all the techniques of root cause analysis here.
Then once you are at (or close) to a defined issue for the learning (a root cause), ask a future-tense question to make it actionable, such as:
Press for specifics. if it is not actionable it is not really a learning.
Learning implies memory, and an organization’s memories usually require procedures, job aids and other tools to be updated and created. In short, lessons should evolve your process. This is often the responsibility of the change management process owner. You need to make sure the lesson actually takes hold.
There are three things to answer in every change
Effectiveness reviews are 1 and 2 (based on a risk based approach) while lessons learned is 3. Lessons learned contributes to the health of the system and drives continuous improvements in the how we make changes.