Managing Events Systematically

Being good at problem-solving is critical to success in an organization. I’ve written quite a bit on problem-solving, but here I want to tackle the amount of effort we should apply.

Not all problems should be treated the same. There are also levels of problems. And these two aspects can contribute to some poor problem-solving practices.

It helps to look at problems systematically across our organization. The iceberg analogy is a pretty popular way to break this done focusing on Events, Patterns, Underlying Structure, and Mental Model.

Iceberg analogy

Events

Events start with the observation or discovery of a situation that is different in some way. What is being observed is a symptom and we want to quickly identify the problem and then determine the effort needed to address it.

This is where Art Smalley’s Four Types of Problems comes in handy to help us take a risk-based approach to determining our level of effort.

Type 1 problems, Troubleshooting, allows us to set problems with a clear understanding of the issue and a clear pathway. Have a flat tire? Fix it. Have a document error, fix it using good documentation practices.

It is valuable to work the way through common troubleshooting and ensure the appropriate linkages between the different processes, to ensure a system-wide approach to problem solving.

Corrective maintenance is a great example of troubleshooting as it involved restoring the original state of an asset. It includes documentation, a return to service and analysis of data. From that analysis of data problems are identified which require going deeper into problem-solving. It should have appropriate tie-ins to evaluate when the impact of an asset breaking leads to other problems (for example, impact to product) which can also require additional problem-solving.

It can be helpful for the organization to build decision trees that can help folks decide if a given problem stays as troubleshooting or if it it also requires going to type 2, “gap from standard.”

Type 2 problems, gap from standard, means that the actual result does not meet the expected and there is a potential of not meeting the core requirements (objectives) of the process, product, or service. This is the place we start deeper problem-solving, including root cause analysis.

Please note that often troubleshooting is done in a type 2 problem. We often call that a correction. If the bioreactor cannot maintain temperature during a run, that is a type 2 problem but I am certainly going to immediately apply troubleshooting as well. This is called a correction.

Take documentation errors. There is a practice in place, part of good documentation practices, for addressing troubleshooting around documents (how to correct, how to record a comment, etc). By working through the various ways documentation can go wrong, applying which ones are solved through troubleshooting and don’t involve type 2 problems, we can create a lot of noise in our system.

Core to the quality system is trending, looking for possible signals that require additional effort. Trending can help determine where problems lay and can also drive up the level of effort necessary.

Underlying Structure

Root Cause Analysis is about finding the underlying structure of the problem that defines the work applied to a type 2 problem.

Not all problems require the same amount of effort, and type 2 problems really have a scale based on consequences, that can help drive the level of effort. This should be based on the impact to the organization’s ability to meet the quality objectives, the requirements behind the product or service.

For example, in the pharma world there are three major criteria:

  •  safety, rights, or well-being of patients (including subjects and participants human and non-human)
  • data integrity (includes confidence in the results, outcome, or decision dependent on the data)
  • ability to meet regulatory requirements (which stem from but can be a lot broader than the first two)

These three criteria can be sliced and diced a lot of ways, but serve our example well.

To these three criteria we add a scale of possible harm to derive our criticality, an example can look like this:

ClassificationDescription
CriticalThe event has resulted in, or is clearly likely to result in, any one of the following outcomes:   significant harm to the safety, rights, or well-being of subjects or participants (human or non-human), or patients; compromised data integrity to the extent that confidence in the results, outcome, or decision dependent on the data is significantly impacted; or regulatory action against the company.
MajorThe event(s), were they to persist over time or become more serious, could potentially, though not imminently, result in any one of the following outcomes:  
harm to the safety, rights, or well-being of subjects or participants (human or non-human), or patients; compromised data integrity to the extent that confidence in the results, outcome, or decision dependent on the data is significantly impacted.
MinorAn isolated or recurring triggering event that does not otherwise meet the definitions of Critical or Major quality impacts.
Example of Classification of Events in a Pharmaceutical Quality System

This level of classification will drive the level of effort on the investigation, as well as drive if the CAPA addresses underlying structures alone or drives to addressing the mental models and thus driving culture change.

Mental Model

Here is where we address building a quality culture. In CAPA lingo this is usually more a preventive action than a corrective action. In the simplest of terms, corrective actions is address the underlying structures of the problem in the process/asset where the event happened. Preventive actions deal with underlying structures in other (usually related) process/assets or get to the Mindsets that allowed the underlying structures to exist in the first place.

Solving Problems Systematically

By applying this system perspective to our problem solving, by realizing that not everything needs a complete rebuild of the foundation, by looking holistically across our systems, we can ensure that we are driving a level of effort to truly build the house of quality.

Risk Management is our Ability to Anticipate

Risk assessment is a pillar of the quality system because it gives us the ability to anticipate in a consistent manner. It is built on some fundamental criteria:

CriteriaAsksEnsure
ExpertiseWhat sort of expertise is relied upon to look into the futureDiversity in expertise. Drive out subjectivity
FrequencyHow often are future threats and opportunities assessed?Living risk assessments, cycles of review.
CommunicationHow are the expectations of future events communicated or shared within the system?Risk register. Knowledge management. Continuous improvement. Management review
StrategyWhat is the model of the futureSensemaking
Time horizonHow far ahead does the system look ahead? Is the time horizon different for different organization areas?System building
Acceptability of RisksWhich risks are considered acceptable and which unacceptable? On which basis?Controls
CultureIs risk awareness part of the organizational culture?Risk-based-thinking. Mindset
Anticipation Criteria to apply to Risk Management

Documents and the Heart of the Quality System

A month back on LinkedIn I complained about a professional society pushing the idea of a document-free quality management system. This has got to be one of my favorite pet peeves that come from Industry 4.0 proponents, and it demonstrates a fundamental failure to understand core concepts. And frankly one of the reasons why many Industry/Quality/Pharma 4.0 initiatives truly fail to deliver. Unfortunately, I didn’t follow through with my idea of proposing a session to that conference, so instead here are my thoughts.

Fundamentally, documents are the lifeblood of an organization. But paper is not. This is where folks get confused. But fundamentally, this confusion is also limiting us.

Let’s go back to basics, which I covered in my 2018 post on document management.

When talking about documents, we really should talk about function and not just by name or type. This allows us to think more broadly about our documents and how they function as the lifeblood.

There are three types of documents:

  • Functional Documents provide instructions so people can perform tasks and make decisions safely effectively, compliantly, and consistently. This usually includes things like procedures, process instructions, protocols, methods, and specifications. Many of these need some sort of training decision. Functional documents should involve a process to ensure they are up-to-date, especially in relation to current practices and relevant standards (periodic review)
  • Records provide evidence that actions were taken, and decisions were made in keeping with procedures. This includes batch manufacturing records, logbooks and laboratory data sheets and notebooks. Records are a popular target for electronic alternatives.
  • Reports provide specific information on a particular topic on a formal, standardized way. Reports may include data summaries, findings, and actions to be taken.

The beating heart of our quality system brings us from functional to record to reports in a cycle of continuous improvement.

Functional documents are how we realize requirements, that is the needs and expectations of our organization. There are multiple ways to serve up the functional documents, the big three being paper, paper-on-glass, and some sort of execution system. That last, an execution system, united function with record, which is a big chunk of the promise of an execution system.

The maturation mind is to go from mostly paper execution, to paper-on-glass, to end-to-end integration and execution to drive up reliability and drive out error. But at the heart, we still have functional documents, records, and reports. Paper goes, but the document is there.

So how is this failing us?

Any process is a way to realize a set of requirements. Those requirements come from external (regulations, standards, etc) and internal (efficiency, business needs) sources. We then meet those requirements through People, Procedure, Principles, and Technology. They are interlinked and strive to deliver efficiency, effectiveness, and excellence.

So this failure to understand documents means we think we can solve this through a single technology application. an eQMS will solve problems in quality events, a LIMS for the lab, an MES for manufacturing. Each of these is a lever for change but alone cannot drive the results we want.

Because of the limitations of this thought process we get systems designed for yesterday’s problems, instead of thinking through towards tomorrow.

We get documentation systems that think of functional documents pretty much the same way we thought of them 30 years ago, as discrete things. These discrete things then interact through a gap with our electronic systems. There is little traceability, which complicates change control and makes it difficult to train experts. The funny thing, is we have the pieces, but because of the limitations of our technology we aren’t leveraging them.

The v-model approach should be leveraged in a risk-based manner to the design of our full system, and not just our technical aspects.

System feasibility matches policy and governance, user requirements allow us to trace to what elements are people, procedure, principles, and/or technology. Everything then stems from there.

Quality-as-Imagined versus Quality-as-Done

Assumptions about how work is carried out is often very different from the reality of the work. This is the difference between work-as-imagined and work-as-done. Assumptions about work as imagined often turn out to be wrong because they are based on a fundamental misunderstanding. Steven Shorrock on Humanistic Systems has been doing a great series on proxies for work-as-done that I recommend you read for more details.

The complexity of our organizations implies a certain level of inevitable unexpected variability and thus a gap between Work-as-Imagined and Work-as-Done. Work-as-Imagined reflects how work is understood by those who are separated from it by time or space; it is an over-simplified version of what is actually going on. Work-as-Done takes account of what it means to function effectively, despite resource-constrained circumstances. The analysis of the gap between Work-As-Imagined and Work-as-Done usually indicates that performance variability is present in both desired and undesired outcomes and, therefore, successful outcomes do not necessarily occur because people are behaving according to Work-as-Imagined.

The same concept applies to the nature and implications of the gap between the prescribed quality practices and policies, Quality-as-Imagined, and the way they are deployed in practice, Quality-as-Done.

This gap should be no surprise. Our organizations are complex systems, and complexity can give rise to unintended consequences.

The interesting thing is that quality can drive a reduction of that gap, solving for complexity.

The Influence of Complexity on Quality
Dynamic InteractionsWide DiversityUnexpected VariabilityResilience
SocialInteractions between employeesEmployees with varying skill levels
Employee turnover
Diversity of functions performed by employees
(e.g. multiskilling)
Errors when operating equipment and tools
Unexpected behaviors
Absenteeism
Variability in human labor demand
Unexpected outcomes from
social interactions (e.g. conflicts and alliances)
Employees’ ability to
anticipate risks
Critical analysis of data
Informal agreements between workers to distribute the workload
TechnicalInteractions between production resources
Interactions due to tightly coupled operations (e.g. time constraints, low inventories, capacity constraints)
Product diversity
Diversity of quality requirements
Diversity of client requirements
Technical disruptions
Resource availability (e.g. maintenance staff)
Variability in production times (e.g. cycle time, lead time)
Dimensional variability (e.g. potential for defects)
Inspection readiness
Corrective, preventive and predictive measures
Work OrganizationInteractions between information sources
Interactions between functions
Interactions between processes
Interactions between performance indicators
Diversity in managerial controls
Diversity in relationships with external agents
Diversity of rules and procedures
Variability in the hiring of new workers
Changing priorities (e.g. frequent rescheduling due to unexpected conditions)
Variability in timing and
accuracy of information
Negotiation, partnership and bargaining power with suppliers and clients
Investments on new resources
Multidisciplinary problem-solving meetings
External EnvironmentInteractions between the organization, suppliers, and clients
Interactions with regulatory bodies
Diversity in suppliers
Diversity in clients
Variability in Demand/Need
Variability in logistics
Capacity and slack management
Examples of Complexity Impact

Seven elements of good problem-solving

Logic

Perhaps more than anything else, we want our people to be able to think and then act rationally in decision making and problem-solving. The basic structure and technique embodied in problem solving is a combination of discipline when executing PDCA mixed with a heavy dose of the scientific method of investigation.

Logical thinking is tremendously powerful because it creates consistent, socially constructed approaches to problems, so that members within the organization spend less time spinning their wheels or trying to figure out how another person is approaching a given situation. This is an important dynamic necessary for quality culture.

The right processes and tools reinforce this as the underlying thinking pattern, helping to promote and reinforce logical thought processes that are thorough and address all important details, consider numerous potential avenues, take into account the effects of implementation, anticipate possible stumbling blocks, and incorporate contingencies. The processes apply to issues of goal setting, policymaking, and daily decision making just as much as they do to problem-solving.

Objectivity

Because human observation is inherently subjective, every person sees the world a little bit differently. The mental representations of the reality people experience can be quite different, and each tends to believe their representation is the “right” one. Individuals within an organization usually have enough common understanding that they can communicate and work together to get things done. But quite often, when they get into the details of the situation, the common understanding starts to break down, and the differences in how we see reality become apparent.

Problem-solving involves reconciling those multiple viewpoints – a view of the situation that includes multiple perspectives tends to be more objective than any single viewpoint. We start with one picture of the situation and make it explicit so that we can better share it with others and test it. Collecting quantitative (that is, objective) facts and discussing this picture with others is a key way in verifying that the picture is accurate. If it is not, appropriate adjustments are made until it is an accurate representation of a co-constructed reality. In other words, it is a co-constructed representation of a co-constructed reality.

Objectivity is a central component to the problem solving mindset. Effective problem-solvers continually test their understanding of a situation for assumptions, biases, and misconceptions. The process begins by framing the problem with relevant facts and details, as objectively as possible. Furthermore, suggested remedies or recommended courses of action should promote the organizational good, not (even if subconsciously) personal agendas.

Results and Process

Results are not favored over the process used to achieve them, nor is process elevated above results. Both are necessary and critical to an effective organization.

Synthesis, Distillation and and Visualization

We want to drive synthesis of the learning acquired in the course of understanding a problem or opportunity and discussing it with others. Through this multiple pieces of information from different sources are integrated into a coherent picture of the situation and recommended future action.

Visual thinking plays a vital role in conveying information and the act of creating the visualization aids the synthesis and distillation process.

Alignment

Effective implementation of a change often hinges on obtaining prior consensus among the parties involved. With consensus, everyone pulls together to overcome obstacles and make the change happen. Problem-solving teams communicates horizontally with other groups in the organization possibly affected by the proposed change and incorporates their concerns into the solution. The team also communicates vertically with individuals who are on the front lines to see how they may be affected, and with managers up the hierarchy to determine whether any broader issues have not been addressed. Finally, it is important that the history of the situation be taken into account, including past remedies, and that recommendations for action consider possible exigencies that may occur in the future. Taking all these into consideration will result in mutually agreeable, innovative solutions.

Coherency and Consistency

Problem-solving efforts are sometimes ineffective simply because the problem-solvers do not maintain coherency. They tackle problems that are not important to the organization’s goals, propose solutions that do not address the root causes, or even outline implementation plans that leave out key pieces of the proposed solution. So coherency within the problem-solving approach is paramount to effective problem resolution.

Consistent approaches to problem-solving speed up communication and aid in establishing shared understanding. Organizational members understand the implicit logic of the approach, so they can anticipate and offer information that will be helpful to the problem-solvers as they move through the process.

Systems Thinking

Good system thinking means good problem-solving.