Risk Assessments Do Not Replace Technical Knowledge

The US Food and Drug Administration (FDA) last month warned Indian generic drugmaker Lupin Limited over three good manufacturing practice (GMP) violations at its facility in Maharashtra, India that identified issues with the company’s written procedures for equipment cleaning, its written procedures for monitoring and controlling the performance of processing steps and the “failure to investigate all critical deviations.”

The FDA said the company “performed multiple risk assessments with the purpose to verify whether existing cleaning procedures and practices eliminate or reduce genotoxic impurities … generated through the manufacture of [redacted] drugs after you detected [redacted] impurities in your [active pharmaceutical ingredient] API.” The company also performed risk assessments to determine whether its cleaning procedures reduced the risk of cross-contamination of intermediates and API. However, FDA said the risk assessments “lacked data to support that existing equipment cleaning procedures are effective in removing [redacted] along with residual API from each respective piece of equipment to acceptable levels. “The identification of genotoxic impurities in quantities near their established limits suggests excursions are possible. All intermediates and API manufactured on non-dedicated equipment used to manufacture [redacted] drugs should be subject to validated sampling and analytical testing to ensure they are not contaminated with unacceptable levels of genotoxic impurities,” FDA said.

At heart this warning letter shows a major weakness in many company’s risk management approach, they use the risk assessment to replace technical inquiry, instead of as a tool to determine the appropriateness of technical understanding and as a way to manage the uncertainty around technical knowledge.

A significant point in the current Q9 draft is to deal with this issue, which we see happen again and again. Risk management cannot tell you whether your cleaning procedures are effective or not. Only a validated testing scheme can. Risk management looks at the aggregate and evaluates possibilities.

Global versus Local Process and Procedure and the eQMS

Companies both large and small grapple with how and when to create standard work at the global level, while still having the scalability to capture different GXP activity families and product modality.

I’ve discussed before on document hierarchy and on the leveling of process and procedure. It is really important to level your processes, and this architecture should be deliberate and shepherded.

This really gets to the heart of work-as-imagined and prescribed, and the concept of standard work.

Benefits of Standard Work

  • Ensures all work is done according to the current best practice
  • Consistency is the essential ingredient of quality
  • Allows organizations to scale rapidly
  • Puts the focus on the process and not an individual or team
  • Makes improvements easier and faster

Global versus Local Process and Procedure in the Document Hierarchy

Most Quality Hierarchies look fairly similar.

A Document Hierarchy

Excluding the Program level (which becomes even more important) we can expand the model in the process band to account for global versus local.

Global and local process within the document hierarchy

Quality Manual and Policy remains global with local input and determine the overall structure of the quality management system.

Global Process is created when a process is majority task and role driven at a global level. It is pan-GXP, pan-modality, pan-geography. It is the standard way of work to drive consistency across and through the organization.

Local Process is created when a process is specific to a specific GXP, product modality, geography.

Procedure, which describes the tasks, can be created off of local or global process. When the global process has localizations (a CAPA is a CAPA but how I build action items may differ across sites), I can build local versions off the global process.

For an example, Document and Record Management.

This approach takes real vision among leaders to drive for consistency and simplicity. This activity is a core component in good system design, no matter the size of the organization.

PrincipleDescriptionApplication for Global and Local Process
BalanceThe system creates value for the multiple stakeholders. While the ideal is to develop a design that maximizes the value for all the key stakeholders, the designer often has to compromise and balance the needs of the various stakeholders.The value of standard work really shines here.
CongruenceThe degree to which the system components are aligned and consistent with each other and the other organizational systems, culture, plans, processes, information, resource decisions, and actions.We gain congruence through ensuring key processes are at the global level.
ConvenienceThe system is designed to be as convenient as possible for the participants to implement (a.k.a. user friendly). System includes specific processes, procedures, and controls only when necessary.The discussion around global versus local will often depend on how you define convenience
CoordinationSystem components are interconnected and harmonized with the other (internal and external) components, systems, plans, processes, information, and resource decisions toward common action or effort. This is beyond congruence and is achieved when the individual components of a system operate as a fully interconnected unit.How we ensure coordination across and through an organization.
EleganceComplexity vs. benefit — the system includes only enough complexity as is necessary to meet the stakeholder’s needs. In other words, keep the design as simple as possible and no more while delivering the desired benefits. It often requires looking at the system in new ways.Keep this in mind as global for the sake of global is not always the right decision.
HumanParticipants in the system are able to find joy, purpose and meaning in their work.Never forget
LearningKnowledge management, with opportunities for reflection and learning (learning loops), is designed into the system. Reflection and learning are built into the system at key points to encourage single- and double-loop learning from experience to improve future implementation and to systematically evaluate the design of the system itself.Building the right knowledge management into the organization is critical to leverage this model
SustainabilityThe system effectively meets the near- and long-term needs of the current stakeholders without compromising the ability of future generations of stakeholders to meet their own needs.Ensure the appropriate tools exist to sustain, including regulatory intelligence. Long-term scalability.
Pillars of Good System Design for Gloval and Local Process

Utilizing the eQMS to drive

The ideal state when implementing (or improving) an eQMS is to establish global processes and allow system functionality to localize as appropriate.

Leveraging the eQMS

So for example, every CAPA is the same (identify problem and root cause, create plan, implement plan, prove implementation is effective. This is a global process. However, one wants specific task detail at a lower level, for example GMP sites may care about certain fields more the GCP, medical device has specific needs, etc. These local task level needs can be mainted within one workflow.

The Key is Fit-For-Purpose Fit-for-Use

A fit for purpose process meets the requirements of the organization.

A fit for use process is usable throughout the lifecycle.

Global and localizing processes is a key part of making both happen.

Thinking of Swiss Cheese: Reason’s Theory of Active and Latent Failures

The Theory of Active and Latent Failures was proposed by James Reason in his book, Human Error. Reason stated accidents within most complex systems, such as health care, are caused by a breakdown or absence of safety barriers across four levels within a system. These levels can best be described as Unsafe Acts, Preconditions for Unsafe Acts, Supervisory Factors, and Organizational Influences. Reason used the term “active failures” to describe factors at the Unsafe Acts level, whereas “latent failures” was used to describe unsafe conditions higher up in the system.

This is represented as the Swiss Cheese model, and has become very popular in root cause analysis and risk management circles and widely applied beyond the safety world.

Swiss Cheese Model

In the Swiss Cheese model, the holes in the cheese depict the failure or absence of barriers within a system. Such occurrences represent failures that threaten the overall integrity of the system. If such failures never occurred within a system (i.e., if the system were perfect), then there would not be any holes in the cheese. We would have a nice Engelberg cheddar.

Not every hole that exists in a system will lead to an error. Sometimes holes may be inconsequential. Other times, holes in the cheese may be detected and corrected before something bad happens. This process of detecting and correcting errors occurs all the time.

The holes in the cheese are dynamic, not static. They open and close over time due to many factors, allowing the system to function appropriately without catastrophe. This is what human factors engineers call “resilience.” A resilient system is one that can adapt and adjust to changes or disturbances.

Holes in the cheese open and close at different rates. The rate at which holes pop up or disappear is determined by the type of failure the hole represents.

  1. Holes that occur at the Unsafe Acts level, and even some at the Preconditions level, represent active failures. Active failures usually occur during the activity of work and are directly linked to the bad outcome. Active failures change during the process of performing, opening, and closing over time as people make errors, catch their errors, and correct them.
  2. Latent failures occur higher up in the system, above the Unsafe Acts level — the Organizational, Supervisory, and Preconditions levels. These failures are referred to as “latent” because when they occur or open, they often go undetected. They can lie “dormant” or “latent” in the system for an extended period of time before they are recognized. Unlike active failures, latent failures do not close or disappear quickly.

Most events (harms) are associated with multiple active and latent failures. Unlike the typical Swiss Cheese diagram above, which shows an arrow flying through one hole at each level of the system, there can be a variety of failures at each level that interact to produce an event. In other words, there can be several failures at the Organizational, Supervisory, Preconditions, and Unsafe Acts levels that all lead to harm. The number of holes in the cheese associated with events are more frequent at the Unsafe Acts and Preconditions levels, but (usually) become fewer as one progresses upward through the Supervisory and Organizational levels.

Given the frequency and dynamic nature of activities, there are more opportunities for holes to open up at the Unsafe and Preconditions levels on a frequent basis and there are often more holes identified at these levels during root cause investigation and risk assessments.

The way the holes in the cheese interact across levels is important:

  • One-to-many mapping of causal factors is when a hole at a higher level (e.g., Preconditions) may result in several holes at a lower level (e.g. Unsafe acts)
  • Many-to-one mapping of causal factors when multiple holes at the higher level (e.g. preconditions) might interact to produce a single hole at the lower level (e.g. Unsafe Acts)

By understand the Swiss Cheese model, and Reason’s wider work in Active and Latent Failures, we can strengthen our approach to problem-solving.

Plus cheese is cool.

Swiss Cheese on a cheese board with knife

Design Problem Solving into the Process

Good processes and systems have ways designed into them to identify when a problem occurs, and ensure it gets the right rigor of problem-solving. A model like Art Smalley’s can be helpful here.

Each and every process should go through the following steps:

  1. Define those problems that should be escalated and those that should not. Everyone working in a process should have the same definition of what is a problem. Often times we end up with a hierarchy of issues that are solved within the process – Level 1 – and those processes that go to a root cause process (deviation/CAPA) – level 2.
  2. Identify the ways to notice a problem. Make the work as visual as possible so it is easier to detect the problem.
  3. Define the escalation method. There should be one clear way to surface a problem. There are many ways to create a signal, but it should be simple, timely, and very clear.

These three elements make up the request for help.

The next two steps make up the response to that request.

  1. Who is the right person to respond? Supervisor? Area management? Process Owner? Quality?
  2. How does the individual respond, and most importantly when? This should be standardized so the other end of that help chain is not wondering whether, when, and in what form that help is going to arrive.

In order for this to work, it is important to identify clear ownership of the problem. There always must be one person clearly accountable, even if only responsible for bits, so they can push the problem forward.

It is easy for problem-solving to stall. So make sure progress is transparent. Knowing what is being worked on, and what is not, is critical.

Prioritization is key. Not every problem needs solving so have a mechanism to ensure the right problems are being solved in the process.

Problem solving within a process

The Risk Question

The risk question established the purpose and scope – the context of the risk assessment. This step is critical since it sets the risk assessment’s direction, tone, and expectations.  From this risk question stems the risk team; the degree, extent, or rigor of the assessment; the risk assessment methodologies; the risk criteria; and levels of acceptable risk.

The risk problem needs to be clear, concise, and well understood by all stakeholders. Every successful risk assessment needs a tightly defined beginning and end, so the assessment team can set good boundaries for the assessment with internal (resources, knowledge, culture, values, etc) and external (technology, legal, regulatory, economy, perceptions of external stakeholders, etc) parameters in mind.

To ensure the risk team focuses on the correct elements, the risk question should clearly explain what is expected. For example:

  • For a risk assessment of potential emergencies/disasters, should the assessment be limited to emergencies/disasters at facility sites or include events off-site? Should it include natural, manmade, or technological emergencies/disasters, or all of them?
  • If the hazards associated with the job of repairing a porch as to be assessed, would it just cover the actual porch repair, or would it include hazards like setting up the space, bringing materials on site, and the hazards associated with use/not-use of the porch?
  • If the risk assessment covers getting a new family dog does it include just those associated with the dog, or does it include changes to the schedule or even next year’s vacation?

Setting the scope too narrow on the risk question might prevent a hazard and the resulting risk from being identified and assessed or making it too broad could prevent the risk assessment from getting to the real purpose.

Risk questions can be broken down in a tree structure to more define scopes, which can help drive effective teams.

For example, if we are doing a risk assessment on changing the family’s diet, it might look like this:

The current draft of ICH Q9 places a lot of importance on the risk question, rightfully so. As a tool it helps focus and define the risk assessment, producing better results.