Tag: success/failure space

Emergence in the Quality System

The concept of emergence—where complex behaviors arise unpredictably from interactions among simpler components—has haunted and inspired quality professionals since Aristotle first observed that “the whole is something besides the parts.” In modern quality systems, this ancient paradox takes new form: our meticulously engineered controls often birth unintended consequences, from phantom batch failures to self-reinforcing compliance gaps. Understanding emergence isn’t just an academic exercise—it’s a survival skill in an era where hyperconnected processes and globalized supply chains amplify systemic unpredictability.

The Spectrum of Emergence: From Predictable to Baffling

Emergence manifests across a continuum of complexity, each type demanding distinct management approaches:

1. Simple Emergence
Predictable patterns emerge from component interactions, observable even in abstracted models. Consider document control workflows: while individual steps like review or approval seem straightforward, their sequencing creates emergent properties like approval cycle times. These can be precisely modeled using flowcharts or digital twins, allowing proactive optimization.

2. Weak Emergence
Behaviors become explainable only after they occur, requiring detailed post-hoc analysis. A pharmaceutical company’s CAPA system might show seasonal trends in effectiveness—a pattern invisible in individual case reviews but emerging from interactions between manufacturing schedules, audit cycles, and supplier quality fluctuations. Weak emergence often reveals itself through advanced analytics like machine learning clustering.

3. Multiple Emergence
Here, system behaviors directly contradict component properties. A validated sterile filling line passing all IQ/OQ/PQ protocols might still produce unpredictable media fill failures when integrated with warehouse scheduling software. This “emergent invalidation” stems from hidden interaction vectors that only manifest at full operational scale.

4. Strong Emergence
Consistent with components but unpredictably manifested, strong emergence plagues culture-driven quality systems. A manufacturer might implement identical training programs across global sites, yet some facilities develop proactive quality innovation while others foster blame-avoidance rituals. The difference emerges from subtle interactions between local leadership styles and corporate KPIs.

5. Spooky Emergence
The most perplexing category, where system behaviors defy both component properties and simulation. A medical device company once faced identical cleanrooms producing statistically divergent particulate counts—despite matching designs, procedures, and personnel. Root cause analysis eventually traced the emergence to nanometer-level differences in HVAC duct machining, interacting with shift-change lighting schedules to alter airflow dynamics.

TypeCharacteristicsQuality System Example
SimplePredictable through component analysisDocument control workflows
WeakExplainable post-occurrence through detailed modelingCAPA effectiveness trends
MultipleContradicts component properties, defies simulationValidated processes failing at scale
StrongConsistent with components but unpredictably manifestedCulture-driven quality behaviors
SpookyDefies component properties and simulation entirelyPhantom batch failures in identical systems

The Modern Catalysts of Emergence

Three forces amplify emergence in contemporary quality systems:

Hyperconnected Processes

IoT-enabled manufacturing equipment generates real-time data avalanches. A biologics plant’s environmental monitoring system might integrate 5,000 sensors updating every 15 seconds. The emergent property? A “data tide” that overwhelms traditional statistical process control, requiring AI-driven anomaly detection to discern meaningful signals.

Compressed Innovation Cycles

Compressed innovation cycles are transforming the landscape of product development and quality management. In this new paradigm, the pressure to deliver products faster—whether due to market demands, technological advances, or public health emergencies—means that the traditional, sequential approach to development is replaced by a model where multiple phases run in parallel. Design, manufacturing, and validation activities that once followed a linear path now overlap, requiring organizations to verify quality in real time rather than relying on staged reviews and lengthy data collection.

One of the most significant consequences of this acceleration is the telescoping of validation windows. Where stability studies and shelf-life determinations once spanned years, they are now compressed into a matter of months or even weeks. This forces quality teams to make critical decisions based on limited data, often relying on predictive modeling and statistical extrapolation to fill in the gaps. The result is what some call “validation debt”—a situation where the pace of development outstrips the accumulation of empirical evidence, leaving organizations to manage risks that may not be fully understood until after product launch.

Regulatory frameworks are also evolving in response to compressed innovation cycles. Instead of the traditional, comprehensive submission and review process, regulators are increasingly open to iterative, rolling reviews and provisional specifications that can be adjusted as more data becomes available post-launch. This shift places greater emphasis on computational evidence, such as in silico modeling and digital twins, rather than solely on physical testing and historical precedent.

The acceleration of development timelines amplifies the risk of emergent behaviors within quality systems. Temporal compression means that components and subsystems are often scaled up and integrated before they have been fully characterized or validated in isolation. This can lead to unforeseen interactions and incompatibilities that only become apparent at the system level, sometimes after the product has reached the market. The sheer volume and velocity of data generated in these environments can overwhelm traditional quality monitoring tools, making it difficult to identify and respond to critical quality attributes in a timely manner.

Another challenge arises from the collision of different quality management protocols. As organizations attempt to blend frameworks such as GMP, Agile, and Lean to keep pace with rapid development, inconsistencies and gaps can emerge. Cross-functional teams may interpret standards differently, leading to confusion or conflicting priorities that undermine the integrity of the quality system.

The systemic consequences of compressed innovation cycles are profound. Cryptic interaction pathways can develop, where components that performed flawlessly in isolation begin to interact in unexpected ways at scale. Validation artifacts—such as artificial stability observed in accelerated testing—may fail to predict real-world performance, especially when environmental variables or logistics introduce new stressors. Regulatory uncertainty increases as control strategies become obsolete before they are fully implemented, and critical process parameters may shift unpredictably during technology transfer or scale-up.

To navigate these challenges, organizations are adopting adaptive quality strategies. Predictive quality modeling, using digital twins and machine learning, allows teams to simulate thousands of potential interaction scenarios and forecast failure modes even with incomplete data. Living control systems, powered by AI and continuous process verification, enable dynamic adjustment of specifications and risk priorities as new information emerges. Regulatory agencies are also experimenting with co-evolutionary approaches, such as shared industry databases for risk intelligence and regulatory sandboxes for testing novel quality controls.

Ultimately, compressed innovation cycles demand a fundamental rethinking of quality management. The focus shifts from simply ensuring compliance to actively navigating complexity and anticipating emergent risks. Success in this environment depends on building quality systems that are not only robust and compliant, but also agile and responsive—capable of detecting, understanding, and adapting to surprises as they arise in real time.

Supply Chain Entanglement

Globalization has fundamentally transformed supply chains, creating vast networks that span continents and industries. While this interconnectedness has brought about unprecedented efficiencies and access to resources, it has also introduced a web of hidden interaction vectors—complex, often opaque relationships and dependencies that can amplify both risk and opportunity in ways that are difficult to predict or control.

At the heart of this complexity is the fragmentation of production across multiple jurisdictions. This spatial and organizational dispersion means that disruptions—whether from geopolitical tensions, natural disasters, regulatory changes, or even cyberattacks—can propagate through the network in unexpected ways, sometimes surfacing as quality issues, delays, or compliance failures far from the original source of the problem.

Moreover, the rise of powerful transnational suppliers, sometimes referred to as “Big Suppliers,” has shifted the balance of power within global value chains. These entities do not merely manufacture goods; they orchestrate entire ecosystems of production, labor, and logistics across borders. Their decisions about sourcing, labor practices, and compliance can have ripple effects throughout the supply chain, influencing not just operational outcomes but also the diffusion of norms and standards. This reconsolidation at the supplier level complicates the traditional view that multinational brands are the primary drivers of supply chain governance, revealing instead a more distributed and dynamic landscape of influence.

The hidden interaction vectors created by globalization are further obscured by limited supply chain visibility. Many organizations have a clear understanding of their direct, or Tier 1, suppliers but lack insight into the lower tiers where critical risks often reside. This opacity can mask vulnerabilities such as overreliance on a single region, exposure to forced labor, or susceptibility to regulatory changes in distant markets. As a result, companies may find themselves blindsided by disruptions that originate deep within their supply networks, only becoming apparent when they manifest as operational or reputational crises.

In this environment, traditional risk management approaches are often insufficient. The sheer scale and complexity of global supply chains demand new strategies for mapping connections, monitoring dependencies, and anticipating how shocks in one part of the world might cascade through the system. Advanced analytics, digital tools, and collaborative relationships with suppliers are increasingly essential for uncovering and managing these hidden vectors. Ultimately, globalization has made supply chains more efficient but also more fragile, with hidden interaction points that require constant vigilance and adaptive management to ensure resilience and sustained performance.

Emergence and the Success/Failure Space: Navigating Complexity in System Design

The interplay between emergence and success/failure space reveals a fundamental tension in managing complex systems: our ability to anticipate outcomes is constrained by both the unpredictability of component interactions and the inherent asymmetry between defining success and preventing failure. Emergence is not merely a technical challenge, but a manifestation of how systems oscillate between latent potential and realized risk.

Success/Failure Space, or Why We Can Sometimes Seem Pessimistic

The Duality of Success and Failure Spaces

Systems exist in a continuum where:

  • Success space encompasses infinite potential pathways to desired outcomes, characterized by continuous variables like efficiency and adaptability.
  • Failure space contains discrete, identifiable modes of dysfunction, often easier to consensus-build around than nebulous success metrics.

Emergence complicates this duality. While traditional risk management focuses on cataloging failure modes, emergent behaviors—particularly strong emergence—defy this reductionist approach. Failures can arise not from component breakdowns, but from unexpected couplings between validated subsystems operating within design parameters. This creates a paradox: systems optimized for success space metrics (e.g., throughput, cost efficiency) may inadvertently amplify failure space risks through emergent interactions.

Emergence as a Boundary Phenomenon

Emergent behaviors manifest at the interface of success and failure spaces:

  1. Weak Emergence
    Predictable through detailed modeling, these behaviors align with traditional failure space analysis. For example, a pharmaceutical plant might anticipate temperature excursion risks in cold chain logistics through FMEA, implementing redundant monitoring systems.
  2. Strong Emergence
    Unpredictable interactions that bypass conventional risk controls. Consider a validated ERP system that unexpectedly generates phantom batch records when integrated with new MES modules—a failure emerging from software handshake protocols never modeled during individual system validation.

To return to a previous analogy of house purchasing to illustrate this dichotomy: while we can easily identify foundation cracks (failure space), defining the “perfect home” (success space) remains subjective. Similarly, strong emergence represents foundation cracks in system architectures that only become visible after integration.

Reconciling Spaces Through Emergence-Aware Design

To manage this complexity, organizations must:

1. Map Emergence Hotspots
Emergence hotspots represent critical junctures where localized interactions generate disproportionate system-wide impacts—whether beneficial innovations or cascading failures. Effectively mapping these zones requires integrating spatial, temporal, and contextual analytics to navigate the interplay between component behaviors and collective outcomes..

2. Implement Ambidextrous Monitoring
Combine failure space triggers (e.g., sterility breaches) with success space indicators (e.g., adaptive process capability) – pairing traditional deviation tracking with positive anomaly detection systems that flag beneficial emergent patterns.

3. Cultivate Graceful Success

Graceful success represents a paradigm shift from failure prevention to intelligent adaptation—creating systems that maintain core functionality even when components falter. Rooted in resilience engineering principles, this approach recognizes that perfect system reliability is unattainable, and instead focuses on designing architectures that fail into high-probability success states while preserving safety and quality.

  1. Controlled State Transitions: Systems default to reduced-but-safe operational modes during disruptions.
  2. Decoupled Subsystem Design: Modular architectures prevent cascading failures. This implements the four layers of protection philosophy through physical and procedural isolation.
  3. Dynamic Risk Reconfiguration: Continuously reassess risk priorities using real-time data brings the concept of fail forward into structured learning modes.

This paradigm shift from failure prevention to failure navigation represents the next evolution of quality systems. By designing for graceful success, organizations transform disruptions into structured learning opportunities while maintaining continuous value delivery—a critical capability in an era of compressed innovation cycles and hyperconnected supply chains.

The Emergence Literacy Imperative

This evolution demands rethinking Deming’s “profound knowledge” for the complexity age. Just as failure space analysis provides clearer boundaries, understanding emergence gives us lenses to see how those boundaries shift through system interactions. The organizations thriving in this landscape aren’t those eliminating surprises, but those building architectures where emergence more often reveals novel solutions than catastrophic failures—transforming the success/failure continuum into a discovery engine rather than a risk minefield.

Strategies for Emergence-Aware Quality Leadership

1. Cultivate Systemic Literacy
Move beyond component-level competence. Trains quality employees in basic complexity science..

2. Design for Graceful Failure
When emergence inevitably occurs, systems should fail into predictable states. For example, you can redesign batch records with:

  • Modular sections that remain valid if adjacent components fail
  • Context-aware checklists that adapt requirements based on real-time bioreactor data
  • Decoupled approvals allowing partial releases while investigating emergent anomalies

3. Harness Beneficial Emergence
The most advanced quality systems intentionally foster positive emergence.

The Emergence Imperative

Future-ready quality professionals will balance three tensions:

  • Prediction AND Adaptation : Investing in simulation while building response agility
  • Standardization AND Contextualization : Maintaining global standards while allowing local adaptation
  • Control AND Creativity : Preventing harm while nurturing beneficial emergence

The organizations thriving in this new landscape aren’t those with perfect compliance records, but those that rapidly detect and adapt to emergent patterns. They understand that quality systems aren’t static fortresses, but living networks—constantly evolving, occasionally surprising, and always revealing new paths to excellence.

In this light, Aristotle’s ancient insight becomes a modern quality manifesto: Our systems will always be more than the sum of their parts. The challenge—and opportunity—lies in cultivating the wisdom to guide that “more” toward better outcomes.

Trusting the Journey: When Uncertainty is a Feature

I spend a lot of time discussing uncertainty and how to address it in our quality system and within our organization. However, we often find ourselves at a crossroads, faced with uncertainty and the unknown in our careers – certainly, the last few years have been hard in biotech. My current approach has been to reframe this uncertainty not as an obstacle but as a feature of my journey—something it might have taken me 54 years to learn. I am striving to embrace the concept of “trusting the process” personally and as a quality practitioner so I can navigate life’s twists and turns with greater ease and purpose. As we go into the New Year, here are my current approaches.

The Power of Small Steps

If you are like me, it is easy to get lost in the day-to-day pressures of work. There is always a new issue, a new course correction. It is easy to focus on the overwhelming big picture to our next best steps and forget that the journey counts. My QA problem-solving self often wants to focus on problem-solving and forgets that we must strike a balance between action and acceptance, recognizing that while we can’t control every outcome, we can control our response to each situation. I am working to maintain agency in the present moment while surrendering to the unfolding path ahead.

Embracing Uncertainty as a Catalyst for Growth

Uncertainty, often viewed as a source of anxiety, can actually be a powerful catalyst for growth and innovation. By reframing uncertainty as a feature, we can open ourselves up to new possibilities and unexpected opportunities. This mindset shift encourages us to:

  1. Remain curious and open-minded
  2. Adapt more readily to changing circumstances
  3. Cultivate resilience in the face of challenges

The Art of Experimentation

One practical way to embrace uncertainty is through the practice of running small experiments. These controlled tests allow us to:

  • Gather quick feedback
  • Minimize risk
  • Foster creativity and innovation

We create a culture of continuous learning and improvement by incorporating regular experimentation into our personal and professional lives. This approach is particularly valuable when balancing the demands of serving an organization while pursuing personal growth.

Balancing Service and Growth

The challenge of running small experiments while fulfilling organizational responsibilities is common. Here are some strategies to help strike that balance:

  1. Integrate experiments into daily work: Look for opportunities to test new ideas or approaches within your existing projects and responsibilities.
  2. Time-box your experiments: Set aside specific, limited time periods for experimentation to ensure it doesn’t interfere with core duties.
  3. Communicate with stakeholders: Share your experimental approach, highlighting how it can benefit.
  4. Learn from successes and failures: Treat every experiment as a learning opportunity, regardless of the outcome.
  5. Start small and scale up: Begin with low-risk, high-potential experiments and gradually expand based on results and buy-in.

Cultivating Trust in the Process

Trusting the journey is not about blind faith or passivity. Instead, it’s about developing a deep relationship with your wisdom and decision-making process. This trust is built over time through:

  • Consistent self-reflection
  • Recognizing patterns in your choices and their outcomes
  • Staying connected to your core values and goals
  • Celebrating small wins and learning from setbacks

As you cultivate this trust, you’ll find yourself better equipped to navigate uncertainty confidently and gracefully.

Embracing the Journey

Trusting the journey can feel counterintuitive in a world that often demands certainty and immediate results. However, by embracing uncertainty as a feature of our growth process, we open ourselves to a richer, more fulfilling experience. Through small experiments, mindful action, and a willingness to surrender to the unknown, we can create a life and career that is both purposeful and adaptable.

Remember, the journey itself is where true growth and discovery happen. By trusting the process and focusing on our next best steps, we can navigate the complexities of life with greater ease and authenticity. So, take that first step, run that small experiment, and trust that the journey will unfold in ways you may never have imagined.

This is my New Year’s plan: to continue to apply to my personal space the skills and mindsets that have made my career so fruitful.

Thinking of Swiss Cheese: Reason’s Theory of Active and Latent Failures

The Theory of Active and Latent Failures was proposed by James Reason in his book, Human Error. Reason stated accidents within most complex systems, such as health care, are caused by a breakdown or absence of safety barriers across four levels within a system. These levels can best be described as Unsafe Acts, Preconditions for Unsafe Acts, Supervisory Factors, and Organizational Influences. Reason used the term “active failures” to describe factors at the Unsafe Acts level, whereas “latent failures” was used to describe unsafe conditions higher up in the system.

This is represented as the Swiss Cheese model, and has become very popular in root cause analysis and risk management circles and widely applied beyond the safety world.

Swiss Cheese Model

In the Swiss Cheese model, the holes in the cheese depict the failure or absence of barriers within a system. Such occurrences represent failures that threaten the overall integrity of the system. If such failures never occurred within a system (i.e., if the system were perfect), then there would not be any holes in the cheese. We would have a nice Engelberg cheddar.

Not every hole that exists in a system will lead to an error. Sometimes holes may be inconsequential. Other times, holes in the cheese may be detected and corrected before something bad happens. This process of detecting and correcting errors occurs all the time.

The holes in the cheese are dynamic, not static. They open and close over time due to many factors, allowing the system to function appropriately without catastrophe. This is what human factors engineers call “resilience.” A resilient system is one that can adapt and adjust to changes or disturbances.

Holes in the cheese open and close at different rates. The rate at which holes pop up or disappear is determined by the type of failure the hole represents.

  1. Holes that occur at the Unsafe Acts level, and even some at the Preconditions level, represent active failures. Active failures usually occur during the activity of work and are directly linked to the bad outcome. Active failures change during the process of performing, opening, and closing over time as people make errors, catch their errors, and correct them.
  2. Latent failures occur higher up in the system, above the Unsafe Acts level — the Organizational, Supervisory, and Preconditions levels. These failures are referred to as “latent” because when they occur or open, they often go undetected. They can lie “dormant” or “latent” in the system for an extended period of time before they are recognized. Unlike active failures, latent failures do not close or disappear quickly.

Most events (harms) are associated with multiple active and latent failures. Unlike the typical Swiss Cheese diagram above, which shows an arrow flying through one hole at each level of the system, there can be a variety of failures at each level that interact to produce an event. In other words, there can be several failures at the Organizational, Supervisory, Preconditions, and Unsafe Acts levels that all lead to harm. The number of holes in the cheese associated with events are more frequent at the Unsafe Acts and Preconditions levels, but (usually) become fewer as one progresses upward through the Supervisory and Organizational levels.

Given the frequency and dynamic nature of activities, there are more opportunities for holes to open up at the Unsafe and Preconditions levels on a frequent basis and there are often more holes identified at these levels during root cause investigation and risk assessments.

The way the holes in the cheese interact across levels is important:

  • One-to-many mapping of causal factors is when a hole at a higher level (e.g., Preconditions) may result in several holes at a lower level (e.g. Unsafe acts)
  • Many-to-one mapping of causal factors when multiple holes at the higher level (e.g. preconditions) might interact to produce a single hole at the lower level (e.g. Unsafe Acts)

By understand the Swiss Cheese model, and Reason’s wider work in Active and Latent Failures, we can strengthen our approach to problem-solving.

Plus cheese is cool.

Swiss Cheese on a cheese board with knife

The Failure Space of Clinical Trials – Protocol Deviations and Events

Let us turn our failure space model, and level of problems, to deviations in a clinical trial. This is one of those areas that regulations and tribal practice have complicated, perhaps needlessly. It is also complicated by the different players of clinical sites, sponsor, and usually these days a number of Contract Research Organizations (CRO).

What is a Protocol Deviation?

Protocol deviation is any change, divergence, or departure from the study design or procedures defined in the approved protocol.

Protocol deviations may include unplanned instances of protocol noncompliance. For example, situations in which the clinical investigator failed to perform tests or examinations as required by the protocol or failures on the part of subjects to complete scheduled visits as required by the protocol, would be considered protocol deviations.

In the case of deviations which are planned exceptions to the protocol such deviations should be reviewed and approved by the IRB, the sponsor, and by the FDA for medical devices, prior to implementation, unless the change is necessary to eliminate apparent immediate hazards to the human subjects (21 CFR 312.66), or to protect the life or physical well-being of the subject (21 CFR 812.150(a)(4)).

The FDA, July 2020. Compliance Program Guidance Manual for Clinical Investigator Inspections (7348.811).

In assessing protocol deviations/violations, the FDA instructs field staff to determine whether changes to the protocol were: (1) documented by an amendment, dated, and maintained with the protocol; (2) reported to the sponsor (when initiated by the clinical investigator); and (3) approved by the IRB and FDA (if applicable) before implementation (except when necessary to eliminate apparent immediate hazard(s) to human subjects).

Regulation/GuidanceStates
ICH E-6 (R2) Section 4.5.1-4.5.44.5.1“trial should be conducted in compliance with the protocol agreed to by the sponsor and, if required by the regulatory authorities…”
4.5.2 The investigator should not implement any deviation from, or changes of, the protocol without agreement by the sponsor and prior review and documented approval/favorable opinion from the IRB/IEC of an amendment, except where necessary to eliminate an immediate hazard(s) to trial subjects, or when the change(s) involves only logistical or administrative aspects of the trial (e.g., change in monitor(s), change of telephone number(s)).
4.5.3 The investigator, or person designated by the investigator, should document and explain any deviation from the approved protocol.
4.5.4 The investigator may implement a deviation from, or a change in, the protocol to eliminate an immediate hazard(s) to trial subjects without prior IRB/IEC approval/favorable opinion.
ICH E3, section 9.6The sponsor should describe the quality management approach implemented in the trial and summarize important deviations from the predefined quality tolerance limits and remedial actions taken in the clinical study report
21CFR 312.53(vi) (a)investigators selected “Will conduct the study(ies) in accordance with the relevant, current protocol(s) and will only make changes in a protocol after notifying the sponsor, except when necessary to protect the safety, the rights, or welfare of subjects.”
21CFR 56.108(a)IRB shall….ensur[e] that changes in approved research….may not be initiated without IRB review and approval except where necessary to eliminate apparent immediate hazards to the human subjects.
21 CFR 56.108(b)“IRB shall….follow written procedures for ensuring prompt reporting to the IRB, appropriate institutional officials, and the Food and Drug Administration of… any unanticipated problems involving risks to human subjects or others…[or] any instance of serious or continuing noncompliance with these regulations or the requirements or determinations of the IRB.”
45 CFR 46.103(b)(5)Assurances applicable to federally supported or conducted research shall at a minimum include….written procedures for ensuring prompt reporting to the IRB….[of] any unanticipated problems involving risks to subjects or others or any serious or continuing noncompliance with this policy or the requirements or determinations of the IRB.
FDA Form-1572 (Section 9)lists the commitments the investigator is undertaking in signing the 1572 wherein the clinical investigator agrees “to conduct the study(ies) in accordance with the relevant, current protocol(s) and will only make changes in a protocol after notifying the sponsor, except when necessary to protect the safety, the rights, or welfare of subjects… [and] not to make any changes in the research without IRB approval, except where necessary to eliminate apparent immediate hazards to the human subjects.”
A few key regulations and guidances (not meant to be a comprehensive list)

How Protocol Deviations are Implemented

Many companies tend to have a failure scale built into their process, differentiating between protocol deviations and violations based on severity. Others use a minor, major, and even critical scale to denote differences in severity. The axis here for severity is the degree to which affects the subject’s rights, safety, or welfare, and/or the integrity of the resultant data (i.e., the sponsor’s ability to use the data in support of the drug).

Other companies divide into protocol deviations and violations:

  • Protocol Deviation: A protocol deviation occurs when, without significant consequences, the activities on a study diverge from the IRB-approved protocol, e.g., missing a visit window because the subject is traveling. Not as serious as a protocol violation.
  • Protocol Violation: A divergence from the protocol that materially (a) reduces the quality or completeness of the data, (b) makes the ICF inaccurate, or (c) impacts a subject’s safety, rights or welfare. Examples of protocol violations may include: inadequate or delinquent informed consent; inclusion/exclusion criteria not met; unreported SAEs; improper breaking of the blind; use of prohibited medication; incorrect or missing tests; mishandled samples; multiple visits missed or outside permissible windows; materially inadequate record-keeping; intentional deviation from protocol, GCP or regulations by study personnel; and subject repeated noncompliance with study requirements.

This is probably a place when nomenclature can serve to get in the way, rather than provide benefit. The EMA says pretty much the same in “ICH guideline E3 – questions and answers (R1).“

Principles of Events in Clinical Practice

  1. Severity of the event is based on degree to which affects the subject’s rights, safety, or welfare, and/or the integrity of the resultant data
  2. Events (problems, deviations, etc) will happen at all levels of a clinical practice (Sponsor, CRO, Site, etc)
  3. Events happen beyond the Protocol. These need to be managed appropriately as well.
  4. The event needs to be categorized, evaluated and trended by the sponsor

Severity of the Event

Starting in the study planning stage, ICH E6(R2) GCP requires sponsors to identify risks to critical study processes and study data and to evaluate these risks based on likelihood, detectability and impact on subject safety and data integrity.

Sponsors then establish key quality indicators (KQIs) and quality tolerance thresholds. KQI is really just a key risk indicator and should be treated similarly.

Study events that exceed the risk threshold should trigger an evaluation to determine if action is needed. In this way, sponsors can proactively manage risk and address protocol noncompliance.

The best practice here is to have a living risk assessment for each study. Evaluate across studies to understand your overall organization risk, and look for opportunities for wide-scale mitigations. Feedup into your risk register.

Event Classification for Clinical Protocols and GCPs

Where the Event happens

Deviations in the clinical space are a great example of the management of supplier events, and at the end of the day there is little difference between a GMP supplier event management, a GLP or a GCP. The individual requirements might be different but the principles and the process are the same.

Each entity in the trial organization should have their own deviation system where they investigate deviations, performing root cause investigation and enacting CAPAs.

This is where it starts to get tricky. first of all, not all sites have the infrastructure to do this well. Second the nature of reporting, usually through the Electronic Data Capture (EDC) system, can lead to balkanization at the site. Site’s need to have strong compliance programs through compiling deviation details into a single sitewide system that allows the site to trend deviations across studies in addition to following sponsor reporting requirements.

Unfortunately too many site’s rely on the sponsor’s program. Sponsors need to be evaluating the strength of this program during site selection and through auditing.

Events Happen

Consistent Event Reporting is Critical

Deviations should be to all process, procedure and plans, and just not the protocol.

Categorizing deviations is usually a pain point and an area where more consistency needs to be driven. I recommend first having a good standard set of categorizations. The industry would benefit from adopting a standard, and I think Norman Goldfarb’s proposal is still the best.

Once you have categories, and understand to your KQIs and other aspects you need to make sure they are consistently done. The key mechanisms of this are:

  1. Training
  2. Monitoring (in all its funny permutations)
  3. Periodic evaluations and Trending

Deviations should be trended, at a minimum, in several ways:

  1. Per site per study
  2. Per site all activities
  3. All sites per study
  4. All sites all activities

And remember, trending doesn’t count of you do not analyze the problem and take appropriate CAPAs.

This will allow trends to be identified and appropriate corrective and preventive actions identified to systematically improve.

Success/Failure Space, or Why We Can Sometimes Seem Pessimistic

When evaluating a system we can look at it in two ways. We can identify ways a thing can fail or the various ways it can succeed.

Success/Failure Space

These are really just two sides of the coin in many ways, with identifiable points in success space coinciding with analogous points in failure space. “Maximum anticipated success” in success space coincides with “minimum anticipated failure” in failure space.

Like everything, how we frame the question helps us find answers. Certain questions require us to think in terms of failure space, others in success. There are advantages in both, but in risk management, the failure space is incredibly valuable.

It is generally easier to attain concurrence on what constitutes failure than it is to agree on what constitutes success. We may desire a house that has great windows, high ceilings, a nice yard. However, the one we buy can have a termite-infested foundation, bad electrical work, and a roof full of leaks. Whether the house is great is a matter of opinion, but we certainly know all it is a failure based on the high repair bills we are going to accrue.

Success tends to be associated with the efficiency of a system, the amount of output, the degree of usefulness. These characteristics are describable by continuous variables which are not easily modeled in terms of simple discrete events, such as “water is not hot” which characterizes the failure space. Failure, in particular, complete failure, is generally easy to define, whereas the event, success, maybe more difficult to tie down

Theoretically the number of ways in which a system can fail and the number of ways in which a system can ·succeed are both infinite, from a practical standpoint there are generally more ways to success than there are to failure. From a practical point of view, the size of the population in the failure space is less than the size of the population in the success space. This leads to risk management focusing on the failure space.

The failure space maps really well to nominal scales for severity, which can be helpful as you build your own scales for risk assessments.

For example, let’s look at an example of a morning commute.

Example of the failure space for a morning commute