Risk, Hazard and Harm

Risk Is….

The combination of the probability of the occurrence of the harm and the severity of that harm.

The effect of uncertainty on objectives

Often characterized by reference to the potential event and consequences or combination of these

Often expressed in terms of a combination of the consequences of an event (including in changes in circumstances) and the associated likelihood of the occurrence

 

Hazard, harm and risk

HazardHarmRisk
Enabling state that leads to the possibility of harmInjury or damageProbability of harm from a situation triggered by the hazard.
Hazard harm and risk

A hazard is defined in ISO 12100 as “The potential source of harm.” This definition is carried through other ISOs and regulatory guidances. The hazard is what could go wrong, our “What If…”, it is when we start engaging the outcome identification loop to query uncertainty about the future.

Harm are those injuries or damages I should care about.

Every risk assessment is really asking “What could go wrong,” and then answering two questions:

  1. If it did go wrong how bad is it – the Harm
  2. And how likely is it to go wrong – Probability.

Risk is then the combination of those things as a magnitude or priority.

Risk assessment tools break down into two major camps. Those that start with the hazards, asking how something can fail; and those that start with the harms, asking what bad things do we want to avoid.

Escalation of Critical Events

Event management systems need to have an escalation mechanism to ensure critical events are quickly elevated to a senior level to ensure organization-wide timely reactions.

Consistent Event Reporting

There are many reasons for a fast escalation.

  • Events that trigger reporting to Regulatory Agencies (e.g. Serious Breach, Urgent Safety Measures (UK), Field Alerts, Biological Product Deviation, Medical Device Report)
  • Events that require immediate action to prevent additional harm from across the organization
  • Events that require marshalling resources from large parts of the organization

GMP

GCP

GPVP

GLP

Research

IT

         Impact to data integrity

       Impact to product quality/supply

       Impact to data integrity

       Data/privacy breach

       Event impacting on-time compliance rates (not isolated/steady state)

       Impact to data integrity

       Impact to data integrity

       Reference GxP area for Impact resulting from/linked to system error/failure

       Product Quality/ CMC events in accordance with MRB criteria (or other events of similar scope of impact)

       Impact to study integrity

       Impact to subject’s safety, rights or welfare

       Gaps in reporting/ collection of potential AEs

       Impact to study integrity

       Impact to study integrity

       System design, testing, deployment, upgrade, etc. event impacting GxP data integrity or regulatory compliance

       Recurring event with broad scope of impact

       Recurring event with broad scope of impact

       Recurring event with broad scope of impact

       Recurring event with broad scope of impact

       Recurring event with broad scope of impact

       Recurring event with broad scope of impact

       Impact to program milestones & corporate goals

       Impact to program milestones & corporate goals

       Impact to program milestones & corporate goals

       Impact to program milestones & corporate goals

       Impact to program milestones & corporate goals

       Potential Falsified or Counterfeit Product

       Potential Fraud or Misconduct

       Potential Fraud or Misconduct

       Credible Risk of Product Shortage

       Quality event with patient safety risk/gap

       GxP Data Breach

       Potential Product Recall

       Significant Quality Event Notified to Regulatory Authority

       System error or failure with significant GxP compliance impact

·       Potential Critical Finding Resulting from Regulatory Authority Inspection or Audit by External Body/Third Party

·       Quality Event/Observation Classified as Critical (Event or Internal Audit) Notification from Regulatory Authority or other External Authority of Findings of Significant/Critical Quality Deficiency (inspection or other than through inspection)

o   e.g.; Refusal to File, Notification of Inadequate Response to Inspection Findings (e.g.; Other Action Indicated (FDA classification), Warning Letter

 

You can drill down to a lower, more practical level, like this

Escalation Criteria

Examples of Quality Events for Escalation

Potential to adversely affect quality, safety, efficacy, performance or compliance of product (commercial or clinical)

       Contamination (product, raw material, equipment, micro; environmental)

       Product defect/deviation from process parameters or specification (on file with agencies)

       Significant GMP deviations

       Incorrect/deficient labeling

       Product complaints (significant PC, trends in PCs)

       OOS/OOT (e.g., stability)

Product counterfeiting, tampering, theft

       Product counterfeiting, tampering, theft reportable to Health Authority (HA)

       Lost/stolen IMP

       Fraud or misconduct associated with counterfeiting, tampering, theft

       Potential to impact product supply (e.g., removal, correction, recall)

Product shortage likely to disrupt patient care and/or reportable to HA

       Disruption of product supply due to product quality events, natural disasters (business continuity disruption), OOS impact, capacity constraints

Potential to cause patient harm associated with a product quality event

       Urgent Safety Measure, Serious Breach, Significant Product Compliant, Safety Signal that are determined associated with a product quality event

Significant GMP non-compliance/event

       Non-compliance or non-conformance event with potential to impact product performance meeting specification, safety efficacy or regulatory requirements

Regulatory Compliance Event

       Significant (critical, repeat) regulatory inspection findings, lack of commitment adherence

       Notification of directed/for cause inspection

       Notification of HA correspondence indicating potential regulatory action

 

The Failure Space of Clinical Trials – Protocol Deviations and Events

Let us turn our failure space model, and level of problems, to deviations in a clinical trial. This is one of those areas that regulations and tribal practice have complicated, perhaps needlessly. It is also complicated by the different players of clinical sites, sponsor, and usually these days a number of Contract Research Organizations (CRO).

What is a Protocol Deviation?

Protocol deviation is any change, divergence, or departure from the study design or procedures defined in the approved protocol.

Protocol deviations may include unplanned instances of protocol noncompliance. For example, situations in which the clinical investigator failed to perform tests or examinations as required by the protocol or failures on the part of subjects to complete scheduled visits as required by the protocol, would be considered protocol deviations.

In the case of deviations which are planned exceptions to the protocol such deviations should be reviewed and approved by the IRB, the sponsor, and by the FDA for medical devices, prior to implementation, unless the change is necessary to eliminate apparent immediate hazards to the human subjects (21 CFR 312.66), or to protect the life or physical well-being of the subject (21 CFR 812.150(a)(4)).

The FDA, July 2020. Compliance Program Guidance Manual for Clinical Investigator Inspections (7348.811).

In assessing protocol deviations/violations, the FDA instructs field staff to determine whether changes to the protocol were: (1) documented by an amendment, dated, and maintained with the protocol; (2) reported to the sponsor (when initiated by the clinical investigator); and (3) approved by the IRB and FDA (if applicable) before implementation (except when necessary to eliminate apparent immediate hazard(s) to human subjects).

Regulation/GuidanceStates
ICH E-6 (R2) Section 4.5.1-4.5.44.5.1“trial should be conducted in compliance with the protocol agreed to by the sponsor and, if required by the regulatory authorities…”
4.5.2 The investigator should not implement any deviation from, or changes of, the protocol without agreement by the sponsor and prior review and documented approval/favorable opinion from the IRB/IEC of an amendment, except where necessary to eliminate an immediate hazard(s) to trial subjects, or when the change(s) involves only logistical or administrative aspects of the trial (e.g., change in monitor(s), change of telephone number(s)).
4.5.3 The investigator, or person designated by the investigator, should document and explain any deviation from the approved protocol.
4.5.4 The investigator may implement a deviation from, or a change in, the protocol to eliminate an immediate hazard(s) to trial subjects without prior IRB/IEC approval/favorable opinion.
ICH E3, section 9.6The sponsor should describe the quality management approach implemented in the trial and summarize important deviations from the predefined quality tolerance limits and remedial actions taken in the clinical study report
21CFR 312.53(vi) (a)investigators selected “Will conduct the study(ies) in accordance with the relevant, current protocol(s) and will only make changes in a protocol after notifying the sponsor, except when necessary to protect the safety, the rights, or welfare of subjects.”
21CFR 56.108(a)IRB shall….ensur[e] that changes in approved research….may not be initiated without IRB review and approval except where necessary to eliminate apparent immediate hazards to the human subjects.
21 CFR 56.108(b)“IRB shall….follow written procedures for ensuring prompt reporting to the IRB, appropriate institutional officials, and the Food and Drug Administration of… any unanticipated problems involving risks to human subjects or others…[or] any instance of serious or continuing noncompliance with these regulations or the requirements or determinations of the IRB.”
45 CFR 46.103(b)(5)Assurances applicable to federally supported or conducted research shall at a minimum include….written procedures for ensuring prompt reporting to the IRB….[of] any unanticipated problems involving risks to subjects or others or any serious or continuing noncompliance with this policy or the requirements or determinations of the IRB.
FDA Form-1572 (Section 9)lists the commitments the investigator is undertaking in signing the 1572 wherein the clinical investigator agrees “to conduct the study(ies) in accordance with the relevant, current protocol(s) and will only make changes in a protocol after notifying the sponsor, except when necessary to protect the safety, the rights, or welfare of subjects… [and] not to make any changes in the research without IRB approval, except where necessary to eliminate apparent immediate hazards to the human subjects.”
A few key regulations and guidances (not meant to be a comprehensive list)

How Protocol Deviations are Implemented

Many companies tend to have a failure scale built into their process, differentiating between protocol deviations and violations based on severity. Others use a minor, major, and even critical scale to denote differences in severity. The axis here for severity is the degree to which affects the subject’s rights, safety, or welfare, and/or the integrity of the resultant data (i.e., the sponsor’s ability to use the data in support of the drug).

Other companies divide into protocol deviations and violations:

  • Protocol Deviation: A protocol deviation occurs when, without significant consequences, the activities on a study diverge from the IRB-approved protocol, e.g., missing a visit window because the subject is traveling. Not as serious as a protocol violation.
  • Protocol Violation: A divergence from the protocol that materially (a) reduces the quality or completeness of the data, (b) makes the ICF inaccurate, or (c) impacts a subject’s safety, rights or welfare. Examples of protocol violations may include: inadequate or delinquent informed consent; inclusion/exclusion criteria not met; unreported SAEs; improper breaking of the blind; use of prohibited medication; incorrect or missing tests; mishandled samples; multiple visits missed or outside permissible windows; materially inadequate record-keeping; intentional deviation from protocol, GCP or regulations by study personnel; and subject repeated noncompliance with study requirements.

This is probably a place when nomenclature can serve to get in the way, rather than provide benefit. The EMA says pretty much the same in “ICH guideline E3 – questions and answers (R1).

Principles of Events in Clinical Practice

  1. Severity of the event is based on degree to which affects the subject’s rights, safety, or welfare, and/or the integrity of the resultant data
  2. Events (problems, deviations, etc) will happen at all levels of a clinical practice (Sponsor, CRO, Site, etc)
  3. Events happen beyond the Protocol. These need to be managed appropriately as well.
  4. The event needs to be categorized, evaluated and trended by the sponsor

Severity of the Event

Starting in the study planning stage, ICH E6(R2) GCP requires sponsors to identify risks to critical study processes and study data and to evaluate these risks based on likelihood, detectability and impact on subject safety and data integrity.

Sponsors then establish key quality indicators (KQIs) and quality tolerance thresholds. KQI is really just a key risk indicator and should be treated similarly.

Study events that exceed the risk threshold should trigger an evaluation to determine if action is needed. In this way, sponsors can proactively manage risk and address protocol noncompliance.

The best practice here is to have a living risk assessment for each study. Evaluate across studies to understand your overall organization risk, and look for opportunities for wide-scale mitigations. Feedup into your risk register.

Event Classification for Clinical Protocols and GCPs

Where the Event happens

Deviations in the clinical space are a great example of the management of supplier events, and at the end of the day there is little difference between a GMP supplier event management, a GLP or a GCP. The individual requirements might be different but the principles and the process are the same.

Each entity in the trial organization should have their own deviation system where they investigate deviations, performing root cause investigation and enacting CAPAs.

This is where it starts to get tricky. first of all, not all sites have the infrastructure to do this well. Second the nature of reporting, usually through the Electronic Data Capture (EDC) system, can lead to balkanization at the site. Site’s need to have strong compliance programs through compiling deviation details into a single sitewide system that allows the site to trend deviations across studies in addition to following sponsor reporting requirements.

Unfortunately too many site’s rely on the sponsor’s program. Sponsors need to be evaluating the strength of this program during site selection and through auditing.

Events Happen

Consistent Event Reporting is Critical

Deviations should be to all process, procedure and plans, and just not the protocol.

Categorizing deviations is usually a pain point and an area where more consistency needs to be driven. I recommend first having a good standard set of categorizations. The industry would benefit from adopting a standard, and I think Norman Goldfarb’s proposal is still the best.

Once you have categories, and understand to your KQIs and other aspects you need to make sure they are consistently done. The key mechanisms of this are:

  1. Training
  2. Monitoring (in all its funny permutations)
  3. Periodic evaluations and Trending

Deviations should be trended, at a minimum, in several ways:

  1. Per site per study
  2. Per site all activities
  3. All sites per study
  4. All sites all activities

And remember, trending doesn’t count of you do not analyze the problem and take appropriate CAPAs.

This will allow trends to be identified and appropriate corrective and preventive actions identified to systematically improve.

Catalent Belgium Form 483 and Contamination Control

The FDA recently released a Form 483 it handed to Catalent Belgium following an inspection of its 265,000 square-foot facility in Brussels in October 2021. Catalent is a pretty sizable entity, so it is very valuable to see what we can learn from their observations.

Failure to adequately assess an unexplained discrepancy or deviation

“Standard Operating Procedure STB-QA-0010, Deviation Management, v21 classifies deviations as minor, major or critical based on the calculation of a risk priority number, with a HEPA filter failure within a Grade A environment often classified as minor. Specifically, Deviation 327567 (Date of occurrence 04 March 2021) was for a HEPA filter failure on the <redacted> fill line, with a breach at the HEPA filter frame.”

This one is more common than it should be. I’ve recently written about categorization and criticality of events. I want to stress the term potential when addressing impact in the classification of events.

Control barriers exist for a reason. You breach that control barrier in any way, you have the potential to impact product or environment. It is really easy for experienced SMEs to say “But this has never had any real impact before” and then downgrade the deviation classification. Before long it becomes the norm that HEPA filter failures are minor because they never have impact. And then one does. Then there are shortages or worse.

It is important to avoid that complacency and treat each and every control barrier failure to the same level of investigation based on their potentiality to impact.

The other problem here is failure to identify trends and deal with them. I can honestly say that the last thing I ever want anyone, especially an inspector, to write about something where I have quality oversight is a failure to investigate multiple control barrier events.

Other GMP manufacturing areas have a similar elevated level of HEPA filter failures, with the root cause of the HEPA filter failures unknown. There is no CAPA in support of correction action. Your firm failed to ensure your investigations identify appropriate root causes and you failed to implement sustainable corrective action and preventive action (CAPA).

Contamination Control function

Observation 2 and 3 are doozies, but there is probably a lack of expertise involved here. The site is using out-of-date and inadequate methods in their validation. Hire a strong contamination control expert and leverage them. Build expertise in the organization through a robust training program. Connect this to all relevant quality systems/processes.

Corrective Maintenance and Troubleshooting

“Equipment and facilities used in the manufacture of drug product are not adequately maintained or appropriately designed to facilitate operations for their intended use.

The asset control lifecycle matters, and corrective maintenance can not be shorted.

This is starting to feel a lot like my upcoming presentation at the 2022 ISPE Aseptic Conference where I will be speaking on “Contamination Control, Risk and the Quality Management System

Contamination Control is a fairly wide term used to mean “getting microbiologists out of the lab” and involved in risk management and the quality management system. This presentation will evaluate best practices in building a contamination control strategy and ensuring its use throughout the quality system. Leveraging a House of Quality approach, participants will learn how to: Create targeted/ risk based measures of contamination avoidance; Implement Key performance indicators to assess status of contamination control; and ensure a defined strategy for deviation management (investigations), CAPA and change management.”

Maybe we can talk more there!

Managing Events Systematically

Being good at problem-solving is critical to success in an organization. I’ve written quite a bit on problem-solving, but here I want to tackle the amount of effort we should apply.

Not all problems should be treated the same. There are also levels of problems. And these two aspects can contribute to some poor problem-solving practices.

It helps to look at problems systematically across our organization. The iceberg analogy is a pretty popular way to break this done focusing on Events, Patterns, Underlying Structure, and Mental Model.

Iceberg analogy

Events

Events start with the observation or discovery of a situation that is different in some way. What is being observed is a symptom and we want to quickly identify the problem and then determine the effort needed to address it.

This is where Art Smalley’s Four Types of Problems comes in handy to help us take a risk-based approach to determining our level of effort.

Type 1 problems, Troubleshooting, allows us to set problems with a clear understanding of the issue and a clear pathway. Have a flat tire? Fix it. Have a document error, fix it using good documentation practices.

It is valuable to work the way through common troubleshooting and ensure the appropriate linkages between the different processes, to ensure a system-wide approach to problem solving.

Corrective maintenance is a great example of troubleshooting as it involved restoring the original state of an asset. It includes documentation, a return to service and analysis of data. From that analysis of data problems are identified which require going deeper into problem-solving. It should have appropriate tie-ins to evaluate when the impact of an asset breaking leads to other problems (for example, impact to product) which can also require additional problem-solving.

It can be helpful for the organization to build decision trees that can help folks decide if a given problem stays as troubleshooting or if it it also requires going to type 2, “gap from standard.”

Type 2 problems, gap from standard, means that the actual result does not meet the expected and there is a potential of not meeting the core requirements (objectives) of the process, product, or service. This is the place we start deeper problem-solving, including root cause analysis.

Please note that often troubleshooting is done in a type 2 problem. We often call that a correction. If the bioreactor cannot maintain temperature during a run, that is a type 2 problem but I am certainly going to immediately apply troubleshooting as well. This is called a correction.

Take documentation errors. There is a practice in place, part of good documentation practices, for addressing troubleshooting around documents (how to correct, how to record a comment, etc). By working through the various ways documentation can go wrong, applying which ones are solved through troubleshooting and don’t involve type 2 problems, we can create a lot of noise in our system.

Core to the quality system is trending, looking for possible signals that require additional effort. Trending can help determine where problems lay and can also drive up the level of effort necessary.

Underlying Structure

Root Cause Analysis is about finding the underlying structure of the problem that defines the work applied to a type 2 problem.

Not all problems require the same amount of effort, and type 2 problems really have a scale based on consequences, that can help drive the level of effort. This should be based on the impact to the organization’s ability to meet the quality objectives, the requirements behind the product or service.

For example, in the pharma world there are three major criteria:

  •  safety, rights, or well-being of patients (including subjects and participants human and non-human)
  • data integrity (includes confidence in the results, outcome, or decision dependent on the data)
  • ability to meet regulatory requirements (which stem from but can be a lot broader than the first two)

These three criteria can be sliced and diced a lot of ways, but serve our example well.

To these three criteria we add a scale of possible harm to derive our criticality, an example can look like this:

ClassificationDescription
CriticalThe event has resulted in, or is clearly likely to result in, any one of the following outcomes:   significant harm to the safety, rights, or well-being of subjects or participants (human or non-human), or patients; compromised data integrity to the extent that confidence in the results, outcome, or decision dependent on the data is significantly impacted; or regulatory action against the company.
MajorThe event(s), were they to persist over time or become more serious, could potentially, though not imminently, result in any one of the following outcomes:  
harm to the safety, rights, or well-being of subjects or participants (human or non-human), or patients; compromised data integrity to the extent that confidence in the results, outcome, or decision dependent on the data is significantly impacted.
MinorAn isolated or recurring triggering event that does not otherwise meet the definitions of Critical or Major quality impacts.
Example of Classification of Events in a Pharmaceutical Quality System

This level of classification will drive the level of effort on the investigation, as well as drive if the CAPA addresses underlying structures alone or drives to addressing the mental models and thus driving culture change.

Mental Model

Here is where we address building a quality culture. In CAPA lingo this is usually more a preventive action than a corrective action. In the simplest of terms, corrective actions is address the underlying structures of the problem in the process/asset where the event happened. Preventive actions deal with underlying structures in other (usually related) process/assets or get to the Mindsets that allowed the underlying structures to exist in the first place.

Solving Problems Systematically

By applying this system perspective to our problem solving, by realizing that not everything needs a complete rebuild of the foundation, by looking holistically across our systems, we can ensure that we are driving a level of effort to truly build the house of quality.