Engineering runs (ERs) represent a critical yet often underappreciated component of modern biopharmaceutical validation strategies. Defined as non-GMP-scale trials that simulate production processes to identify risks and optimize parameters, Engineering Runs bridge the gap between theoretical process design and manufacturing. Their integration into the ASTM E2500 verification framework creates a powerful synergy – combining Good Engineering Practice (GEP) with Quality Risk Management (QRM) to meet evolving regulatory expectations.
When aligned with ICH Q10’s pharmaceutical quality system (PQS) and the ASTM E2500 lifecycle approach, ERs transform from operational exercises into strategic tools for:
Design space verification per ICH Q8
Scale-up risk mitigation during technology transfer
Specification & Design The standard mandates “right-sized” documentation – detailed enough to ensure product quality without unnecessary bureaucracy.
Verification This phase provides a unified verification approach focusing on:
Critical process parameters (CPPs)
Worst-case scenario testing
Leveraging vendor testing data
Acceptance & Release Final review incorporates ICH Q10’s management responsibilities, ensuring traceability from initial risk assessments to verification outcomes.
Engineering runs serve as a critical bridge between design verification and formal Process Performance Qualification (PPQ). ERs validate critical aspects of manufacturing systems by confirming:
Equipment functionality under simulated GMP conditions
Process parameter boundaries for Critical Process Parameters (CPPs)
Facility readiness through stress-testing utilities, workflows, and contamination controls
Demonstration/ Training Run prior to GMP area
Shakedown. Demonstration/Training Run in GMP area
Engineering Run
cGMP Manufacturing
Room and Equipment
Room
N/A
IOQ Post-Approval
Released and Active
Process Gas
Generation and Distribution Released Point of use assembly PQ complete
Process utility
Process Equipment
Functionally verified or calibrated as required (commissioned)
IOQ Approved
Full released
Analytical Equipment
Released
Alarms
N/A
Alarm ranges and plan defined
Alarms qualified
Raw Materials
Bill of Materials
RM in progress
Approved
Suppliers
Approval in Progress
Approved
Specifications
In Draft
Effective
Release
Non-GMP Usage decision
Released
Process Documentation
Source Documentation
To be defined in Tech Transfer Plan
Engineering Run Protocol
Tech Transfer closed
Batch Records and product specific Work Instructions
Draft
Reviewed Draft
Approved
Process and Equipment SOPs
N/A
Draft
Effective
Product Labels
N/A
Draft Labels
Approved Labels
QC Testing and Documentation
BSC and Personnel Environmental Monitoring
N/A
Effective
Analytical Methods
Suitable for use
Phase Appropriate Validation
Stability
N/A
In place
Certificate of Analysis
N/A
Defined in Engineering Protocol
Effective
Sampling Plan
Draft
Draft use as defined in engineering protocol
Effective
Operations/Execution
Operator Training
Observe and perform operations to gain hands on experience with SME observation
Process specific equipment OJT Gown qualifiedBSC OJT Aseptic OJT Material Transfer OJT (All training in eQMS)
Training in Use
Process Lock
As defined in Tech Transfer Plan
6-week prior to execution
Approved Process Description
Deviations
N/A
N/A
Process – Per Engineering Run protocol FUSE – per SOP
Two concepts—impact and risk — are often discussed but sometimes conflated within quality systems. While related, these concepts serve distinct purposes and drive different decisions throughout the quality system. Let’s explore.
The Fundamental Difference: Impact vs. Risk
The difference between impact and risk is fundamental to effective quality management. The difference between impact and risk is critical. Impact is best thought of as ‘What do I need to do to make the change.’ Risk is ‘What could go wrong in making this change?'”
Impact assessment focuses on evaluating the effects of a proposed change on various elements such as documentation, equipment, processes, and training. It helps identify the scope and reach of a change. Risk assessment, by contrast, looks ahead to identify potential failures that might occur due to the change – it’s preventive and focused on possible consequences.
This distinction isn’t merely academic – it directly affects how we approach actions and decisions in our quality systems, impacting core functions of CAPA, Change Control and Management Review.
Aspect
Impact
Risk
Definition
The effect or influence a change, event, or deviation has on product quality, process, or system
The probability and severity of harm or failure occurring as a result of a change, event, or deviation
Focus
What is affected and to what extent (scope and magnitude of consequences)
What could go wrong, how likely it is to happen, and how severe the outcome could be
Assessment Type
Evaluates the direct consequences of an action or event
Evaluates the likelihood and severity of potential adverse outcomes
Typical Use
Used in change control to determine which documents, systems, or processes are impacted
Used to prioritize actions, allocate resources, and implement controls to minimize negative outcomes
Measurement
Usually described qualitatively (e.g., minor, moderate, major, critical)
Often quantified by combining probability and impact scores to assign a risk level (e.g., low, medium, high)
Example
A change in raw material supplier impacts the manufacturing process and documentation.
The risk is that the new supplier’s material could fail to meet quality standards, leading to product defects.
Change Control: Different Questions, Different Purposes
Within change management, the PIC/S Recommendation PI 054-1 notes that “In some cases, especially for simple and minor/low risk changes, an impact assessment is sufficient to document the risk-based rationale for a change without the use of more formal risk assessment tools or approaches.”
Impact Assessment in Change Control
Determines what documentation requires updating
Identifies affected systems, equipment, and processes
Identifies potential failures that could result from the change
Evaluates possible consequences to product quality and patient safety
Determines likelihood of those consequences occurring
Guides preventive measures
A common mistake is conflating these concepts or shortcutting one assessment. For example, companies often rush to designate changes as “like-for-like” without supporting data, effectively bypassing proper risk assessment. This highlights why maintaining the distinction is crucial.
Impact in validation relates to identifying what aspects of product quality could be affected by a system or process. For example, when qualifying manufacturing equipment, we determine which critical quality attributes (CQAs) might be influenced by the equipment’s performance.
Risk assessment in validation explores what could go wrong with the equipment or process that might lead to quality failures. Risk management plays a pivotal role in validation by enabling a risk-based approach to defining validation strategies, ensuring regulatory compliance, mitigating product quality and safety risks, facilitating continuous improvement, and promoting cross-functional collaboration.
In Design Qualification, we verify that the critical aspects (CAs) and critical design elements (CDEs) necessary to control risks identified during the quality risk assessment (QRA) are present in the design. This illustrates how impact assessment (identifying critical aspects) works together with risk assessment (identifying what could go wrong).
When we perform Design Review and Design Qualification, we focus on Critical Aspects: Prioritize design elements that directly impact product quality and patient safety. Here, impact assessment identifies critical aspects, while risk assessment helps prioritize based on potential consequences.
Following Design Qualification, Verification activities such as Installation Qualification (IQ), Operational Qualification (OQ), and Performance Qualification (PQ) serve to confirm that the system or equipment performs as intended under actual operating conditions. Here, impact assessment identifies the specific parameters and functions that must be verified to ensure no critical quality attributes are compromised. Simultaneously, risk assessment guides the selection and extent of tests by focusing on areas with the highest potential for failure or deviation. This dual approach ensures that verification not only confirms the intended impact of the design but also proactively mitigates risks before routine use.
Validation does not end with initial qualification. Continuous Validation involves ongoing monitoring and trending of process performance and product quality to confirm that the validated state is maintained over time. Impact assessment plays a role in identifying which parameters and quality attributes require ongoing scrutiny, while risk assessment helps prioritize monitoring efforts based on the likelihood and severity of potential deviations. This continuous cycle allows quality systems to detect emerging risks early and implement corrective actions promptly, reinforcing a proactive, risk-based culture that safeguards product quality throughout the product lifecycle.
Data Integrity: A Clear Example
Data integrity offers perhaps the clearest illustration of the impact-risk distinction.
As I’ve previously noted, Data quality is not a risk. It is a causal factor in the failure or severity. Poor data quality isn’t itself a risk; rather, it’s a factor that can influence the severity or likelihood of risks.
When assessing data integrity issues:
Impact assessment identifies what data is affected and which processes rely on that data
Risk assessment evaluates potential consequences of data integrity lapses
In my risk-based data integrity assessment methodology, I use a risk rating system that considers both impact and risk factors:
Risk Rating
Action
Mitigation
>25
High Risk-Potential Impact to Patient Safety or Product Quality
Mandatory
12-25
Moderate Risk-No Impact to Patient Safety or Product Quality but Potential Regulatory Risk
Recommended
<12
Negligible DI Risk
Not Required
This system integrates both impact (on patient safety or product quality) and risk (likelihood and detectability of issues) to guide mitigation decisions.
The Golden Day: Impact and Risk in Deviation Management
The Golden Day concept for deviation management provides an excellent practical example. Within the first 24 hours of discovering a deviation, we conduct:
An impact assessment to determine:
Which products, materials, or batches are affected
Potential effects on critical quality attributes
Possible regulatory implications
A risk assessment to evaluate:
Patient safety implications
Product quality impact
Compliance with registered specifications
Level of investigation required
This impact assessment is also the initial risk assessment, which will help guide the level of effort put into the deviation. This statement shows how the two concepts, while distinct, work together to inform quality decisions.
Quality Escalation: When Impact Triggers a Response
In quality escalation, we often use specific criteria based on both impact and risk:
Escalation Criteria
Examples of Quality Events for Escalation
Potential to adversely affect quality, safety, efficacy, performance or compliance of product
– Contamination – Product defect/deviation from process parameters or specification – Significant GMP deviations
Product counterfeiting, tampering, theft
– Product counterfeiting, tampering, theft reportable to Health Authority – Lost/stolen IMP
Product shortage likely to disrupt patient care
– Disruption of product supply due to product quality events
Potential to cause patient harm associated with a product quality event
These criteria demonstrate how we use both impact (what’s affected) and risk (potential consequences) to determine when issues require escalation.
Both Are Essential
Understanding the difference between impact and risk fundamentally changes how we approach quality management. Impact assessment without risk assessment may identify what’s affected but fails to prevent potential issues. Risk assessment without impact assessment might focus on theoretical problems without understanding the actual scope.
The pharmaceutical quality system requires both perspectives:
Impact tells us the scope – what’s affected
Risk tells us the consequences – what could go wrong
By maintaining this distinction and applying both concepts appropriately across change control, validation, and data integrity management, we build more robust quality systems that not only comply with regulations but actually protect product quality and patient safety.
As we celebrate International Workers’ Day this May 1st, it is an opportune moment to reflect on the profound connection between workers’ rights and effective quality management. The pursuit of quality cannot be separated from how we treat, empower, and respect the rights of those who create that quality daily. Today’s post examines this critical relationship, drawing from the principles I’ve advocated throughout my blog, and challenges us to reimagine quality management as fundamentally worker-centered.
The Historical Connection Between Workers’ Rights and Quality
International Workers’ Day commemorates the historic struggles and gains made by workers and the labor movement. This celebration reminds us that the evolution of quality management has paralleled the fight for workers’ rights. Quality is inherently a progressive endeavor, fundamentally anti-Taylorist in nature. Frederick Taylor’s scientific management approach reduced workers to interchangeable parts in a machine, stripping them of autonomy and creativity – precisely the opposite of what modern quality management demands.
The quality movement, from Deming onwards, has recognized that treating workers as mere cogs undermines the very foundations of quality. When we champion human rights and center those whose rights are challenged, we’re not engaging in politics separate from quality – we’re acknowledging the fundamental truth that quality cannot exist without empowered, respected workers.
Driving Out Fear: The Essential Quality Right
“No one can put in his best performance unless he feels secure,” wrote Deming thirty-five years ago. Yet today, fear remains ubiquitous in corporate culture, undermining the very quality we seek to create. As quality professionals, we must confront this reality at every opportunity.
Fear in the workplace manifests in multiple ways, each destructive to quality:
Source of Fear
Description
Impact on Quality
Competition
Managers often view anxiety generated by competition between co-workers as positive, encouraging competition for scarce resources, power, and status
Undermines collaboration necessary for system-wide quality improvements
Focus on finding fault rather than improving systems, often centered around the concept of “human error”
Discourages reporting of issues, driving quality problems underground
When workers operate in fear, quality inevitably suffers. They hide mistakes rather than report them, avoid innovation for fear of failure, and focus on protecting themselves rather than improving systems. Driving out fear isn’t just humane – it’s essential for quality.
Key Worker Rights in Quality Management
Quality management systems that respect workers’ rights create environments where quality can flourish. Based on workplace investigation principles, these rights extend naturally to all quality processes.
The Right to Information
In any quality system, clarity is essential. Workers have the right to understand quality requirements, the rationale behind procedures, and how their work contributes to the overall quality system. Transparency sets the stage for collaboration, where everyone works toward a common quality goal with full understanding.
The Right to Confidentiality and Non-Retaliation
Workers must feel safe reporting quality issues without fear of punishment. This means protecting their confidentiality when appropriate and establishing clear non-retaliation policies. One of the pillars of workplace equity is ensuring that employees are shielded from retaliation when they raise concerns, reinforcing a commitment to a culture where individuals can voice quality issues without fear.
The Right to Participation and Representation
The Who-What Matrix is a powerful tool to ensure the right people are involved in quality processes. By including a wider set of people, this approach creates trust, commitment, and a sense of procedural justice-all essential for quality success. Workers deserve representation in decisions that affect their ability to produce quality work.
Worker Empowerment: The Foundation of Quality Culture
Empowerment is not just a nice-to-have; it’s a foundational element of any true quality culture. When workers are entrusted with authority to make decisions, initiate actions, and take responsibility for outcomes, both job satisfaction and quality improve. Unfortunately, empowerment rhetoric is sometimes misused within quality frameworks like TQM, Lean, and Six Sigma to justify increased work demands rather than genuinely empowering workers.
The concept of empowerment has its roots in social movements, including civil rights and women’s rights, where it described the process of gaining autonomy and self-determination for marginalized groups. In quality management, this translates to giving workers real authority to improve processes and address quality issues.
Mary Parker Follett’s Approach to Quality Through Autonomy
Follett emphasized giving workers autonomy to complete their jobs effectively, believing that when workers have freedom, they become happier, more productive, and more engaged. Her “power with” principle suggests that power should be shared broadly rather than concentrated, fostering a collaborative environment where quality can thrive.
Rejecting the Great Man Fallacy
Quality regulations often fall into the trap of the “Great Man Fallacy” – the misguided notion that one person through education, experience, and authority can ensure product safety, efficacy, and quality. This approach is fundamentally flawed.
People only perform successfully when they operate within well-built systems. Process drives success by leveraging the right people at the right time making the right decisions with the right information. No single person can ensure quality, and thinking otherwise sets up both individuals and systems for failure.
Instead, we need to build processes that leverage teams, democratize decisions, and drive reliable results. This approach aligns perfectly with respecting workers’ rights and empowering them as quality partners rather than subjects of quality control.
Quality Management as a Program: Centering Workers’ Rights
Quality needs to be managed as a program, walking a delicate line between long-term goals, short-term objectives, and day-to-day operations. As quality professionals, we must integrate workers’ rights into this program approach.
The challenges facing quality today-from hyperautomation to shifting customer expectations-can only be addressed through worker empowerment. Consider how these challenges demand a worker-centered approach:
Challenge
Impact on Quality Management
Worker-Centered Approach
Advanced Analytics
Requires holistic data analysis and application
Develop talent strategies that upskill workers rather than replacing them
Hyper-Automation
Tasks previously done by humans being automated
Involve workers in automation decisions; focus on how automation can enhance rather than replace human work
Virtualization of Work
Rethinking how quality is executed in digital environments
Ensure workers have input on how virtual quality processes are designed
Need to adapt to changing risk levels in real-time
Enable employees to make faster decisions by building quality-informed judgment
Digitally Native Workforce
Changed expectations for how work is managed
Connect quality to values employees care about: autonomy, innovation, social issues
To meet these challenges, we must shift from viewing quality as a function to quality as an interdisciplinary, participatory process. We need to break down silos and build autonomy, encouraging personal buy-in through participatory quality management.
May Day as a Reminder of Our Quality Mission
As International Workers’ Day approaches, I’m reminded that our quality mission is inseparable from our commitment to workers’ rights. This May Day, I encourage all quality professionals to:
Evaluate how your quality systems either support or undermine workers’ rights
Identify and eliminate sources of fear in your quality processes
Create mechanisms for meaningful worker participation in quality decisions
Reject hierarchical quality models in favor of democratic, empowering approaches
Recognize that centering workers’ rights isn’t just ethical-it’s essential for quality
Quality management without respect for workers’ rights is not just morally questionable-it’s ineffective. The future of quality lies in approaches that are predictive, connected, flexible, and embedded. These can only be achieved when workers are treated as valued partners with protected rights and real authority.
This May Day, let’s renew our commitment to driving out fear, empowering workers, and building quality systems that respect the dignity and rights of every person who contributes to them. In doing so, we honor not just the historical struggles of workers, but also the true spirit of quality that puts people at its center.
What steps will you take this International Workers’ Day to strengthen the connection between workers’ rights and quality in your organization?
How you respond in the first 24 hours after discovering a deviation can make the difference between a minor quality issue and a major compliance problem. This critical window-what I call “The Golden Day”-represents your best opportunity to capture accurate information, contain potential risks, and set the stage for a successful investigation. When managed effectively, this initial day creates the foundation for identifying true root causes and implementing effective corrective actions that protect product quality and patient safety.
Why the First 24 Hours Matter: The Evidence
The initial response to a deviation is crucial for both regulatory compliance and effective problem-solving. Industry practice and regulatory expectations align on the importance of quick, systematic responses to deviations.
Regulatory expectations explicitly state that deviation investigation and root cause determination should be completed in a timely manner, and industry expectations usually align on deviations being completed within 30 days of discovery.
In the landmark U.S. v. Barr Laboratories case, “the Court declared that all failure investigations must be performed promptly, within thirty business days of the problem’s occurrence”
Best practices recommend assembling a cross-functional team immediately after deviation discovery and conduct initial risk assessment within 24 hours”
Initial actions taken in the first day directly impact the quality and effectiveness of the entire investigation process
When you capitalize on this golden window, you’re working with fresh memories, intact evidence, and the highest chance of observing actual conditions that contributed to the deviation.
Clear, precise problem definition forms the foundation of any effective investigation. Vague or incomplete problem statements lead to misdirected investigations and ultimately, inadequate corrective actions.
Document using specific, factual language that describes what occurred versus what was expected
Include all relevant details such as procedure and equipment numbers, product names and lot numbers
Apply the 5W2H method (What, When, Where, Who, Why if known, How much is involved, and How it was discovered)
Avoid speculation about causes in the initial description
Remember that the description should incorporate relevant records and photographs of discovered defects.
5W2H
Typical questions
Contains
Who?
Who are the people directly concerned with the problem? Who does this? Who should be involved but wasn’t? Was someone involved who shouldn’t be?
User IDs, Roles and Departments
What?
What happened?
Action, steps, description
When?
When did the problem occur?
Times, dates, place In process
Where?
Where did the problem occur?
Location
Why is it important?
Why did we do this? What are the requirements? What is the expected condition?
Justification, reason
How?
How did we discover. Where in the process was it?
Method, process, procedure
How Many? How Much?
How many things are involved? How often did the situation happen? How much did it impact?
Number, frequency
The quality of your deviation documentation begins with this initial identification. As I’ve emphasized in previous posts, the investigation/deviation report should tell a story that can be easily understood by all parties well after the event and the investigation. This narrative begins with clear identification on day one.
Elements
Problem Statement
Is used to…
Understand and target a problem. Providing a scope. Evaluate any risks. Make objective decisions
Answers the following… (5W2H)
What? (problem that occurred);When? (timing of what occurred); Where? (location of what occurred); Who? (persons involved/observers); Why? (why it matters, not why it occurred); How Much/Many? (volume or count); How Often? (First/only occurrence or multiple)
Contains…
Object (What was affected?); Defect (What went wrong?)
GEMBA-the actual place where work happens-is a cornerstone concept in quality management. When a deviation occurs, there is no substitute for being physically present at the location.
Observe the actual conditions and environment firsthand
Notice details that might not be captured in written reports
Understand the workflow and context surrounding the deviation
Gather physical evidence before it’s lost or conditions change
Create the opportunity for meaningful conversations with operators
Human error occurs because we are human beings. The extent of our knowledge, training, and skill has little to do with the mistakes we make. We tire, our minds wander and lose concentration, and we must navigate complex processes while satisfying competing goals and priorities – compliance, schedule adherence, efficiency, etc.
Foremost to understanding human performance is knowing that people do what makes sense to them given the available cues, tools, and focus of their attention at the time. Simply put, people come to work to do a good job – if it made sense for them to do what they did, it will make sense to others given similar conditions. The following factors significantly shape human performance and should be the focus of any human error investigation:
Physical Environment Environment, tools, procedures, process design
Organizational Culture Just- or blame-culture, attitude towards error
Management and Supervision Management of personnel, training, procedures
We do not want to see or experience human error – but when we do, it’s imperative to view it as a valuable opportunity to improve the system or process. This mindset is the heart of effective human error prevention.
Conducting an Effective GEMBA Walk for Deviations
When conducting your GEMBA walk specifically for deviation investigation:
Arrive with a clear purpose and structured approach
Observe before asking questions
Document observations with photos when appropriate
Look for environmental factors that might not appear in reports
Pay attention to equipment configuration and conditions
Note how operators interact with the process or equipment
A deviation gemba is a cross-functional team meeting that is assembled where a potential deviation event occurred. Going to the gemba and “freezing the scene” as close as possible to the time the event occurred will yield valuable clues about the environment that existed at the time – and fresher memories will provide higher quality interviews. This gemba has specific objectives:
Obtain a common understanding of the event: what happened, when and where it happened, who observed it, who was involved – all the facts surrounding the event. Is it a deviation?
Clearly describe actions taken, or that need to be taken, to contain impact from the event: product quarantine, physical or mechanical interventions, management or regulatory notifications, etc.
Interview involved operators: ask open-ended questions, like how the event unfolded or was discovered, from their perspective, or how the event could have been prevented, in their opinion – insights from personnel experienced with the process can prove invaluable during an investigation.
Deviation GEMBA Tips
Typically there is time between when notification of a deviation gemba goes out and when the team is scheduled to assemble. It is important to come prepared to help facilitate an efficient gemba:
Assemble procedures and other relevant documents and records. This will make references easier during the gemba.
Keep your team on-track – the gemba should end with the team having a common understanding of the event, actions taken to contain impact, and the agreed-upon next steps of the investigation.
You will gain plenty of investigational leads from your observations and interviews at the gemba – which documents to review, which personnel to interview, which equipment history to inspect, and more. The gemba is such an invaluable experience that, for many minor events, root cause and CAPA can be determined fairly easily from information gathered solely at the gemba.
Informal Rubric for Conducting a Good Deviation GEMBA
Describe the timeliness of the team gathering at the gemba.
Were all required roles and experts present?
Was someone leading or facilitating the gemba?
Describe any interviews the team performed during the gemba.
Did the team get sidetracked or off-topic during the gemba
Was the team prepared with relevant documentation or information?
Did the team determine batch impact and any reportability requirements?
Did the team satisfy the objectives of the gemba?
What did the team do well?
What could the team improve upon?
Speaking with Operators: The Power of Cognitive Interviewing
Interviewing personnel who were present when the deviation occurred requires special techniques to elicit accurate, complete information. Traditional questioning often fails to capture critical details.
Cognitive interviewing, as I outlined in my previous post on “Interviewing,” was originally created for law enforcement and later adopted during accident investigations by the National Transportation Safety Board (NTSB). This approach is based on two key principles:
Witnesses need time and encouragement to recall information
Retrieval cues enhance memory recall
How to Apply Cognitive Interviewing in Deviation Investigations
Mental Reinstatement: Encourage the interviewee to mentally recreate the environment and people involved
In-Depth Reporting: Encourage the reporting of all the details, even if it is minor or not directly related
Multiple Perspectives: Ask the interviewee to recall the event from others’ points of view
Several Orders: Ask the interviewee to recount the timeline in different ways. Beginning to end, end to beginning
Most importantly, conduct these interviews at the actual location where the deviation occurred. A key part of this is that retrieval cues access memory. This is why doing the interview on the scene (or Gemba) is so effective.
Component
What It Consists of
Mental Reinstatement
Encourage the interviewee to mentally recreate the environment and people involved.
In-Depth Reporting
Encourage the reporting of all the details.
Multiple Perspectives
Ask the interviewee to recall the event from others’ points of view.
Several Orders
Ask the interviewee to recount the timeline in different ways.
Approach the Interviewee Positively:
Ask for the interview.
State the purpose of the interview.
Tell interviewee why he/she was selected.
Avoid statements that imply blame.
Focus on the need to capture knowledge
Answer questions about the interview.
Acknowledge and respond to concerns.
Manage negative emotions.
Apply these Four Components:
Use mental reinstatement.
Report everything.
Change the perspective.
Change the order.
Apply these Two Principles:
Witnesses need time and encouragement to recall information.
Retrieval cues enhance memory recall.
Demonstrate these Skills:
Recreate the original context and had them walk you through process.
Tell the witness to actively generate information.
Adopt the witness’s perspective.
Listen actively, do not interrupt, and pause before asking follow-up questions.
Ask open-ended questions.
Encourage the witness to use imagery.
Perform interview at the Gemba.
Follow sequence of the four major components.
Bring support materials.
Establish a connection with the witness.
Do Not tell them how they made the mistake.
Initial Impact Assessment: Understanding the Scope
Within the first 24 hours, a preliminary impact assessment is essential for determining the scope of the deviation and the appropriate response.
Apply a risk-based approach to categorize the deviation as critical, major, or minor
Evaluate all potentially affected products, materials, or batches
Consider potential effects on critical quality attributes
Assess possible regulatory implications
Determine if released products may be affected
This impact assessment is also the initial risk assessment, which will help guide the level of effort put into the deviation.
Factors to Consider in Initial Risk Assessment
Patient safety implications
Product quality impact
Compliance with registered specifications
Potential for impact on other batches or products
Regulatory reporting requirements
Level of investigation required
This initial assessment will guide subsequent decisions about quarantine, notification requirements, and the depth of investigation needed. Remember, this is a preliminary assessment that will be refined as the investigation progresses.
Immediate Actions: Containing the Issue
Once you’ve identified the deviation and assessed its potential impact, immediate actions must be taken to contain the issue and prevent further risk.
Quarantine potentially affected products or materials to prevent their release or further use
Notify key stakeholders, including quality assurance, production supervision, and relevant department heads
Implement temporary corrective or containment measures
Document the deviation in your quality management system
Secure relevant evidence and documentation
Consider whether to stop related processes
Industry best practices emphasize that you should Report the deviation in real-time. Notify QA within 24 hours and hold the GEMBA. Remember that “if you don’t document it, it didn’t happen” – thorough documentation of both the deviation and your immediate response is essential.
Affected vs Related Batches
Not every Impact is the same, so it can be helpful to have two concepts: Affected and Related.
Affected Batch: Product directly impacted by the event at the time of discovery, for instance, the batch being manufactured or tested when the deviation occurred.
Related Batch: Product manufactured or tested under the same conditions or parameters using the process in which the deviation occurred and determined as part of the deviation investigation process to have no impact on product quality.
Setting Up for a Successful Full Investigation
The final step in the golden day is establishing the foundation for the comprehensive investigation that will follow.
Assemble a cross-functional investigation team with relevant expertise
Define clear roles and responsibilities for team members
Establish a timeline for the investigation (remembering the 30-day guideline)
Identify additional data or evidence that needs to be collected
Plan for any necessary testing or analysis
Schedule follow-up interviews or observations
In my post on handling deviations, I emphasized that you must perform a time-sensitive and thorough investigation within 30 days. The groundwork laid during the golden day will make this timeline achievable while maintaining investigation quality.
Planning for Root Cause Analysis
During this setup phase, you should also begin planning which root cause analysis tools might be most appropriate for your investigation. Select tools based on the event complexity and the number of potential root causes and when “human error” appears to be involved, prepare to dig deeper as this is rarely the true root cause
Identifying Phase of your Investigation
If
Then you are at
The problem is not understood. Boundaries have not been set. There could be more than one problem
Problem Understanding
Data needs to be collected. There are questions about frequency or occurrence. You have not had interviews
Data Collection
Data has been collected but not analyszed
Data Analysis
The root cause needs to be determined from the analyzed data
A picture of the separate steps of a process in sequential order, including:
materials or services entering or leaving the process (inputs and outputs)
decisions that must be made
people who become involved
time involved at each step, and/or
process measurements.
Critical Incident Technique (CIT)
A process used for collecting direct observations of human behavior that
have critical significance, and
meet methodically defined criteria.
Comparative Analysis
A technique that focuses a problem-solving team on a problem. It compares one or more elements of a problem or process to evaluate elements that are similar or different (e.g. comparing a standard process to a failing process).
Performance Matrix
A tool that describes the participation by various roles in completing tasks or deliverables for a project or business process. Note: It is especially useful in clarifying roles and responsibilities in cross-functional/departmental positions.
5W2H Analysis
An approach that defines a problem and its underlying contributing factors by systematically asking questions related to who, what, when, where, why, how, and how much/often.
Data Collection
Surveys
A technique for gathering data from a targeted audience based on a standard set of criteria.
Check Sheets
A technique to compile data or observations to detect and show trends/patterns.
Cognitive Interview
An interview technique used by investigators to help the interviewee recall specific memories from a specific event.
KNOT Chart
A data collection and classification tool to organize data based on what is
Known
Need to know
Opinion, and
Think we know.
Data Analysis
Pareto Chart
A technique that focuses efforts on problems offering the greatest potential for improvement.
Histogram
A tool that
summarizes data collected over a period of time, and
graphically presents frequency distribution.
Scatter Chart
A tool to study possible relationships between changes in two different sets of variables.
Run Chart
A tool that captures study data for trends/patterns over time.
Affinity Diagram
A technique for brainstorming and summarizing ideas into natural groupings to understand a problem.
Root Cause Analysis
Interrelationship Digraphs
A tool to identify, analyze, and classify cause and effect relationships among issues so that drivers become part of an effective solution.
Why-Why
A technique that allows one to explore the cause-and-effect relationships of a particular problem by asking why; drilling down through the underlying contributing causes to identify root cause.
Is/Is Not
A technique that guides the search for causes of a problem by isolating the who, what, when, where, and how of an event. It narrows the investigation to factors that have an impact and eliminates factors that do not have an impact. By comparing what the problem is with what the problem is not, we can see what is distinctive about a problem which leads to possible causes.
Structured Brainstorming
A technique to identify, explore, and display the
factors within each root cause category that may be affecting the problem/issue, and/or
effect being studied through this structured idea-generating tool.
Cause and Effect Diagram (Ishikawa/Fishbone)
A tool to display potential causes of an event based on root cause categories defined by structured brainstorming using this tool as a visual aid.
Causal Factor Charting
A tool to
analyze human factors and behaviors that contribute to errors, and
identify behavior-influencing factors and gaps.
Other Tools
Prioritization Matrix
A tool to systematically compare choices through applying and weighting criteria.
Control Chart
A tool to monitor process performance over time by studying its variation and source.
Process Capability
A tool to determine whether a process is capable of meeting requirements or specifications.
Making the Most of Your Golden Day
The first 24 hours after discovering a deviation represent a unique opportunity that should not be wasted. By following the structured approach outlined in this post-identifying the problem clearly, going to the GEMBA, interviewing operators using cognitive techniques, conducting an initial impact assessment, taking immediate containment actions, and setting up for the full investigation-you maximize the value of this golden day.
Remember that excellent deviation management is directly linked to product quality, patient safety, and regulatory compliance. Each well-managed deviation is an opportunity to strengthen your quality system.
I encourage you to assess your current approach to the first 24 hours of deviation management. Are you capturing the full value of this golden day, or are you letting critical information slip away? Implement these strategies, train your team on proper deviation triage, and transform your deviation response from reactive to proactive.
Your deviation management effectiveness doesn’t begin when the investigation report is initiated-it begins the moment a deviation is discovered. Make that golden day count.
Quality needs to be managed as a program, and as such, it must walk a delicate line between setting long-term goals, short-term goals, improvement priorities, and interacting with a suite of portfolios, programs, and KPIs. As quality professionals navigate increasingly complex regulatory landscapes, technological disruptions, and evolving customer expectations, the need for structured approaches to quality planning has never been greater.
At the heart of this activity, I use an x-matrix, a powerful tool at the intersection of strategic planning and quality management. The X-Matrix provides a comprehensive framework that clarifies the chaos, visually representing how long-term quality objectives cascade into actionable initiatives with clear ownership and metrics – connecting the dots between aspiration and execution in a single, coherent framework.
Understanding the X-Matrix: Structure and Purpose
The X-Matrix is a strategic planning tool from Hoshin Kanri methodology that brings together multiple dimensions of organizational strategy onto a single page. Named for its distinctive X-shaped pattern of relationships, this tool enables us to visualize connections between long-term breakthroughs, annual objectives, improvement priorities, and measurable targets – all while clarifying ownership and resource allocation.
The X-Matrix is structured around four key quadrants that create its distinctive shape:
South Quadrant (3-5 Year Breakthrough Objectives): These are the foundational, long-term quality goals that align with organizational vision and regulatory expectations. In quality contexts, these might include achieving specific quality maturity levels, establishing new quality paradigms, or fundamentally transforming quality systems.
West Quadrant (Annual Objectives): These represent the quality priorities for the coming year that contribute directly to the longer-term breakthroughs. These objectives are specific enough to be actionable within a one-year timeframe.
North Quadrant (Improvement Priorities): These are the specific initiatives, projects, and process improvements that will be undertaken to achieve the annual objectives. Each improvement priority should have clear ownership and resource allocation.
East Quadrant (Targets/Metrics): These are the measurable indicators that will be used to track progress toward both annual objectives and breakthrough goals. In quality planning, these often include process capability indices, deviation rates, right-first-time metrics, and other key performance indicators.
The power of the X-Matrix lies in the correlation points where these quadrants intersect. These intersections show how initiatives support objectives and how objectives align with long-term goals. They create a clear line of sight from strategic quality vision to daily operations and improvement activities.
Why the X-Matrix Excels for Quality Planning
Traditional quality planning approaches often suffer from disconnection between strategic objectives and tactical activities. Quality initiatives may be undertaken in isolation, with limited understanding of how they contribute to broader organizational goals. The X-Matrix addresses this fragmentation through its integrated approach to planning.
The X-Matrix provides visibility into the interdependencies within your quality system. By mapping the relationships between long-term quality objectives, annual goals, improvement priorities, and key metrics, quality leaders can identify potential resource conflicts, capability gaps, and opportunities for synergy.
Developing an X-Matrix necessitates cross-functional input and alignment to ensure that quality objectives are not isolated but integrated with operations, regulatory, supply chain, and other critical functions. The development of an X-Matrix encourages the back-and-forth dialogue necessary to develop realistic, aligned goals.
Perhaps most importantly for quality organizations, the X-Matrix provides the structure and rigor to ensure quality planning is not left to chance. As the FDA and other regulatory bodies increasingly emphasize Quality Management Maturity (QMM) as a framework for evaluating pharmaceutical operations, the disciplined approach embodied in the X-Matrix becomes a competitive advantage. The matrix systematically considers resource constraints, capability requirements, and performance measures – all essential components of mature quality systems.
Mapping Modern Quality Challenges to the X-Matrix
The quality landscape is evolving rapidly, with several key challenges that must be addressed in any comprehensive quality planning effort. The X-Matrix provides an ideal framework for addressing these challenges systematically. Building on the post “The Challenges Ahead for Quality” we can start to build our an X-matrix.
Advanced Analytics and Digital Transformation
As data sources multiply and processing capabilities expand, quality organizations face increased expectations for data-driven insights and decision-making. An effective X-Matrix for quality planning couldinclude:
3-5 Year Breakthrough: Establish a predictive quality monitoring system that leverages advanced analytics to identify potential quality issues before they manifest.
Annual Objectives: Implement data visualization tools for key quality metrics; establish data governance framework for GxP data; develop predictive models for critical quality attributes.
Improvement Priorities: Create cross-functional data science capability; implement automated data capture for batch records; develop real-time dashboards for process parameters.
Metrics: Percentage of quality decisions made with data-driven insights; predictive model accuracy; reduction in quality investigation cycle time through analytics.
Operational Stability in Complex Supply Networks
As pharmaceutical manufacturing becomes increasingly globalized with complex supplier networks, operational stability emerges as a critical challenge. Operational stability represents the state where manufacturing and quality processes exhibit consistent, predictable performance over time with minimal unexpected variation. The X-Matrix can address this through:
3-5 Year Breakthrough: Achieve Level 4 (Proactive) operational stability across all manufacturing sites, networks and key suppliers.
Annual Objectives: Implement statistical process control for critical processes; establish supplier quality alignment program; develop operational stability metrics and monitoring system.
Improvement Priorities: Deploy SPC training and tools; conduct operational stability risk assessments; implement regular supplier quality reviews; establish cross-functional stability team.
Metrics: Process capability indices (Cp, Cpk); right-first-time batch rates; deviation frequency and severity patterns; supplier quality performance.
Using the X-Matrix to Address Validation Challenges
Validation presents unique challenges in modern pharmaceutical operations, particularly as data systems become more complex and interconnected. Handling complex data types and relationships can be time-consuming and difficult, while managing validation rules across large datasets becomes increasingly costly and challenging. The X-Matrix offers a structured approach to addressing these validation challenges:
3-5 Year Breakthrough: Establish a risk-based, continuous validation paradigm that accommodates rapidly evolving systems while maintaining compliance.
Annual Objectives: Implement risk-based validation approach for all GxP systems; establish automated testing capabilities for critical applications; develop validation strategy for AI/ML applications.
Improvement Priorities: Train validation team on risk-based approaches; implement validation tool for automated test execution; develop validation templates for different system types; establish validation center of excellence.
Metrics: Validation cycle time reduction; percentage of validation activities conducted via automated testing; validation resource efficiency; validation effectiveness (post-implementation defects).
This X-Matrix approach to validation challenges ensures that validation activities are not merely compliance exercises but strategic initiatives that support broader quality objectives. By connecting validation priorities to annual objectives and long-term breakthroughs, organizations can justify the necessary investments and resources while maintaining a clear focus on business value.
Connecting X-Matrix Planning to Quality Maturity Models
The FDA’s Quality Management Maturity (QMM) model provides a framework for assessing an organization’s progression from reactive quality management to optimized, continuous improvement. This model aligns perfectly with the X-Matrix planning approach, as both emphasize systematic progression toward excellence.
The X-Matrix can be structured to support advancement through quality maturity levels by targeting specific capabilities associated with each level:
Maturity Level
X-Matrix Breakthrough Objective
Annual Objectives
Improvement Priorities
Reactive (Level 1)
Move from reactive to controlled quality operations
Continuous process verification; innovation workshops; supplier development program
Innovative (Level 5)
Maintain and leverage innovative quality capabilities
Establish industry leading practices; develop quality thought leadership; implement next-generation quality approaches
Quality research initiatives; external benchmarking; technology innovation pilots
This alignment between the X-Matrix and quality maturity models offers several advantages. First, it provides a clear roadmap for progression through maturity levels. Second, it helps organizations prioritize initiatives based on their current maturity level and desired trajectory. Finally, it creates a framework for measuring and communicating progress toward maturity goals.
Implementation Best Practices for Quality X-Matrix Planning
Implementing an X-Matrix approach to quality planning requires careful consideration of several key factors.
1. Start With Clear Strategic Quality Imperatives
The foundation of any effective X-Matrix is a clear set of strategic quality imperatives that align with broader organizational goals. These imperatives should be derived from:
Regulatory expectations and trends
Customer quality requirements
Competitive quality positioning
Organizational quality vision
These imperatives form the basis for the 3-5 year breakthrough objectives in the X-Matrix. Without this clarity, the remaining elements of the matrix will lack focus and alignment.
2. Leverage Cross-Functional Input
Quality does not exist in isolation; it intersects with every aspect of the organization. Effective X-Matrix planning requires input from operations, regulatory affairs, supply chain, R&D, and other functions. This cross-functional perspective ensures that quality objectives are realistic, supported by appropriate capabilities, and aligned with broader organizational priorities.
The catchball process from Hoshin Kanri provides an excellent framework for this cross-functional dialogue, allowing for iterative refinement of objectives, priorities, and metrics based on input from various stakeholders.
3. Focus on Critical Few Priorities
The power of the X-Matrix lies in its ability to focus organizational attention on the most critical priorities. Resist the temptation to include too many initiatives, objectives, or metrics. Instead, identify the vital few that will drive meaningful progress toward quality maturity and operational excellence.
This focus is particularly important in regulated environments where resource constraints are common and compliance demands can easily overwhelm improvement initiatives. A well-designed X-Matrix helps quality leaders maintain strategic focus amid the daily demands of compliance activities.
4. Establish Clear Ownership and Resource Allocation
The X-Matrix should clearly identify who is responsible for each improvement priority and what resources they will have available. This clarity is essential for execution and accountability. Without explicit ownership and resource allocation, even the most well-conceived quality initiatives may fail to deliver results.
The structure of the X-Matrix facilitates this clarity by explicitly mapping resources to initiatives and objectives. This mapping helps identify potential resource conflicts early and ensures that critical initiatives have the support they need.
Balancing Structure with Adaptability in Quality Planning
A potential criticism of highly structured planning approaches like the X-Matrix is that they may constrain adaptability and innovation. However, a well-designed X-Matrix actually enhances adaptability by providing a clear framework for evaluating and integrating new priorities. The structure of the matrix makes it apparent when new initiatives align with strategic objectives and when they represent potential distractions. This clarity helps quality leaders make informed decisions about where to focus limited resources when disruptions occur.
The key lies in building what might be called “bounded flexibility”—freedom to innovate within well-understood boundaries. By thoroughly understanding which process parameters truly impact critical quality attributes, organizations can focus stability efforts where they matter most while allowing flexibility elsewhere. The X-Matrix supports this balanced approach by clearly delineating strategic imperatives (where stability is essential) from tactical initiatives (where adaptation may be necessary).
Change management systems represent another critical mechanism for balancing stability with innovation. Well-designed change management ensures that innovations are implemented in a controlled manner that preserves operational stability. The X-Matrix can incorporate change management as a specific improvement priority, ensuring that the organization’s ability to adapt is explicitly addressed in quality planning.
The X-Matrix as the Engine of Quality Excellence
The X-Matrix represents a powerful approach to quality planning that addresses the complex challenges facing modern quality organizations. By providing a structured framework for aligning long-term quality objectives with annual goals, specific initiatives, and measurable targets, the X-Matrix helps quality leaders navigate complexity while maintaining strategic focus.
As regulatory bodies evolve toward Quality Management Maturity models, the systematic approach embodied in the X-Matrix will become increasingly valuable. Organizations that establish and maintain strong operational stability through structured planning will find themselves well-positioned for both compliance and competition in an increasingly demanding pharmaceutical landscape.
The journey toward quality excellence is not merely technical but cultural and organizational. It requires systematic approaches, appropriate metrics, and balanced objectives that recognize quality not as an end in itself but as a means to deliver value to patients, practitioners, and the business. The X-Matrix provides the framework needed to navigate this journey successfully, translating quality vision into tangible results that advance both organizational performance and patient outcomes.
By adopting the X-Matrix approach to quality planning, organizations can ensure that their quality initiatives are not isolated efforts but components of a coherent strategy that addresses current challenges while building the foundation for future excellence. In a world of increasing complexity and rising expectations, this structured yet flexible approach to quality planning may well be the difference between merely complying and truly excelling.
Investigations of a Dog 