I have a little trick when reviewing a Quality Risk Management SOP. I go to the process/procedure map section, and if I see only the illustration from ICH Q9, I know I am looking at an organization that hasn’t actually thought about risk management.
A risk management process needs more than the methodology behind individual risk management (assess, control, review). It needs to include the following:
Risk Plan: How do you manage risk management holistically? Which systems/processes have living risk assessments? What are your planned reviews? What significant initiatives around quality risk management are included?
Risk Register: How do you manage your entire portfolio of risks? Link to quality management review.
Selection of tools, and even more importantly, development of tools.
Mechanisms and tools for risk treatment
Improvement strategy for the quality risk management program. How do we know if the program is working as intended?
How to define, select, and train risk owners
How to engage the appropriate stakeholders in the risk process
Too many quality risk management SOPs do not read like process or procedure. They read like a regurgitation of ICH Q9 or the ISO31000 documents. Neither is a good thing. You must go deeper and create an executable process to govern the system.
To be honest, too often, we perform a risk assessment not to make decisions but to justify an already existing risk assessment. The risk assessment may help define a few additional action items and determine how rigorous to be about a few things. It actually didn’t make much of an impact on the already-decided path forward. This is some pretty bad risk management and decision-making.
For highly important decisions with high uncertainty or complexity, it is useful to consider the options/alternatives that exist and assess the benefits and risks of each before deciding on a path forward. Thoroughly identifying options/alternatives and assessing the benefits and risks of each can help the decision-making process and ultimately reduce risk.
An effective, highly structured decision-making process can help answer the question, ‘How can we compare the consequences of the various options before deciding?
The most challenging risk decisions are characterized by having several different, important things to consider in an environment where there are often multiple stakeholders and, often, multiple decision-makers.
In Multi-Criteria Decision-Making (MCDM), the primary objective is the structured consideration of the available alternatives (options) for achieving the objectives in order to make the most informed decision, leading to the best outcome.
In a Quality Risk Management context, the decision-making concerns making informed decisions in the face of uncertainty about risks related to the quality (and/or availability) of medicines.
Key Concepts of MCDM
Conflicting Criteria: MCDM deals with situations where criteria conflict. For example, when purchasing a car, one might need to balance cost, comfort, safety, and fuel economy, which often do not align perfectly.
Explicit Evaluation: Unlike intuitive decision-making, MCDM involves a structured approach to explicitly evaluate multiple criteria, which is crucial when the stakes are high, such as deciding whether to build additional manufacturing capacity for a product under development.
Multiple-Criteria Evaluation Problems: These involve a finite number of alternatives known at the beginning. The goal is to find the best alternative or a set of good alternatives based on their performance across multiple criteria.
Multiple-Criteria Design Problems: In these problems, alternatives are not explicitly known and must be found by solving a mathematical model. The number of alternatives can be very large, often exponentially.
Preference Information: The methods used in MCDM often require preference information from decision-makers (DMs) to differentiate between solutions. This can be done at various stages of the decision-making process, such as prior articulation of preferences, which transforms the problem into a single-criterion problem.
MCDM focuses on risk and uncertainty by explicitly weighing criteria and trade-offs between them. Multi-criteria decision-making (MCDM) differs from traditional decision-making methods in several key ways:
Explicit Consideration of Multiple Criteria: Traditional decision-making often focuses on a single criterion like cost or profit. MCDM explicitly considers multiple criteria simultaneously, which may be conflicting, such as cost, quality, safety, and environmental impact[1]. This allows for a more comprehensive evaluation of alternatives.
Structured Approach: MCDM provides a structured framework for evaluating alternatives against multiple criteria rather than relying solely on intuition or experience. It involves techniques like weighting criteria, scoring alternatives, and aggregating scores to rank or choose the best option.
Transparency and Consistency: MCDM methods aim to make decision-making more transparent, consistent, and less susceptible to individual biases. The criteria, weights, and evaluation process are explicitly defined, allowing for better justification and reproducibility of decisions.
Quantitative Analysis: Many MCDM methods employ quantitative techniques, such as mathematical models, optimization algorithms, and decision support systems. This enables a more rigorous and analytical approach compared to traditional qualitative methods.
Handling Complexity: MCDM is particularly useful for complex decision problems involving many alternatives, conflicting objectives, and multiple stakeholders. Traditional methods may struggle to handle such complexity effectively.
Stakeholder Involvement: Some MCDM methods, like the Analytic Hierarchy Process (AHP), facilitate the involvement of multiple stakeholders and the incorporation of their preferences and judgments. This can lead to more inclusive and accepted decisions.
Trade-off Analysis: MCDM techniques often involve analyzing trade-offs between criteria, helping decision-makers understand the implications of prioritizing certain criteria over others. This can lead to more informed and balanced decisions.
While traditional decision-making methods rely heavily on experience, intuition, and qualitative assessments, MCDM provides a more structured, analytical, and comprehensive approach, particularly in complex situations with conflicting criteria.
Multi-Criteria Decision-Making (MCDM) is typically performed following these steps:
Define the Decision Problem: Clearly state the problem or decision to be made, identify the stakeholders involved, and determine the desired outcome or objective.
Establish Criteria: Identify the relevant criteria that will be used to evaluate the alternatives. These criteria should be measurable, independent, and aligned with the objectives. Involve stakeholders in selecting and validating the criteria.
Generate Alternatives: Develop a comprehensive list of potential alternatives or options that could solve the problem. Use techniques like brainstorming, benchmarking, or scenario analysis to generate diverse alternatives.
Gather Performance Data: Assess how each alternative performs against each criterion. This may involve quantitative data, expert judgments, or qualitative assessments.
Assign Criteria Weights: By assigning weights, determine each criterion’s relative importance or priority. This can be done through methods like pairwise comparisons, swing weighting, or direct rating. Stakeholder input is crucial here.
Apply MCDM Method: Choose an appropriate MCDM technique based on the problem’s nature and the available data. Some popular methods include: Analytic Hierarchy Process (AHP); Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS); ELimination and Choice Expressing REality (ELECTRE); Preference Ranking Organization METHod for Enrichment of Evaluations (PROMETHEE); and, Multi-Attribute Utility Theory (MAUT).
Evaluate and Rank Alternatives: Apply the chosen MCDM method to evaluate and rank the alternatives based on their performance against the weighted criteria. This may involve mathematical models, software tools, or decision support systems.
Sensitivity Analysis: Perform sensitivity analysis to assess the robustness of the results and understand how changes in criteria weights or performance scores might affect the ranking or choice of alternatives.
Make the Decision: Based on the MCDM analysis, select the most preferred alternative or develop an action plan based on the ranking of alternatives. Involve stakeholders in the final decision-making process.
Monitor and Review: Implement the chosen alternative and monitor its performance. Review the decision periodically, and if necessary, repeat the MCDM process to adapt to changing circumstances or new information.
MCDM is an iterative process; stakeholder involvement, transparency, and clear communication are crucial. Additionally, the specific steps and techniques may vary depending on the problem’s complexity, the data’s availability, and the decision-maker’s preferences.
MCDM Technique
Description
Application
Key Features
Analytic Hierarchy Process (AHP)
A structured technique for organizing and analyzing complex decisions, using mathematics and psychology.
Widely used in business, government, and healthcare for prioritizing and decision-making.
Pairwise comparisons, consistency checks, and hierarchical structuring of criteria and alternatives.
Technique for Order Preference by Similarity to Ideal Solution (TOPSIS)
Based on the concept that the chosen alternative should have the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution.
Frequently used in engineering, management, and human resource management for ranking and selection problems.
Compensatory aggregation, normalization of criteria, and calculation of geometric distances.
Elimination and Choice Expressing Reality (ELECTRE)
An outranking method that compares alternatives by considering both qualitative and quantitative criteria. It uses a pairwise comparison approach to eliminate less favorable alternatives.
Commonly used in project selection, resource allocation, and environmental management.
Use of concordance and discordance indices, handling of both qualitative and quantitative data, and ability to deal with incomplete rankings.
Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE)
An outranking method that uses preference functions to compare alternatives based on multiple criteria. It provides a complete ranking of alternatives.
Applied in various fields such as logistics, finance, and environmental management.
Preference functions, visual interactive modules (GAIA), and sensitivity analysis.
Multi-Attribute Utility Theory (MAUT)
Involves converting multiple criteria into a single utility function, which is then used to evaluate and rank alternatives. It takes into account the decision-maker’s risk preferences and uncertainties.
Used in complex decision-making scenarios involving risk and uncertainty, such as policy analysis and strategic planning.
Utility functions, probabilistic weights, and handling of uncertainty.
The revised ICH Q9 (R1) guideline shifts from “Risk Identification” to “Hazard Identification” to reflect a more precise approach to identifying potential sources of harm (hazards) rather than broadly identifying risks.
Alignment with Risk Assessment Definition: The term “Hazard Identification” is more consistent with the established definition of Risk Assessment, which involves identifying hazards and analyzing and evaluating the associated risks.
Clarity and Precision: By focusing on hazards, the guideline aims to improve the clarity and precision of the risk management process. This helps better understand and assess the potential harms associated with identified hazards, leading to more effective risk management.
Improved Perception and Assessment: The change is expected to enhance how hazards are perceived and assessed, making the risk management process more robust and scientifically grounded. This is particularly important for ensuring patient safety and product quality.
Consistency in Terminology: The revision aims to standardize the terminology used in quality risk management, reducing confusion and ensuring all stakeholders understand the terms and processes involved.
ICH Q9 (r1) Figure 1: Overview of a typical quality risk management process
This small change in terminology can lead to better risk-based decisions by highlighting the need to identify hazards and not risks during the first step of the risk assessment process to remove any distractions about risks that may interfere with the hazard identification activity. When a Risk Assessment team focuses only on identifying hazards, they do not have to think about any related probabilities of occurrence – they only have to consider the potential hazards concerning the risk question under consideration. This is also the case of the severity of harm during hazard identification. There is no need to work to estimate the severity of the harm that may be presented by a hazard that comes later after the hazards have been identified.
ASTM E2500, the Standard Guide for Specification, Design, and Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment, is intended to “satisfy international regulatory expectations in ensuring that manufacturing systems and equipment are fit for the intended use and to satisfy requirements for design, installation, operation, and performance.”
The ASTM E2500 approach is a comprehensive framework for specification setting, design, and verification of pharmaceutical and biopharmaceutical manufacturing systems and equipment. It emphasizes a risk- and science-based methodology to ensure that systems are fit for their intended use, ultimately aiming to enhance product quality and patient safety.
Despite its 17-year history, it is fair to say it is not the best-implemented standard. There are still many unrealized opportunities and some major challenges. I don’t think a single organization I’ve been in has fully aligned, and ASTM E2500 can feel aspirational.
Key Principles
Risk Management: The approach integrates risk management principles from ICH Q8, Q9, and Q10, focusing on identifying and mitigating risks to product quality and patient safety throughout the lifecycle of the manufacturing system.
Good Engineering Practices (GEP): It incorporates GEP to ensure systems are correctly designed, installed, and operated.
Flexibility and Efficiency: It strives for a more flexible and efficient organization of verification activities that can be adapted to each project’s specific context.
Regulatory agencies expect drugmakers to persuade them that we know our processes and that our facilities, equipment, systems, utilities, and procedures have been established based on concrete data and a thorough risk assessment. The ASTM E2500 standard provides a means of demonstrating that all of these factors have been validated in consideration of carefully evaluated risks.
What the Standard Calls for
Four Main Steps
Requirements: Define the system’s needs and critical aspects. Subject Matter Experts (SMEs) play a crucial role in this phase by defining needs, identifying critical aspects, and developing the verification strategy.
Specification & Design: Develop detailed specifications and design the system to meet the requirements. This step involves thorough design reviews and risk assessments to ensure the system functions as intended.
Verification: Conduct verification activities to confirm that the system meets all specified requirements. This step replaces the traditional FAT/SAT/IQ/OQ/PQ sequence with a more streamlined verification process that can be tailored to the project’s needs.
Acceptance & Release: Finalize the verification process and release the system for operational use. This step includes the final review and approval of all verification activities and documentation.
Four Cross-Functional Processes
Good Engineering Practices (GEP): Ensure all engineering activities adhere to industry standards and best practices.
Quality Risk Management: Continuously assess and manage risks to product quality and patient safety throughout the project.
Design Review: Regularly reviews the system design to ensure it meets all requirements and addresses identified risks.
Change Management: Implement a structured process for managing system changes to ensure that all modifications are appropriately evaluated and documented.
Applications and Benefits
Applicability: The ASTM E2500 approach can be applied to new and existing manufacturing systems, including laboratory, information, and medical device manufacturing systems.
Lifecycle Coverage: It applies throughout the manufacturing system’s lifecycle, from concept to retirement.
Regulatory Compliance: The approach is designed to conform with FDA, EU, and other international regulations, ensuring that systems are qualified and meet all regulatory expectations.
Efficiency and Cost Management: By focusing on critical aspects and leveraging risk management tools, the ASTM E2500 approach can streamline project execution, reduce time to market, and optimize resource utilization.
The ASTM E2500 approach provides a structured, risk-based framework for specifying, designing, and verifying pharmaceutical and biopharmaceutical manufacturing systems. It emphasizes flexibility, efficiency, and regulatory compliance, making it a valuable tool for ensuring product quality and patient safety.
What Makes it Different?
ASTM E2500
The more traditional approach
Testing Approach
It emphasizes a risk-based approach, focusing on identifying and managing risks to product quality and patient safety throughout the manufacturing system’s lifecycle. This approach allows for flexibility in organizing verification activities based on the specific context and critical aspects of the system.
Typically follows a prescriptive sequence of tests (FAT, SAT, IQ, OQ, PQ) as outlined in guidelines like EU GMP Annex 15. This method is more rigid and less adaptable to the specific needs and risks of each project.
Verification vs Qualification
The term “verification” encompasses all testing activities, which can be organized more freely and rationally to optimize efficiency. Verification activities are tailored to the project’s needs and focus on critical aspects.
Follows a structured qualification process (Installation Qualification, Operational Qualification, Performance Qualification) with predefined steps and documentation requirements.
Role of Subject Matter Experts
SMEs play a crucial role from the start of the project, contributing to the definition of needs, identification of critical aspects, system design review, and development of the verification strategy. They are involved throughout the project lifecycle.
SMEs are typically involved at specific points in the project lifecycle, primarily during the qualification phases, and may not have as continuous a role as in the ASTM E2500 approach.
Integration of Good Engineering Practices
Offers greater flexibility in organizing verification activities, allowing for a more efficient and streamlined process. This can lead to reduced time to market and optimized resource utilization.
While GEP is also important, the focus is more on the qualification steps rather than integrating GEP throughout the entire project lifecycle.
Change Management
It emphasizes early and continuous change management, starting from the supplier’s site, to avoid test duplication and ensure that changes are properly evaluated and documented.
It emphasizes early and continuous change management, starting from the supplier’s site, to avoid test duplication and ensure that changes are properly evaluated and documented.
Documentation
Documentation is focused on risk management and verification activities, ensuring compliance with international regulations (FDA, EU, ICH Q8, Q9, Q10). The approach is designed to meet regulatory expectations while allowing for flexibility in documentation.
quires extensive documentation for each qualification step, which can be more cumbersome and less adaptable to specific project needs.
Opinion
I’m watching to see what the upcoming update to Annex 15 will do to address the difficulties some see between an ATSM E2500 approach and the European regulations. I also hope we will see an update to ISPE Baseline® Guide Volume 5: Commissioning and Qualification to align an approach.
ISPE Baseline® Guide Volume 5
ATSM E2500
Design inputs Impact assessment Design Qualification Commissioning Multiple trial runs to get things right IQ, OQ, PQ, and acceptance criteria GEP Scope and QA Scope overlapped Focused on Documentation Deliverables Change Management
Design inputs Design Review Risk Mitigation Critical Control Parameters define Acceptance Criteria Verification Testing Performance Testing GEP Scope and QA Scope have a clear boundary Process, Product Quality and Patient Safety Quality by Design, Design Space, and Continuous Improvement
To be honest I don’t think ATSM E2500, ISPE Guide 5, or anything else has the balance just right. And your program ends up being a triangulation between these and the regulations. And don’t even bring in trying to align GAMP5 or USP <1058> or…or…or…
And yes, I do consider this part of my 3-year plan. I look forward to the challenges of a culture shift, increased SME involvement, formalization of GEPs (and teaching engineers how to write), effective change management, timely risk assessments, and comprehensive implementation planning.
Risk management plays a pivotal role in validation by enabling a risk-based approach to defining validation strategies, ensuring regulatory compliance, mitigating product quality and safety risks, facilitating continuous improvement, and promoting cross-functional collaboration. Integrating risk management principles into the validation lifecycle is essential for maintaining control and consistently producing high-quality products in regulated industries such as biotech and medical devices.
We will conduct various risk assessments in our process lifecycle—many ad hoc (static) and a few living (dynamic). Understanding how they fit together in a larger activity set is crucial.
In the Facility, Utilities, Systems, and Equipment (FUSE) space, we are taking the process understanding, translating it into a design, and then performing Design Qualification (DQ) to verify that the critical aspects (CAs) and critical design elements (CDEs) necessary to control risks identified during the quality risk assessment (QRA) are present in the design. This helps mitigate risks to product quality and patient safety. To do this, we need to properly understand the process. Unfortunately, we often start with design before understanding the process and then need to go back and perform rework. Too often I see a dFMEA ignored or as an input to the pFMEA instead of working together in a full risk management cycle.
The Preliminary Hazard Analysis (PHA) supports a pFMEA, which supports a dFMEA, which supports the pFMEA (which also benefits at this stage from a HAACP). Tools fit together to provide the approach. Tools do not become the approach.
Design and Process FMEAs
DFMEA (Design Failure Mode and Effects Analysis) and PFMEA (Process Failure Mode and Effects Analysis) are both methodologies used within the broader FMEA framework to identify and mitigate potential failures. Still, they focus on different aspects of development and manufacturing.
DFMEA
PFMEA
Scope and Focus
Primarily scrutinizes design to preempt flaws.
Focuses on processes to ensure effectiveness, efficiency and reliability.
Stakeholder Involvement
Engages design-oriented teams like engineering, quality engineers, and reliability engineers.
Involves operation-centric personnel such as manufacturing, quality control, quality operations, and process engineers.
Inputs and Outputs
Relies on design requirements, product specs, and component interactions to craft a robust product.
Utilizes process steps, equipment capabilities, and parameters to design a stable operational process.
Stages in lifecycle
Conducted early in development, concurrent with the design phase, it aids in early issue detection and minimizes design impact.
Executed in production planning post-finalized design, ensuring optimized operations prior to full-scale production.
Updated When
Executed in production planning post-finalized design, ensuring optimized operations before full-scale production.
The design qualification phase is especially suitable for determining risks for products and patients stemming from the equipment or machine. These risks should be identified during the design qualification and reflected by appropriate measures in the draft design so that the operator can effectively eliminate, adequately control, and monitor or observe them. To identify design defects (mechanical) or in the creation of systems (electronics) on time and to eliminate them at a low cost, it is advisable to perform the following risk analysis activities for systems, equipment, or processes:
Categorize the GMP criticality and identify the critical quality attributes and process parameters;
Categorize the requirements regarding the patient impact and product impact (for example, in the form of a trace matrix);
Identify critical functions and system elements (e.g., the definition of a calibration concept and preventive maintenance);
Investigate functions for defect recognition. This includes checking alarms and fault indications, operator error, etc. The result of this risk analysis may be the definition of further maintenance activities, a different assessment of a measurement point, or the identification of topics to include in the operating manuals or procedures.
Additional risk analyses for verifying the design may include usability studies using equipment mock-ups or preliminary production trials (engineering studies) regarding selected topics to prove the feasibility of specific design aspects (e.g., interaction between machine and materials).
Too often, we misunderstand risk assessments and start doing them at the most granular level. This approach allows us to right-size our risk assessments and holistically look at the entire lifecycle.
Investigations of a Dog 