The Lack of Objectivity in Quality Management

ICH Q9(r1) can be reviewed as a revision that addresses long-standing issues of subjectivity in risk management. Subjectivity is a widespread problem throughout the quality sphere, posing significant challenges because it introduces personal biases, emotions, and opinions into decision-making processes that should ideally be driven by objective data and facts.

Inconsistent Decision-Making: Subjective decision-making can lead to inconsistencies because different individuals may have varying opinions and biases. This inconsistency can result in unpredictable outcomes and make it challenging to establish standardized processes. For example, one manager might prioritize customer satisfaction based on personal experiences, while another might focus on cost-cutting, leading to conflicting strategies within the same organization.
Bias and Emotional Influence: Subjectivity often involves emotional influence, which can cloud judgment and lead to decisions not in the organization’s best interest. For instance, a business owner might make decisions based on a personal attachment to a product or service rather than its market performance or profitability. This emotional bias can prevent the business from making necessary changes or investments, ultimately harming its growth and sustainability.
Risk Management Issues: In risk assessments, subjectivity can significantly impact the identification and evaluation of risks. Subjective assessments may overlook critical risks or overemphasize less significant ones, leading to inadequate risk management strategies. Objective, data-driven risk assessments are essential to accurately identify and mitigate potential threats to the business. See ICHQ9(r1).
Difficulty in Measuring Performance: Subjective criteria are often more complicated to quantify and measure, making it challenging to track performance and progress accurately. Objective metrics, such as key performance indicators (KPIs), provide clear, measurable data that can be used to assess the effectiveness of business processes and make informed decisions.
Potential for Misalignment: Subjective decision-making can lead to misalignment between business goals and outcomes. For example, if subjective opinions drive project management decisions, the project may deviate from its original scope, timeline, or budget, resulting in unmet objectives and dissatisfied stakeholders.
Impact on Team Dynamics: Subjectivity can also affect team dynamics and morale. Decisions perceived as biased or unfair can lead to dissatisfaction and conflict among team members. Objective decision-making, based on transparent criteria and data, helps build trust and ensures that all team members are aligned with the business’s goals.

Every organization I’ve been in has a huge problem with subjectivity, and I’m confident in asserting none of us are doing enough to deal with the lack of objectivity, and we mostly rely on our intuition instead of on objective guidelines that will create unambiguous, holistic, and
universally usable models.

Understand the Decisions We Make

Every day, we make many decisions, sometimes without even noticing it. These decisions fall into four categories:

Acceptances: It is a binary choice between accepting or rejecting;
Choices: Opting for a subset from a group of alternatives;
Constructions: Creating an ideal solution given accessible resources;
Evaluations: Here, commitments back up the statements of worth to act

These decisions can be simple or complex, with manifold criteria and several perspectives. Decision-making is the process of choosing an option among manifold alternatives.

The Fallacy of Expert Immunity is a Major Source of Subjectivity

There is a widely incorrect belief that experts are impartial and immune to biases. However, the truth is that no one is immune to bias, not even experts. In many ways, experts are more susceptible to certain biases. The very making of expertise creates and underpins many of the biases. For example, experience and training make experts engage in more selective attention, use chunking and schemas (typical activities and their sequence), and rely on heuristics and expectations arising from past base rate experiences, utilizing a whole range of top-down cognitive processes that create a priori assumptions and expectations.

These cognitive processes often enable experts to make quick and accurate decisions. However, these mechanisms also create bias that can lead them in the wrong direction. Regardless of the utilities (and vulnerability) of such cognitive processing in experts, they do not make experts immune from bias, and indeed, expertise and experience may actually increase (or even cause) certain biases. Experts across domains are subject to cognitive vulnerabilities.

Even when experts are made aware of and acknowledge their biases, they nevertheless think they can overcome them by mere willpower. This is the illusion of control. Combating and countering these biases requires taking specific steps—willpower alone is inadequate to deal with the various manifestations of bias.

In fact, trying to deal with bias through the illusion of control may actually increase the bias due to “ironic processing” or “ironic rebound.” Hence, trying to minimize bias by willpower makes you think of it more and increases its effect. This is similar to a judge instructing jurors to disregard specific evidence. By doing so, the judge makes the jurors notice this evidence even more.

Such fallacies’ beliefs prevent dealing with biases because they dismiss their powers and existence. We need to acknowledge the impact of biases and understand their sources to take appropriate measures when needed and when possible to combat their effects.

Fallacy	Incorrect Belief
Ethical Issues	It only happens to corrupt and unscrupulous individuals, an issue of morals and personal integrity, a question of personal character.
Bad Apples	It only happens to corrupt and unscrupulous individuals. It is an issue of morals and personal integrity, a question of personal character.
Expert Immunity	Experts are impartial and are not affected because bias does not impact competent experts doing their job with integrity.
Technological Protection	Using technology, instrumentation, automation, or artificial intelligence guarantees protection from human biases.
Blind Spot	Other experts are affected by bias, but not me. I am not biased; it is the other experts who are biased.
Illusion of Control	I am aware that bias impacts me, and therefore, I can control and counter its affect. I can overcome bias by mere willpower.

Six Fallacies that Increase Subjectivity

Mitigating Subjectivity

There are four basic strategies to mitigate the impact of subjectivity.

Data-Driven Decision Making

Utilize data and analytics to inform decisions, reducing reliance on personal opinions and biases.

Establish clear metrics with key performance indicators (KPI), key behavior indicators (KBI), and key risk indicators (KRI) that are aligned with objectives.
Implement robust data collection and analysis systems to gather relevant, high-quality data.
Use data visualization tools to present information in an easily digestible format.
Train employees on data literacy and interpretation to ensure proper use of data insights.
Regularly review and update data sources to maintain relevance and accuracy.

Standardized Processes

Implement standardized processes and procedures to ensure consistency and fairness in decision-making.

Document and formalize decision-making procedures across the organization.
Create standardized templates, checklists, and rubrics for evaluating options and making decisions.
Implement a consistent review and approval process for major decisions.
Regularly audit and update standardized processes to ensure they remain effective and relevant.

Education, Training, and Awareness

Educate and train employees and managers on the importance of objective decision-making and recognizing and minimizing personal biases.

Conduct regular training sessions on cognitive biases and their impact on decision-making.
Provide resources and tools to help employees recognize and mitigate their own biases.
Encourage a culture of open discussion and constructive challenge to promote diverse perspectives.
Implement mentoring programs to share knowledge and best practices for objective decision-making.

Digital Tools

Leverage digital tools and software to automate and streamline processes, reducing the potential for subjective influence. The last two is still more aspiration than reality.

Implement workflow management tools to ensure consistent application of standardized processes.
Use collaboration platforms to facilitate transparent and inclusive decision-making processes.
Adopt decision support systems that use algorithms and machine learning to provide recommendations based on data analysis.
Leverage artificial intelligence and predictive analytics to identify patterns and trends that may not be apparent to human decision-makers.

The Mistake I See in Most Quality Risk Management SOPs

I have a little trick when reviewing a Quality Risk Management SOP. I go to the process/procedure map section, and if I see only the illustration from ICH Q9, I know I am looking at an organization that hasn’t actually thought about risk management.

A risk management process needs more than the methodology behind individual risk management (assess, control, review). It needs to include the following:

Risk Plan: How do you manage risk management holistically? Which systems/processes have living risk assessments? What are your planned reviews? What significant initiatives around quality risk management are included?
Risk Register: How do you manage your entire portfolio of risks? Link to quality management review.
Selection of tools, and even more importantly, development of tools.
Mechanisms and tools for risk treatment
Improvement strategy for the quality risk management program. How do we know if the program is working as intended?
How to define, select, and train risk owners
How to engage the appropriate stakeholders in the risk process

Too many quality risk management SOPs do not read like process or procedure. They read like a regurgitation of ICH Q9 or the ISO31000 documents. Neither is a good thing. You must go deeper and create an executable process to govern the system.

Global versus Local Process and Procedure and the eQMS

Companies both large and small grapple with how and when to create standard work at the global level, while still having the scalability to capture different GXP activity families and product modality.

I’ve discussed before on document hierarchy and on the leveling of process and procedure. It is really important to level your processes, and this architecture should be deliberate and shepherded.

This really gets to the heart of work-as-imagined and prescribed, and the concept of standard work.

Benefits of Standard Work

Ensures all work is done according to the current best practice
Consistency is the essential ingredient of quality
Allows organizations to scale rapidly
Puts the focus on the process and not an individual or team
Makes improvements easier and faster

Global versus Local Process and Procedure in the Document Hierarchy

Most Quality Hierarchies look fairly similar.

Excluding the Program level (which becomes even more important) we can expand the model in the process band to account for global versus local.

Global and local process within the document hierarchy

Quality Manual and Policy remains global with local input and determine the overall structure of the quality management system.

Global Process is created when a process is majority task and role driven at a global level. It is pan-GXP, pan-modality, pan-geography. It is the standard way of work to drive consistency across and through the organization.

Local Process is created when a process is specific to a specific GXP, product modality, geography.

Procedure, which describes the tasks, can be created off of local or global process. When the global process has localizations (a CAPA is a CAPA but how I build action items may differ across sites), I can build local versions off the global process.

For an example, Document and Record Management.

This approach takes real vision among leaders to drive for consistency and simplicity. This activity is a core component in good system design, no matter the size of the organization.

Principle	Description	Application for Global and Local Process
Balance	The system creates value for the multiple stakeholders. While the ideal is to develop a design that maximizes the value for all the key stakeholders, the designer often has to compromise and balance the needs of the various stakeholders.	The value of standard work really shines here.
Congruence	The degree to which the system components are aligned and consistent with each other and the other organizational systems, culture, plans, processes, information, resource decisions, and actions.	We gain congruence through ensuring key processes are at the global level.
Convenience	The system is designed to be as convenient as possible for the participants to implement (a.k.a. user friendly). System includes specific processes, procedures, and controls only when necessary.	The discussion around global versus local will often depend on how you define convenience
Coordination	System components are interconnected and harmonized with the other (internal and external) components, systems, plans, processes, information, and resource decisions toward common action or effort. This is beyond congruence and is achieved when the individual components of a system operate as a fully interconnected unit.	How we ensure coordination across and through an organization.
Elegance	Complexity vs. benefit — the system includes only enough complexity as is necessary to meet the stakeholder’s needs. In other words, keep the design as simple as possible and no more while delivering the desired benefits. It often requires looking at the system in new ways.	Keep this in mind as global for the sake of global is not always the right decision.
Human	Participants in the system are able to find joy, purpose and meaning in their work.	Never forget
Learning	Knowledge management, with opportunities for reflection and learning (learning loops), is designed into the system. Reflection and learning are built into the system at key points to encourage single- and double-loop learning from experience to improve future implementation and to systematically evaluate the design of the system itself.	Building the right knowledge management into the organization is critical to leverage this model
Sustainability	The system effectively meets the near- and long-term needs of the current stakeholders without compromising the ability of future generations of stakeholders to meet their own needs.	Ensure the appropriate tools exist to sustain, including regulatory intelligence. Long-term scalability.

Pillars of Good System Design for Gloval and Local Process

Utilizing the eQMS to drive

The ideal state when implementing (or improving) an eQMS is to establish global processes and allow system functionality to localize as appropriate.

So for example, every CAPA is the same (identify problem and root cause, create plan, implement plan, prove implementation is effective. This is a global process. However, one wants specific task detail at a lower level, for example GMP sites may care about certain fields more the GCP, medical device has specific needs, etc. These local task level needs can be mainted within one workflow.

The Key is Fit-For-Purpose Fit-for-Use

A fit for purpose process meets the requirements of the organization.

A fit for use process is usable throughout the lifecycle.

Global and localizing processes is a key part of making both happen.

Design Problem Solving into the Process

Good processes and systems have ways designed into them to identify when a problem occurs, and ensure it gets the right rigor of problem-solving. A model like Art Smalley’s can be helpful here.

Each and every process should go through the following steps:

Define those problems that should be escalated and those that should not. Everyone working in a process should have the same definition of what is a problem. Often times we end up with a hierarchy of issues that are solved within the process – Level 1 – and those processes that go to a root cause process (deviation/CAPA) – level 2.
Identify the ways to notice a problem. Make the work as visual as possible so it is easier to detect the problem.
Define the escalation method. There should be one clear way to surface a problem. There are many ways to create a signal, but it should be simple, timely, and very clear.

These three elements make up the request for help.

The next two steps make up the response to that request.

Who is the right person to respond? Supervisor? Area management? Process Owner? Quality?
How does the individual respond, and most importantly when? This should be standardized so the other end of that help chain is not wondering whether, when, and in what form that help is going to arrive.

In order for this to work, it is important to identify clear ownership of the problem. There always must be one person clearly accountable, even if only responsible for bits, so they can push the problem forward.

It is easy for problem-solving to stall. So make sure progress is transparent. Knowing what is being worked on, and what is not, is critical.

Prioritization is key. Not every problem needs solving so have a mechanism to ensure the right problems are being solved in the process.