The Lack of Objectivity in Quality Management

ICH Q9(r1) can be reviewed as a revision that addresses long-standing issues of subjectivity in risk management. Subjectivity is a widespread problem throughout the quality sphere, posing significant challenges because it introduces personal biases, emotions, and opinions into decision-making processes that should ideally be driven by objective data and facts.

Inconsistent Decision-Making: Subjective decision-making can lead to inconsistencies because different individuals may have varying opinions and biases. This inconsistency can result in unpredictable outcomes and make it challenging to establish standardized processes. For example, one manager might prioritize customer satisfaction based on personal experiences, while another might focus on cost-cutting, leading to conflicting strategies within the same organization.
Bias and Emotional Influence: Subjectivity often involves emotional influence, which can cloud judgment and lead to decisions not in the organization’s best interest. For instance, a business owner might make decisions based on a personal attachment to a product or service rather than its market performance or profitability. This emotional bias can prevent the business from making necessary changes or investments, ultimately harming its growth and sustainability.
Risk Management Issues: In risk assessments, subjectivity can significantly impact the identification and evaluation of risks. Subjective assessments may overlook critical risks or overemphasize less significant ones, leading to inadequate risk management strategies. Objective, data-driven risk assessments are essential to accurately identify and mitigate potential threats to the business. See ICHQ9(r1).
Difficulty in Measuring Performance: Subjective criteria are often more complicated to quantify and measure, making it challenging to track performance and progress accurately. Objective metrics, such as key performance indicators (KPIs), provide clear, measurable data that can be used to assess the effectiveness of business processes and make informed decisions.
Potential for Misalignment: Subjective decision-making can lead to misalignment between business goals and outcomes. For example, if subjective opinions drive project management decisions, the project may deviate from its original scope, timeline, or budget, resulting in unmet objectives and dissatisfied stakeholders.
Impact on Team Dynamics: Subjectivity can also affect team dynamics and morale. Decisions perceived as biased or unfair can lead to dissatisfaction and conflict among team members. Objective decision-making, based on transparent criteria and data, helps build trust and ensures that all team members are aligned with the business’s goals.

Every organization I’ve been in has a huge problem with subjectivity, and I’m confident in asserting none of us are doing enough to deal with the lack of objectivity, and we mostly rely on our intuition instead of on objective guidelines that will create unambiguous, holistic, and
universally usable models.

Understand the Decisions We Make

Every day, we make many decisions, sometimes without even noticing it. These decisions fall into four categories:

Acceptances: It is a binary choice between accepting or rejecting;
Choices: Opting for a subset from a group of alternatives;
Constructions: Creating an ideal solution given accessible resources;
Evaluations: Here, commitments back up the statements of worth to act

These decisions can be simple or complex, with manifold criteria and several perspectives. Decision-making is the process of choosing an option among manifold alternatives.

The Fallacy of Expert Immunity is a Major Source of Subjectivity

There is a widely incorrect belief that experts are impartial and immune to biases. However, the truth is that no one is immune to bias, not even experts. In many ways, experts are more susceptible to certain biases. The very making of expertise creates and underpins many of the biases. For example, experience and training make experts engage in more selective attention, use chunking and schemas (typical activities and their sequence), and rely on heuristics and expectations arising from past base rate experiences, utilizing a whole range of top-down cognitive processes that create a priori assumptions and expectations.

These cognitive processes often enable experts to make quick and accurate decisions. However, these mechanisms also create bias that can lead them in the wrong direction. Regardless of the utilities (and vulnerability) of such cognitive processing in experts, they do not make experts immune from bias, and indeed, expertise and experience may actually increase (or even cause) certain biases. Experts across domains are subject to cognitive vulnerabilities.

Even when experts are made aware of and acknowledge their biases, they nevertheless think they can overcome them by mere willpower. This is the illusion of control. Combating and countering these biases requires taking specific steps—willpower alone is inadequate to deal with the various manifestations of bias.

In fact, trying to deal with bias through the illusion of control may actually increase the bias due to “ironic processing” or “ironic rebound.” Hence, trying to minimize bias by willpower makes you think of it more and increases its effect. This is similar to a judge instructing jurors to disregard specific evidence. By doing so, the judge makes the jurors notice this evidence even more.

Such fallacies’ beliefs prevent dealing with biases because they dismiss their powers and existence. We need to acknowledge the impact of biases and understand their sources to take appropriate measures when needed and when possible to combat their effects.

Fallacy	Incorrect Belief
Ethical Issues	It only happens to corrupt and unscrupulous individuals, an issue of morals and personal integrity, a question of personal character.
Bad Apples	It only happens to corrupt and unscrupulous individuals. It is an issue of morals and personal integrity, a question of personal character.
Expert Immunity	Experts are impartial and are not affected because bias does not impact competent experts doing their job with integrity.
Technological Protection	Using technology, instrumentation, automation, or artificial intelligence guarantees protection from human biases.
Blind Spot	Other experts are affected by bias, but not me. I am not biased; it is the other experts who are biased.
Illusion of Control	I am aware that bias impacts me, and therefore, I can control and counter its affect. I can overcome bias by mere willpower.

Six Fallacies that Increase Subjectivity

Mitigating Subjectivity

There are four basic strategies to mitigate the impact of subjectivity.

Data-Driven Decision Making

Utilize data and analytics to inform decisions, reducing reliance on personal opinions and biases.

Establish clear metrics with key performance indicators (KPI), key behavior indicators (KBI), and key risk indicators (KRI) that are aligned with objectives.
Implement robust data collection and analysis systems to gather relevant, high-quality data.
Use data visualization tools to present information in an easily digestible format.
Train employees on data literacy and interpretation to ensure proper use of data insights.
Regularly review and update data sources to maintain relevance and accuracy.

Standardized Processes

Implement standardized processes and procedures to ensure consistency and fairness in decision-making.

Document and formalize decision-making procedures across the organization.
Create standardized templates, checklists, and rubrics for evaluating options and making decisions.
Implement a consistent review and approval process for major decisions.
Regularly audit and update standardized processes to ensure they remain effective and relevant.

Education, Training, and Awareness

Educate and train employees and managers on the importance of objective decision-making and recognizing and minimizing personal biases.

Conduct regular training sessions on cognitive biases and their impact on decision-making.
Provide resources and tools to help employees recognize and mitigate their own biases.
Encourage a culture of open discussion and constructive challenge to promote diverse perspectives.
Implement mentoring programs to share knowledge and best practices for objective decision-making.

Digital Tools

Leverage digital tools and software to automate and streamline processes, reducing the potential for subjective influence. The last two is still more aspiration than reality.

Implement workflow management tools to ensure consistent application of standardized processes.
Use collaboration platforms to facilitate transparent and inclusive decision-making processes.
Adopt decision support systems that use algorithms and machine learning to provide recommendations based on data analysis.
Leverage artificial intelligence and predictive analytics to identify patterns and trends that may not be apparent to human decision-makers.

Acountable People

We tend to jumble forms of accountability in an organization, often confusing between a people manager and a technical manager. I think its very important to differentiate between the two.

People managers deal with human resources and team dynamics, while technical managers deal with managing design, execution, and improvement. They can be the same person, but we need to recognize the differences and resource appropriately. Too often we blur the two roles and as a result neither is done well.

I’ve talked on this blog about a few of the technical manager types: Process Owners, the ASTM E2500 SME/Molecule Steward, and Knowledge Owners. There are certainly others out there. In the table below I added two more for comparison:

a qualified person from OSHA, because I think this is a great generic look at the concept
The EU Qualifed Person. Industry relevant and one that often gets confused in execution.

Aspect	Qualified Person (OSHA Definition)	Qualified Person (EU)	Knowledge Owner	ASTM E2500 SME	Process Owner
Primary Focus	Ensuring compliance with safety standards and solving technical problems	Certifying that each batch of a medicinal product meets all required provisions	Managing and maintaining knowledge within a specific domain	Ensuring manufacturing systems meet quality and safety standards	Managing and optimizing a specific business process
Key Responsibilities	Solve or resolve problems related to the subject matter, work, or project	Certify batches meet GMP and regulatory standards	Maintain and update knowledge base	Define system needs and identify critical aspects	Define process goals, purpose, and KPIs
	Design and install systems to improve safety	Ensure compliance with market authorization requirements	Validate and broadcast new knowledge	Develop and execute verification strategies	Communicate with key players and stakeholders
	Ensure compliance with laws and standards	Oversee quality control and assurance processes	Provide training and support	Review system designs and manage risks	Analyze process performance and identify improvements
	May not have the authority to stop work	Conduct audits and inspections	Monitor and update knowledge assets	Lead quality risk management efforts	Ensure process compliance with regulations and standards
Skills Required	Technical expertise in the area	Degree in pharmacy, biology, chemistry, or related field	Subject matter expertise in specific knowledge domain	Technical understanding of manufacturing systems and equipment	Leadership and communication skills
	Certification, degree, or other professional recognition	Several years of experience in pharmaceutical manufacturing	Analytical and validation skills	Risk management and verification skills	Analytical and problem-solving skills
	Ability to solve technical problems	Registered with the competent authority in the EU member state	Training and support skills	Continuous improvement and change management skills	Ability to define and monitor KPIs
Authority	Authority to design and install safety systems	Authority to certify batches and ensure compliance	Authority over knowledge management processes and content	Authority to define and verify critical aspects of systems	Authority to make decisions and implement changes in the process
Interaction with Others	Collaborates with production and quality control teams	Works with quality control, assurance, and regulatory teams	Works with various departments to ensure knowledge is shared and utilized	Collaborates with project stakeholders and engineering teams	Communicates with project leaders, process users, and other stakeholders
Examples of Activities	Reviewing batch documentation and certifying products	Certifying each batch of medicinal products before release	Validating new knowledge submissions	Conducting quality risk analyses and verification tests	Defining process objectives and mission statements
	Ensuring compliance with GMP and regulatory standards	Ensuring compliance with GMP and regulatory standards	Providing training on knowledge management systems	Reviewing system designs and managing changes	Monitoring process performance and compliance
	Overseeing investigations related to quality issues	Overseeing quality control and assurance processes	Updating and maintaining knowledge databases	Leading continuous improvement efforts	Identifying and implementing process improvements
Industry Context	Primarily in construction, manufacturing, and safety-critical industries	Pharmaceutical and biotechnology industries within the EU	Applicable across various industries, especially information-heavy sectors	Primarily in pharmaceutical and biotechnology industries	Applicable in any industry with defined business processes

Comparison table

Qualified Person (OSHA Definition): Focuses on ensuring compliance with safety standards and solving technical problems. They possess technical expertise and professional recognition and are responsible for designing and installing safety systems.
Qualified Person (EU): Ensures that each batch of medicinal products meets all required provisions before release. They are responsible for compliance with GMP and regulatory standards and must be registered with the competent authority in the EU member state.
Knowledge Owner: Manages and disseminates knowledge within an organization. They ensure that knowledge is accurate, up-to-date, and accessible, and they provide training and support to facilitate knowledge sharing.
ASTM E2500 SME: Ensures that manufacturing systems meet quality and safety standards. They define system needs, develop verification strategies, manage risks, and lead continuous improvement efforts.
Process Owner: Manages and optimizes specific business processes. They define process goals, monitor performance, ensure compliance with standards, and implement improvements to enhance efficiency and effectiveness.

Common Themes

Subject Matter Expertise

All roles require a high level of subject matter expertise in their respective domains, whether it’s technical knowledge, regulatory compliance, manufacturing processes, or business processes.
This expertise is typically gained through formal education, certifications, extensive training, and practical experience.

Ensuring Compliance and Quality

A key responsibility across these roles is ensuring compliance with relevant laws, regulations, standards, and quality requirements.

Risk Identification and Management

These roles are all responsible for identifying potential risks, hazards, or process inefficiencies.
They are expected to develop and implement strategies to mitigate or eliminate these risks, ensuring the safety of operations and the quality of products or processes.

Continuous Improvement and Change Management

They are involved in continuous improvement efforts, identifying areas for optimization and implementing changes to enhance efficiency, quality, and knowledge sharing.
They are responsible for managing change processes, ensuring smooth transitions, and minimizing disruptions.

Authority and Decision-Making

Most of these roles have a certain level of authority and decision-making power within their respective domains.

Collaboration and Knowledge Sharing

Effective collaboration and knowledge sharing are essential for these roles to succeed.

While these roles have distinct responsibilities and focus areas, they share common goals of ensuring compliance, managing risks, driving continuous improvement, and leveraging subject matter expertise to achieve organizational objectives and maintain high standards of quality and safety. They are more similar than dissimilar and should be looked at holistically within the organization.

Build Your Knowledge Base

Engaging with knowledge and Knowledge Management are critical parts of development. The ability to navigate the flood of available data to find accurate information is tied directly to individuals’ existing knowledge and their skills at distinguishing credible information from misleading content.

There is ample evidence that many individuals lack the ability to accurately judge their understanding or the quality and accuracy of their performance (i.e., calibration). To truly develop our knowledge, we need to be engaged in deliberative practice. But to truly calibrate requires feedback, guidance, and coaching that you may not have access to within our organizations. This requires effort and deliberate building of a system and processes.

Information can be found with little mental effort but without critical analysis of its legitimacy or validity, the ease of information can actually work against the development of deeper-processing strategies. It is really easy to go-online and get an answer, but unless learners put themselves in positions to struggle cognitively with an issue, and unless they have occasions to transform or reframe problems, their likelihood of progressing into competence is jeopardized.

Creative teams

The more learners forge principled knowledge in a professional domain, the greater their reported interest in and identity with that field. Therefore, without the active pursuit of knowledge, these individuals’ interest in professional development may wane and their progress toward expertise may stall. This is why I find professional societies so critical, and why I am always pushing people to step up.

My constant goal as a mentor is to help people do the following:

Refuse to be lulled into accepting a role as passive consumers of information, striving instead to be active producers of knowledge
Probe and critically analyze the information they encounter, rather
than accepting quick, simple answers
Forge a meaningful interest in the profession and personal connections to members
of professional communities, instead of relying on moment-by-moment stimulation and superficial relationships

Competence

If we are going to step up to the challenges ahead of us, to address the skill gaps we are seeing, we each need to be deliberate in how we develop and deliberate in how we build our organizations to support development.

Expert Intuition and Risk Management

Saturday Morning Breakfast Cereal source http://smbc-comics.com/comic/horrible

Risk management is a crucial aspect of any organization or project. However, it is often subject to human errors in subjective risk judgments. This is because most risk assessment methods rely on subjective inputs from experts. Without certain precautions, experts can make consistent errors in judgment about uncertainty and risk.

There are methods that can correct the systemic errors that people make, but very few organizations implement them. As a result, there is often an almost universal understatement of risk. We need to keep in mind a few rules about experience and expertise.

Experience is a nonrandom, nonscientific sample of events throughout our lifetime.
Experience is memory-based and we are very selective regarding what we choose to remember,
What we conclude from our experience can be full of logical errors
Unless we get reliable feedback on past decisions, there is no reason to believe our experience will tell us much.

No matter how much experience we accumulate, we seem to be very inconsistent in its application.

Experts have unconscious heuristics and biases that impact their judgment, some important ones include:

Misconceptions of chance: If you flip a coin six times, which result is more likely (H= heads, T= tails): HHHTTT or HTHTTH? They are both equal, but many people assume that because the first series looks “less random” than the second, it must be less likely. This is an example of representativeness bias. We appear to judge odds based on what we assume to be representative scenarios. Human beings easily confuse patterns and randomness.
The conjunction fallacy: We often see specific events as more likely than broader categories of events.
Irrational belief in small samples
Disregarding variance in small samples. Small samples have more random variance that large samples is considered less than it should be.
Insensitivity to prior probabilities: People tend to ignore the past and focus on new information when making subjective estimates.

This is all about overconfidence as an expert, which will consistently underestimate risks.

What are some ways to overcome this? I recommend the following be built into your risk management system.

Pretend you are in the future looking back at failure. Start with the assumption that a major disaster did happen and describe how it happened.
Look to risks from others. Gather a list of related failures, for example, regulatory agency observations, and think of risks in relation to those.
Include Everyone. Your organization has numerous experts on all sorts of specific risks. Make the effort to survey representatives of just about every job level.
Do peer reviews. Check assumptions by showing them to peers who are not immersed in the assessment.
Implement metrics for performance. The Brier score is a way to evaluate the result of predictions both by how often the team was right and by the probability the estimated for getting a correct answer.

Communities of Practice

Knowledge management is a key enabler for quality, and should firmly be part of our standards of practice and competencies. There is a host of practices, and one tool that should be in our toolboxes as quality professionals is the Community of Practice (COP).

What is a Community of Practice?

Wenger, Trayner, and de Laat (2011) defined a Community of Practice as a “learning partnership among people who find it useful to learn from and with each other about a particular domain. They use each other’s experience of practice as a learning resource.” Etienne Wagner is the theoretical origin of the idea of a Community of Practice, as well as a great deal of the subsequent development of the concept.

Communities of practice are groups of people who share a passion for something that they know how to do, and who interact regularly in order to learn how to do it better. As such, they are a great tool for continuous improvement.

These communities can be defined by disciplines, by problems, or by situations. They can be internal or external. A group of deviation investigators who want to perform better investigations, contamination control experts sharing across sites, the list is probably endless for whenever there is a shared problem to be solved.

The idea is to enable practitioners to manage knowledge. Practitioners have a special connection with each other because they share actual experiences. They understand each other’s stories, difficulties, and insights. This allows them to learn from each other and build on each other’s expertise.

There are three fundamental characteristics of communities:

Domain: the area of knowledge that brings the community together, gives it its identity, and defines the key issues that members need to address. A community of practice is not just a personal network: it is about something. Its identity is defined not just by a task, as it would be for a team, but by an “area” of knowledge that needs to be explored and developed.
Community: the group of people for whom the domain is relevant, the quality of the relationships among members, and the definition of the boundary between the inside and the outside. A community of practice is not just a Web site or a library; it involves people who interact and who develop relationships that enable them to address problems and share knowledge.
Practice: the body of knowledge, methods, tools, stories, cases, documents, which members share and develop together. A community of practice is not merely a community of interest. It brings together practitioners who are involved in doing something. Over time, they accumulate practical knowledge in their domain, which makes a difference to their ability to act individually and collectively.

The combination of domain, community, and practice is what enables communities of practice to manage knowledge. Domain provides a common focus; community builds relationships that enable collective learning; and practice anchors the learning in what people do. Cultivating communities of practice requires paying attention to all three elements.

Communities of Practice are different than workgroups or project teams.

	What’s the purpose?	Who belongs?	What holds it together?	How long does it last?
Community of Practice	To develop members’ capabilities. To build and exchange knowledge	Members who share domain and community	Commitment from the organization. Identification with the group’s expertise. Passion	As long as there is interest in maintaining the group
Formal work group	To deliver a product or service	Everyone who reports to the group’s manager	Job requirements and common goals	Until the next reorganization
Project team	To accomplish a specific task	Employee’s assigned by management	The project’s milestones and goals	Until the project has been completed
Informal network	To collect and pass on business information	Friends and business acquantainces	Mutual needs	As long as people have a reason to connect

Types of organizing blocks

Establishing a Community of Practice

Sponsorship

For a Community of Practice to thrive it is crucial for the organization to provide adequate
sponsorship. Sponsorship are those leaders who sees that a community can deliver value and therefore makes sure that the community has the resources it needs to function and that its ideas and proposals find their way into the organization. While there is often one specific sponsor, it is more useful to think about the sponsorship structure that enables the communities to thrive and have an impact on the performance of the organization. This includes high-level executive sponsorship as well as the sponsorship of line managers who control the time usage of employees. The role of sponsorship includes:

Translating strategic imperatives into a knowledge-centric vision of the organization
Legitimizing the work of communities in terms of strategic priorities
Channeling appropriate resources to ensure sustained success
Giving a voice to the insights and proposals of communities so they affect the way business is conducted
Negotiating accountability between line operations and communities (e.g., who decides which “best practices” to adopt)

Support Structure

Communities of Practice need organizational support to function. This support includes:

A few explicit roles, some of which are recognized by the formal organization and resourced with dedicated time
Direct resources for the nurturing of the community infrastructure including meeting places, travel funds, and money for specific projects
Technological infrastructure that enables members to communicate regularly and to accumulate documents

It pays when you use communities of practice in a systematic way to put together a small “support team” of internal
consultants who provide logistic and process advice for communities, including coaching community leaders, educational activities to raise awareness and skills, facilitation services, communication with management, and
coordination across the various community of practices. But this is certainly not needed.

Process Owners and Communities of Practice go hand-in-hand. Often it is either the Process Owner in a governance or organizing role; or the community of practice is made up of process owners across the network.

Recognition Structure

Communities of Practice allows its participants to build reputation, a crucial asset in the knowledge economy. Such reputation building depends on both peer and organizational recognition.

Peer recognition: community-based feedback and acknowledgement mechanisms that celebrate community participation
Organizational recognition: rubric in performance appraisal for community contributions and career paths for people who take on community leadership

Tag: expertise