Quality Review

What does a quality reviewer do?

Maintaining high-quality products is paramount, and a critical component of ensuring quality is implementing a robust review of work by a second or third person, a peer review, and/or quality review—also known as a work product review process. Like many tools, it can be underutilized. It also gets to the heart of the question of Quality Unit oversight.

Introduction to Work Product Review

Work product review systematically evaluates the output from various processes or tasks to ensure they meet predefined quality standards. This review is crucial in environments where the quality of the final product directly impacts safety and efficacy, such as in pharmaceutical manufacturing. Work product review aims to identify any deviations or defects early in the process, allowing for timely corrections and minimizing the risk of non-compliance with regulatory requirements.

Criteria for Work Product Review

To ensure that work product reviews are effective, several key criteria should be established:

Integration with Quality Management Systems: Integrate risk-based thinking into the quality management system to ensure that work product reviews are aligned with overall quality objectives. This involves regularly reviewing and updating risk assessments to reflect changes in processes or new information.
Clear Objectives: The review should have well-defined objectives that align with the process they exist within and regulatory requirements. For instance, in pharmaceutical manufacturing, these objectives might include ensuring that all documentation is accurate and complete and that manufacturing processes adhere to GMP standards.
Risk-Based: Apply work product reviews to areas identified as high-risk during the risk assessment. This ensures that resources are allocated efficiently, focusing on processes that have the greatest potential impact on quality.
Standardized Procedures: Standardized procedures should be established for conducting the review. These procedures should outline the steps involved, the reviewers’ roles and responsibilities, and the criteria for accepting or rejecting the work product.
Trained Reviewers: Reviewers should be adequately trained and competent in the subject matter. This means understanding not just the deliverable being reviewed but the regulatory framework it sits within and how it applies to the specific work products being reviewed in a GMP environment.
Documentation: All reviews should be thoroughly documented. This documentation should include the review’s results, any findings or issues identified, and actions taken to address these issues.
Feedback Loop: There should be a mechanism for feedback from the review process to improve future work products. This could involve revising procedures or providing additional training to personnel.

Bridging the Gap Between Work-as-Imagined, Work-as-Prescribed, and Work-as-Done

Work product review is a systematic process that evaluates the output from various tasks to ensure they meet predefined quality standards connecting to work-as-imagined, work-as-prescribed, and work-as-done. Work product review serves as a bridge between these concepts by systematically evaluating the output of work processes. Here’s how it connects:

Alignment with Work-as-Prescribed: Work product review ensures that outputs comply with established standards and procedures (work-as-prescribed), helping to maintain regulatory compliance and quality standards.
Insight into Work-as-Done: Through the review process, organizations gain insight into how work is actually being performed (work-as-done). This helps identify any deviations from prescribed procedures and allows for adjustments to improve alignment between work-as-prescribed and work-as-done.
Closing the Gap with Work-as-Imagined: By documenting and addressing discrepancies between work-as-imagined and work-as-done, work product review facilitates communication and feedback that can refine policies and procedures. This helps to bring work-as-imagined closer to the realities of work-as-done, improving the effectiveness of quality oversight.

Work product review is essential for ensuring that the quality of work outputs aligns with both prescribed standards and the realities of how work is actually performed. By bridging the gaps between work-as-imagined, work-as-prescribed, and work-as-done, organizations can enhance their quality management systems and maintain high standards of quality, safety and efficacy.

Aligning to the Role of Quality Unit Oversight

While work product review does not guarantee Quality Unit Oversight, it is a potential control to ensure this oversight.

In the pharmaceutical industry, the Quality Unit plays a pivotal role in ensuring drug products’ safety, efficacy, and quality. It oversees all quality-related aspects, from raw material selection to final product release. However, the Quality Unit must be enabled appropriately and structured within the organization to effectively exercise its authority and fulfill its responsibilities. This blog post explores what it means for a Quality Unit to have the necessary authority and how insufficient implementation of its responsibilities can impact pharmaceutical manufacturing.

Responsibilities of the Quality Unit

Establishing and Maintaining the Quality System: The Quality Unit must set up and continuously update the quality management system to ensure compliance with GxPs and industry best practices.

Auditing and Compliance: Conduct internal audits to ensure adherence to policies and procedures, and report quality system performance metrics.

Approving and Rejecting Components and Products: The Quality Unit has the authority to approve or reject components, drug products, and packaging materials based on quality standards.

Investigating Nonconformities: Ensuring thorough investigations into production errors, discrepancies, and complaints related to product quality.

Keeping Management Informed: Reporting on product, process, and system risks, as well as outcomes of regulatory inspections.

What It Means for a Quality Unit to Be Enabled

For a Quality Unit to be effectively enabled, it must have:

Independence: The Quality Unit should operate independently of production units to avoid conflicts of interest and ensure unbiased decision-making.
Authority: It must have the authority to approve or reject the work product without undue influence from other departments.
Resources: Adequate personnel are essential for conducting the quality unit functions.
Documentation and Procedures: Clear, documented procedures outlining responsibilities and processes are crucial for maintaining consistency and compliance.

Insufficient Implementation of Responsibilities

When a Quality Unit insufficiently implements its responsibilities, it can lead to significant issues, including:

Regulatory Noncompliance: Failure to adhere to GxPs and regulatory standards can result in regulatory action.
Product Quality Issues: Inadequate oversight can lead to the release of substandard products, posing risks to patient safety and public health.
Lack of Continuous Improvement: Without effective quality systems in place, opportunities for process improvements and innovation may be missed.

The Quality Unit is the backbone of pharmaceutical manufacturing, ensuring that products meet the highest standards of quality and safety. By understanding the Quality Unit’s responsibilities and ensuring it has the necessary authority and resources, pharmaceutical companies can maintain compliance, protect public health, and foster a culture of continuous improvement. Inadequate implementation of these responsibilities can have severe consequences, emphasizing the importance of a well-structured and empowered Quality Unit.

By understanding these responsibilities, we can take a risk-based approach to applying quality review.

When to Apply Quality Review as Work Product Review

Work product review by Quality should be applied at critical stages to guarantee critical-to-quality attributes, including adherence to the regulations. This should be a risk-based approach. As such, it should be identified as controls in a living risks assessment and adjusted (add more, remove where unnecessary) as appropriate.

Closely scrutinize the responsibilities of the Quality Unit in the regulations to ensure all are met.

Best Practices in Quality Review

Rubrics are a great way to standardize quality reviews. If it is important enough to require a work review, it is important enough to standardize. The process owner should develop and maintain these rubrics with an appropriate group of stakeholder custodians. This is a key part of knowledge management. Having this cross-functional perspective on the output and what quality looks like is critical. This rubric should include:

Definition of prescribed work and the intended output that is being reviewed
Potential outcomes related to critical attributes, including definitions of technical accuracy
Methods and techniques used to generate the outcome
Operating experience and lessons learned
Risks, hazards, and user-centered design considerations
Requirements, standards, and code compliance
Planning, oversight, and acceptance testing
Input data and sources
Assumptions
Documentation required
Reviews and approvals required
Program or procedural obstacles to desired performance
Surprise situations, for example, unanticipated risk factors, schedule or scope changes, and organizational issues
Engineering human performance tool(s) applicable to activities being reviewed.

The rubric should have an assessment component, and that assessment should feed back into the originator’s qualified state.

Work product reviews must be early enough to allow feedback into the normal work for repetitive tasks. This should lead to gates in processes, quality-on-the-floor, or better-trained supervisors performing better and more effective reviews. This feedback should always be to the responsible person – the originator—and should be, wherever possible, face-to-face feedback to resolve the particular issues identified. This dialogue is critical.

Conclusion

Work product review is a powerful tool for enhancing quality oversight. By aligning this process with the responsibilities of the Quality Unit and implementing best practices such as standardized rubrics and a risk-based approach, companies can ensure that their products meet the highest standards of quality and safety. Effective work product review not only supports regulatory compliance but also fosters a culture of continuous improvement, which is essential for maintaining excellence in the pharmaceutical industry.

Review of Process/Procedure

Review of documents are a critical part of the document management lifecycle.

In the post Process/Procedure Lifecycle there are some fundamental stakeholders:

The Process Owner defines the process, including people, process steps, and technology, as well as the connections to other processes. They are accountable for change management, training, monitoring and control of the process and supporting procedure. The Process Owners owns the continuous improvement of the overall process.
Quality is ultimately responsible for the decisions made and that they align, at a minimum, with all regulatory requirements and internal standards.
Functional Area Management represents the areas that have responsibilities in the process and has a vested interest or concern in the ongoing performance of a process. This can include stakeholders who are process owners in upstream or downstream processes.
A Subject Matter Expert (SME) is typically an expert on a narrow division of a process, such as a specific tool, system, or set of process steps. A process may have multiple subject matter experts associated with it, each with varying degrees of understanding of the over-arching process.

A Risk Based Approach

The level of review of a new or revised process/procedure is guided by three fundamental risk questions:

What might go wrong with the associated process? (risk identification)
What is the likelihood that this will go wrong? (risk analysis)
What are the consequences? How severe are they if this goes wrong? (risk analysis)

Conducting risk identification is real about understanding how complicated and complex the associated process is. This looks at the following criteria:

Interconnectedness: the organization and interaction of system components and other processes
Repeatability: the amount of variance in the process
Information content: the amount of information needed to interact with the process

What Happens During a Review of Process and Procedure

The review of a process/procedure ensures that the proposed changes add value to the process and attain the outcome the organization wants. There are three levels of review (which can and often do happen simultaneously):

Functional review
Expert review by subject matter experts
Step-by-step real-world challenge

Functional review is the vetting of the process/procedure. Process stakeholders, including functional area management affected by the change has the opportunity to review the draft, suggest changes and agree to move forward.

Functional review supplies the lowest degree of assurance. This review looks for potential impact of the change on the function – usually focused on responsibilities – but does not necessarily assures a critical review.

In the case of expert review, the SMEs will review the draft for both positive and negative elements. On the positive side, they will look for the best practices, value-adding steps, flexibility in light of changing demands, scalability in light of changing output targets, etc. On the negative side, they will look for bottlenecks in the process, duplication of effort, unnecessary tasks, non-value-adding steps, role ambiguities (i.e. several positions responsible for a task, or no one responsible for a task), etc.

Expert review provides a higher degree of assurance because it is a compilation of expert opinion and it is focused on the technical content of the procedure.

The real-world challenge tests the process/procedure’s applicability by challenging it step-by-step in as much as possible the actual conditions of use. Tis involves selecting seasoned employee(s) within the scope of the draft procedure – not necessarily a SME – and comparing the steps as drafted with the actual activities. It is important to ascertain if they align. It is equally important to consider evidence of resistance, repetition and human factor problems.

Sometimes it can be more appropriate to do the real-world test as a tabletop or simulation exercise.

As sufficient reviews are obtained, the comments received are incorporated, as appropriate. Significant changes incorporated during the review process may require the procedure be re-routed for review, and may require the need to add additional reviews.

Repeat as a iterative process as necessary.

The process/procedure lifecycle can be seen as the iterative design lifecycle.

Design Thinking: Determine process needs.

Collect and document business requirements
Map current-state processes.
Observe and interview process workers.
Design process to-be.

Startup: Create process documentation, workflows, and support materials. Review and described above

Continuous Improvement: Use the process; Collect, analyze, and report; Improve

2019 State of the Blog

I started this blog as an exercise in deliberate practice, as well as reflective. In order to grow it is important to engage in critical reflection, which requires a process of mutual learning, a consciously organised process of deliberative and distributed reflection. Which is what I strive to do in my blog posts.

At the end of last year, I evaluated my blog goals through an ACORN exercise, as well as updating a SWOT. These stand up pretty well, even in a year of changes where I took on member leader responsibilities as the chair of the ASQ’s Team and Workplace Excellence Forum and took a new job.

I met my posting goal, which was 1.5 posts a week, with 81 posts and 33.5k words.

The top 5 posts of 2019 are:

FDA signals – no such thing as a planned deviation: Written in 2018 this post directs a lot of traffic to the blog from search engines, and has the largest geographic spread. Key message here continues to be all temporary changes, all planned departures, need to go through a change control system of appropriate rigor based on the risk involved.
Risk Based Data Integrity Assessment: Data Integrity and Risk Management are two of my favorite topics and in this post I combine the two and provide a fairly usable tool. I wrote this post while at the ASQ’s Audit Conference, where I presented on data integrity.
Lessons Learned as a Developing Leader: I am very gratified that this piece of introspection was viewed as many times as it was. Three months into my current job and this post, and the followup, are a good roadmap.
Decision Quality: How we make decisions, deal with subjectivity and uncertainty and problem-solve are all big concerns for our organizations. This post serves as a good anchor for my thought and practice, as well as the direction of future endeavors.
Driving for Mature Quality Organizations – FDA recent perspective: Building a quality culture, driving maturity in our organizations are critical. The FDA is spot-on, and companies really need to be coming to grips and dealing with this systematically.

Looking ahead to 2020 for the blog, I am going to take a bit of direction from Luigi Sille who set the following goals for himself:

Build up my expertise
Grow my network
Continuously improve every single day

For building expertise, I want to continue to focus on building tools and methods to: deal with subjectivity and uncertainty around decision making and risk management; proactively build a culture of quality and excellence, especially dealing with aspects of data integrity; and, find connections between the larger organizational/leadership/operational bodies of work and adapt them to the quality profession.

This blog is a large part of growing my network and I want to get to 2 blog posts a week consistently. I’ll continue to work with the ASQ as chair of the Team and Workplace Excellence Forum, including holding at least 2 events (including an unconference!). I am also trying to pull together a group of speakers to bring data integrity and quality culture as a stream to ASQ BosCon. I’ll speak at least 2 ASQ Conferences. I’ll also deepen some ties with the PDA, including speaking at one conference.

As I continuously work to improve, I will bring the topics I’m learning and implementing back to this blog.

Four Types of Problems by Art Smalley

Four Types of Problems: from reactive troubleshooting to creative innovation by Art Smalley (2018, Lean Enterprise Institute, Inc. ) is a fascinating book focusing on dividing problem solving and innovation into four major types:

Troubleshooting: A reactive process of rapidly fixing abnormal conditions by returning things to immediately known standards. While beneficial in the immediate term this approach often fails to solve the problem’s root cause.
Gap-from-standard: A structured problem-solving process that aims more at the root cause through problem definition, goal setting, analysis, countermeasure implementation, checks, standards, and follow-up activities.
Target-state: Continuous improvement that goes beyond existing levels of performance to achieve new and better standards or conditions.
Open-ended and Innovation: Unrestricted pursuit through creativity and synthesis of a vision or ideal condition that entail radical improvements and unexpected products, processes, systems, or value for the customer beyond current levels.

Art Smalley is a well known Lean expert, and this book definitely grows out of the wisdom and is a pretty good read. He shares
the strengths and weakness of each problem solving technique providing many points of introspection, such as the questions at the end of each chapter and excellent illustrations.

This book provides s a framework, a mental model, to effectively approach and assess a situation in order to seek and bring the appropriate kind of thinking to calmly, confidently address the problem at hand.

In many ways this book was my favorite quality book of 2018. I think it could serve as a valuable primer and I’m contemplating how to use it for internal training this year.