What does a quality reviewer do?

ICH Q10 states , “Leadership is essential to establish and maintain a company-wide commitment to quality and for the performance of the pharmaceutical quality system.” One of the central roles of the Quality reviewer/approver is to provide leadership, driving each event/record/etc, and the system as a whole, to continually improve. Quality does this in three ways:

Ensuring the process is followed
Validating the decisions made
Creating a good history

Ensuring the process is followed

The quality review provides a coaching/mentoring opportunity to build and/or enhance capabilities and behaviors and recognize and reinforce desired behaviors.

Questions to Ask

1. What is the process? What process and steps apply?

2. What is expected? Look for: Standard work, expected state, variation to the expected state

3. What is working well? Look for: process being followed, ideas being generated, lessons shared

4. What is not being followed? Look for: variation to procedural requirements, poor technical writing

Validating the decisions made

Quality is ultimately responsible for the decisions made. For each decision we do the following:

Identify alternatives. Once you have a clear understanding of the record from ensuring the process was followed, it’s time to identify the various options for each decision.
Weigh the evidence. In this step, you’ll need to “evaluate for feasibility, acceptability and desirability” to know which alternative is best. It may be helpful to seek out a trusted second opinion to gain a new perspective on the issue at hand.
Choose among alternatives. When it’s time to make your decision, be sure that you understand the risks involved with your chosen route. You may also choose a combination of alternatives now that you fully grasp all relevant information and potential risks.
Take action. Ensure the quality system/process reflects the action.
Review your decision. An often-overlooked but important step in the decision making process is evaluating your decision for effectiveness. Ask yourself what you did well and what can be improved next time.

Creating a good history

If it Isn’t Written Down, then it Didn’t Happen” is a guiding principle of the quality profession.

There are four major types of writing in quality: instructional, informational, persuasive and transactional. A well written event is both instructional and transactional.

Our quality systems record what happened, finalize recommendations and action plans, and to act as an archive. A well written report allows the reader to easily grasp the content and, if applicable, make informed decision. Report writing is a cornerstone of an Event/CAPA system (from incident identification to root cause through CAPA completion and effectiveness review), validation, risk management and so much more.

In short, reports are our stories, they form the narrative. And how we tell that narrative determines how we think of an issue, and how we will continue to think of it in the future.

Bystander Effect, Open Communication and Quality Culture

Our research suggests that the bystander effect can be real and strong in organizations, especially when problems linger out in the open to everyone’s knowledge.
Insiya Hussain and Subra Tangirala (January 2019) “Why Open Secrets Exist in Organizations” Harvard Business Review

The bystander effect occurs when the presence of others discourages an individual from intervening in an emergency situation. When individuals relinquish responsibility for addressing a problem, the potential negative outcomes are wide-ranging. While a great deal of the research focuses on helping victims, the overcoming the bystander effect is very relevant to building a quality culture.

The literature on this often follows after social psychologists John M. Darley and Bibb Latané who identified the concept in the late ’60s. They defined five characteristics bystanders go through:

Notice that something is going on
Interpret the situation as being an emergency
Degree of responsibility felt
Form of assistance
Implement the action choice

This is very similar to the 5 Cs of trouble-shooting: Concern (Notice), Cause (Interpret), Countermeasure (Form of Assistance and Implement), Check results.

What is critical here is that degree of responsibility felt. Without it we see people looking at a problem and shrugging, and then the problem goes on and on. It is also possible for people to just be so busy that the degree of responsibility is felt to the wrong aspect, such as “get the task done” or “do not slow down operations” and it leads to the wrong form of assistance – the wrong troubleshooting.

When building a quality culture, and making sure troubleshooting is an ingrained activity, it is important to work with employees so they understand that their voices are not redundant and that they need to share their opinions even if others have the same information. As the HBR article says: “If you see something, say something (even if others see the same thing).”

Building a quality culture is all about building norms which encourage detection of potential threats or problems and norms which encouraged improvements and innovation.

Lessons Learned and Change Management

One of the hallmarks of a quality culture is learning from our past experiences, to eliminate repeat mistakes and to reproduce success. The more times you do an activity, the more you learn, and the better you get (within limits for simple activities). Knowledge management is an enabler of quality systems, in part, to focus on learning and thus accelerate learning across the organization as a whole, and not just one person or a team.

This is where the” lessons learned” process comes in. There are a lot of definitions of lessons learned out there, but the definition I keep returning to is that a lessons learned is a change in personal or organizational behavior as a result from learning from experience. Ideally, this is a permanent, institutionalized change, and this is often where our quality systems can really drive continuous improvement.

Lessons learned is activity to lessons identified to updated processes — Lessons Learned

Part of Knowledge Management

The lessons learned process is an application of knowledge management.

Lessons identified is generate, assess, and share.

Updated processes (and documents) is contextualize, apply and update.

Lessons Learned in the Context of Knowledge Management

Identify Lessons Learned

Identifying lessons needs to be done regularly, the closer to actual change management and control activities the better. The formality of this exercise depends on the scale of the change. There are basically a few major forms:

After action reviews: held daily (or other regular cycle) for high intensity learning. Tends to be very focused on questions of the day.
Retrospective: Held at specific periods (for example project gates or change control status changes. Tends to have a specific focus on a single project.
Consistency discussions: Held periodically among a community of practice, such as quality reviewers or multiple site process owners. This form looks holistically at all changes over a period of time (weekly, monthly, quarterly). Very effective when linked to a set of leading and lagging indicators.
Incident and events: Deviations happen. Make sure you learn the lessons and implement solutions.

The chosen formality should be based on the level of change. A healthy organization will be utilizing all of these.

Level of Change	Form of Lesson Learned
Transactional	Consistency discussion After action (when things go wrong)
Organizational	Retrospective After action (weekly, daily as needed)
Transformational	Retrospective After action (daily)

Successful lessons learned:

Are based on solid performance data: Based on facts and the analysis of facts.
Look at positive and negative experiences.
Refer back to the change management process, objectives of the change, and other success criteria
Separate experience from opinion as much as possible. A lesson arises from actual experience and is an objective reflection on the results.
Generate distinct lessons from which others can learn and take action. A good action avoids generalities.

In practice there are a lot of similarities between the techniques to facilitate a good lessons learned and a root cause analysis. Start with a good core of questions, starting with the what:

What were some of the key issues?
What were the success factors?
What worked well?
What did not work well?
What were the challenges and pitfalls?
What would you approach differently if you ever did this again?

From these what questions, we can continue to narrow in on the learnings by asking why and how questions. Ask open questions, and utilize all the techniques of root cause analysis here.

Then once you are at (or close) to a defined issue for the learning (a root cause), ask a future-tense question to make it actionable, such as:

What would your advice be for someone doing this in the future?
What would you do next time?

Press for specifics. if it is not actionable it is not really a learning.

Update the Process

Learning implies memory, and an organization’s memories usually require procedures, job aids and other tools to be updated and created. In short, lessons should evolve your process. This is often the responsibility of the change management process owner. You need to make sure the lesson actually takes hold.

Differences between effectiveness reviews and lesson’s learned

There are three things to answer in every change

Was the change effective – did it meet the intended purposes
Did the change have any unexpected effects
What can we learn from this change for the next change?

Effectiveness reviews are 1 and 2 (based on a risk based approach) while lessons learned is 3. Lessons learned contributes to the health of the system and drives continuous improvements in the how we make changes.

Citations

Lesson learned management model for solving incidents. (2017). 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), Information Systems and Technologies (CISTI), 2017 12th Iberian Conference On, 1.
Fowlin, J. j & Cennamo, K. (2017). Approaching Knowledge Management Through the Lens of the Knowledge Life Cycle: a Case Study Investigation. TechTrends: Linking Research & Practice to Improve Learning, 61(1), 55–64.
Michell, V., & McKenzie, J. (2017). Lessons learned: Structuring knowledge codification and abstraction to provide meaningful information for learning. VINE: The Journal of Information & Knowledge Management Systems, 47(3), 411–428.
Milton, N. J. (2010). The Lessons Learned Handbook : Practical Approaches to Learning From Experience. Burlington: Chandos Publishing.
Paul R. Carlile. (2004). Transferring, Translating, and Transforming: An Integrative Framework for Managing Knowledge across Boundaries. Organization Science, (5), 555.
Secchi, P. (Ed.) (1999). Proceedings of Alerts and Lessons Learned: An Effective way to prevent failures and problems. Technical Report WPP-167. Noordwijk, The Netherlands: ESTEC

Quality and Ethics are Inseparable

There is a strong correlation between quality and ethics. Leadership’s demonstration of their philosophy and practice of ethical behavior impacts the whole organization in education, government or commercial enterprises
Dennis Sergent, The Ethics of Quality. The W. Edwards Deming Institute Blog

Quality is a management methodology, a set of ethics and a grab-bag of technical skills and tools (many of which are not unique to quality). Dennis Sergent does a good job riffing off of Deming’s Code of Professional Conduct, and in light of my recent post “Being a Quality Leader” I wanted to briefly talk about how leadership is perhaps the most effective lever in producing an ethical organization.

There are three major parts of ethical leadership:

Conscientiousness
Moral identity
Cognitive moral development, meaning how sophisticated one’s thinking is about ethical issues

Ethics and Quality are hand-in-hand. You cannot create a quality product if you do not have an ethical framework. I often think this is a part of Deming’s message that has been lost.

Being a Quality Leader

Domain Knowledge

Having recently said farewell to a leader in our quality organization, I have been reflecting on quality leaders and what makes one great. As I often do, I look to standards, in this case the American Society of Quality (ASQ).

The Certified Manager of Quality/Organizational Excellence (CMQ/OE)leads and champions process improvement initiatives—that can have regional or global focus—in various service and industrial settings. A CMQ/OE facilitates and leads team efforts to establish and monitor customer/supplier relations,supports strategic planning and deployment initiatives, and helps develop measurement systems to determine organizational improvement.
American Society of Quality

The ASQ’s Certified Manager of Quality/Operation Excellence (CMQ/OE) body of knowledge‘s first section is on leadership.

To be honest, the current body of knowledge (bok) is a hodge-podge collection of stuff that is sort of related but often misses a real thematic underpinning. The bok (and the exam) could use a healthy dose of structure when laying out the principles of roles and responsibilities, change management, leadership techniques and empowerment.

There are fundamental skills to being a leader:

Shape a vision that is exciting and challenging for your team (or division/unit/organization).
Translate that vision into a clear strategy about what actions to take, and what not to do.
Recruit, develop, and reward a team of great people to carry out the strategy.
Focus on measurable results.
Foster innovation and learning to sustain your team (or organization) and grow new leaders.
Lead yourself — know yourself, improve yourself, and manage the appropriate balance in your own life.

In order to do these things a leader needs to demonstrate skills in communication, critical thinking, problem solving, and skills motivating and leading teams (and self).

The best leaders know a lot about the domain in which they are leading, and part of what makes them successful in a management role is technical competence. A Quality leader needs to know quality as a domain AND the domain of the industry they are within.

Three domains necessary for a quality leader

In my industry it is just not enough to know quality (for now we’ll define that as the ASQ BoK) nor is it enough to know pharmaceuticals (with regulatory being a subdomain). It is not enough just to have leadership skills. It is critical to be able to operate in all three areas.

To excel as a leader in practice, you also need a lot of expertise in a particular domain.

As an example, take the skill of thinking critically in order to find the essence of a situation. To do that well, you must have specific, technical expertise. The critical information an engineer needs to design a purification system is different from the knowledge used to understand drug safety, and both of those differ in important ways from what is needed to negotiate a good business deal.

When you begin to look at any of the core skills that leaders have, it quickly becomes clear that domain-specific expertise is bound up in all of them. And the domains of expertise required may also be fairly specific. Even business is not really a single domain. Leadership in pharmaceuticals, transportation, and internet (for example) all require a lot of specific knowledge.

Similarly, with only leadership and technical, you are going to fumble. Quality brings a set of practices necessary for success. A domain filled with analytical and decision making capabilities that cross-over with leadership (critical thinking and problem-solving) but are deepened with that perspective.

There are also other smaller domains, or flavors of domains. If I was building this model out more seriously I would have an interesting cluster of Health and Safety with Quality (the wider bucket of compliance even). I’m simplifying for this post.

To go a step further. These three domains are critical for any quality professional. What changes is the development of wisdom and the widening of scope. This is why tenure is important. People need to be able to settle down and develop the skills they need to be successful in all three domains.

Good quality leaders recognize all this and look to build their organizations to reflect the growth of technical, quality and leadership domain.