Coherence and Quality

Sonja Blignaut on More Beyond wrote a good post “All that jazz … making coherence coherent” on coherence where she states at the end “In order to remain competitive and thrive in the new world of work, we need to focus our organization design, leadership and strategic efforts on the complex contexts and create the conditions for coherence. “

Ms. Blignaut defines coherence mainly through analogy and metaphor, so I strongly recommend reading the original post.

In my post “Forget the technology, Quality 4.0 is all about thinking” I spelled out some principles of system design.

Principle	Description
Balance	The system creates value for the multiple stakeholders. While the ideal is to develop a design that maximizes the value for all the key stakeholders, the designer often has to compromise and balance the needs of the various stakeholders.
Congruence	The degree to which the system components are aligned and consistent with each other and the other organizational systems, culture, plans, processes, information, resource decisions, and actions.
Convenience	The system is designed to be as convenient as possible for the participants to implement (a.k.a. user friendly). System includes specific processes, procedures, and controls only when necessary.
Coordination	System components are interconnected and harmonized with the other (internal and external) components, systems, plans, processes, information, and resource decisions toward common action or effort. This is beyond congruence and is achieved when the individual components of a system operate as a fully interconnected unit.
Elegance	Complexity vs. benefit — the system includes only enough complexity as is necessary to meet the stakeholder’s needs. In other words, keep the design as simple as possible and no more while delivering the desired benefits. It often requires looking at the system in new ways.
Human	Participants in the system are able to find joy, purpose and meaning in their work.
Learning	Knowledge management, with opportunities for reflection and learning (learning loops), is designed into the system. Reflection and learning are built into the system at key points to encourage single- and double-loop learning from experience to improve future implementation and to systematically evaluate the design of the system itself.
Sustainability	The system effectively meets the near- and long-term needs of the current stakeholders without compromising the ability of future generations of stakeholders to meet their own needs.

I used the term congruence to summarize the point Ms. Blignaut is reaching with alignment and coherence. I love her putting these against the Cynefin framework, it makes a great of sense to see alignment for the obvious domain and the need for coherence driving from complexity.

So what might a drive for coherence look like? Well if we start with coherence being the long-range order (the jazz analogy) we are building systems that build order through their function – they learn and are sustainable.

To apply this in the framework of ICHQ10 or the US FDA’s “Guidance for Industry Quality Systems Approach to Pharmaceutical CGMP Regulations” one way to drive for coherence is to use similar building blocks across our systems: risk management, data integrity, and knowledge management are all examples of that.

Review of Audit Trails

One of the requirements for data integrity that has changed in detail as the various guidances (FDA, MHRA, PIC/S) have gone through draft has been review of audit trails. This will also probably be one of the more controversial in certain corners as it can be seen by some as going beyond what has traditionally been the focus of good document practices and computer system validation.

What the guidances say

Audit trail review is similar to assessing cross-outs on paper when reviewing data. Personnel responsible for record review under CGMP should review the audit trails that capture changes to data associated with the record as they review the rest of the record (e.g., §§ 211.22(a), 211.101(c) and (d), 211.103, 211.182, 211.186(a), 211.192, 211.194(a)(8), and 212.20(d)). For example, all production and control records, which includes audit trails, must be reviewed and approved by the quality unit (§ 211.192). The regulations provide flexibility to have some activities reviewed by a person directly supervising or checking information (e.g., § 211.188). FDA recommends a quality system approach to implementing oversight and review of CGMP records.
US FDA. “Who should review audit trails?” Data Integrity and Compliance With Drug CGMP Questions and Answers Guidance for Industry. Section 7, page 8

If the review frequency for the data is specified in CGMP regulations, adhere to that frequency for the audit trail review. For example, § 211.188(b) requires review after each significant step in manufacture, processing, packing, or holding, and § 211.22 requires data review before batch release. In these cases, you would apply the same review frequency for the audit trail.If the review frequency for the data is not specified in CGMP regulations, you should determine the review frequency for the audit trail using knowledge of your processes and risk assessment tools. The risk assessment should include evaluation of data criticality, control mechanisms, and impact on product quality. Your approach to audit trail review and the frequency with which you conduct it should ensure that CGMP requirements are met, appropriate controls are implemented, and the reliability of the review is proven.

US FDA. “How often should audit trails be reviewed?” Data Integrity and Compliance With Drug CGMP Questions and Answers Guidance for Industry. Section 8, page 8

	Expectations	Potential risk of not meeting expectations / items to be checked
1	Consideration should be given to data management and integrity requirements when purchasing and implementing computerised systems. Companies should select software that includes appropriate electronic audit trail functionality. Companies should endeavour to purchase and upgrade older systems to implement software that includes electronic audit trail functionality. It is acknowledged that some very simple systems lack appropriate audit trails; however, alternative arrangements to verify the veracity of data must be implemented, e.g. administrative procedures, secondary checks and controls. Additional guidance may be found under section 9.9 regarding Hybrid Systems. Audit trail functionality should be verified during validation of the system to ensure that all changes and deletions of critical data associated with each manual activity are recorded and meet ALCOA+ principles. Audit trail functionalities must be enabled and locked at all times and it must not be possible to deactivate the functionality. If it is possible for administrative users to deactivate the audit trail functionality, an automatic entry should be made in the audit trail indicating that the functionality has been deactivated. Companies should implement procedures that outline their policy and processes for the review of audit trails in accordance with risk management principles. Critical audit trails related to each operation should be independently reviewed with all other records related to the operation and prior to the review of the completion of the operation, e.g. prior to batch release, so as to ensure that critical data and changes to it are acceptable. This review should be performed by the originating department, and where necessary verified by the quality unit, e.g. during self-inspection or investigative activities.	Validation documentation should demonstrate that audit trails are functional, and that all activities, changes and other transactions within the systems are recorded, together with all metadata. Verify that audit trails are regularly reviewed (in accordance with quality risk management principles) and that discrepancies are investigated. If no electronic audit trail system exists a paper based record to demonstrate changes to data may be acceptable until a fully audit trailed (integrated system or independent audit software using a validated interface) system becomes available. These hybrid systems are permitted, where they achieve equivalence to integrated audit trail, such as described in Annex 11 of the PIC/S GMP Guide. Failure to adequately review audit trails may allow manipulated or erroneous data to be inadvertently accepted by the Quality Unit and/or Authorised Person. Clear details of which data are critical, and which changes and deletions must be recorded (audit trail) should be documented.
2	Where available, audit trail functionalities for electronic-based systems should be assessed and configured properly to capture any critical activities relating to the acquisition, deletion, overwriting of and changes to data for audit purposes. Audit trails should be configured to record all manually initiated processes related to critical data. The system should provide a secure, computer generated, time stamped audit trail to independently record the date and time of entries and actions that create, modify, or delete electronic records. The audit trail should include the following parameters: – Who made the change – What was changed, incl. old and new values – When the change was made, incl. date and time – Why the change was made (reason) – Name of any person authorising the change. The audit trail should allow for reconstruction of the course of events relating to the creation, modification, or deletion of an electronic record. The system must be able to print and provide an electronic copy of the audit trail, and whether looked at in the system or in a copy, the audit trail should be available in a meaningful format. If possible, the audit trail should retain the dynamic functionalities found in the computer system, e.g. search functionality and export to e.g. Excel	Verify the format of audit trails to ensure that all critical and relevant information is captured. The audit trail must include all previous values and record changes must not obscure previously recorded information. Audit trail entries should be recorded in true time and reflect the actual time of activities. Systems recording the same time for a number of sequential interactions, or which only make an entry in the audit trail, once all interactions have been completed, may not in compliance with expectations to data integrity, particularly where each discrete interaction or sequence is critical, e.g. for the electronic recording of addition of 4 raw materials to a mixing vessel. If the order of addition is a CPP, then each addition should be recorded individually, with time stamps. If the order of addition is not a CCP then the addition of all 4 materials could be recored as a single timestamped activity.

PIC/S. PI 041-1 “Good Practices for Data Management and Data Integrity in regulated GMP/GDP Environments“ (3rd draft) section 9.4 “Audit trail for computerised systems” page 36

Thoughts

It has long been the requirement that computer systems have audit trails and that these be convertible to a format that can be reviewed as appropriate. What these guidances are stating is:

There are key activities captured in the audit trail. These key determined in a risk-based manner.
These key activities need to be reviewed when making decisions based on them (determine a frequency)
The audit trail needs to be able to show the reviewer the key activity
These reviews needs to be captured in the quality system (proceduralized, recorded)
This is part of the validated state of your system

So for example, my deviation system is evaluated and the key activity that needs to be reviewed in the decision to forward process. In this deviation decision quality makes the determination at several points of the workflow. The audit trail review would thus be looking at who made the decision when and did that meet criteria. The frequency might be established at the point of disposition for any deviation still in an opened state and upon closure.

What we are being asked is to evaluate all your computer systems and figure out what parts of the audit trail need to be reviewed when.

Now here’s the problem. Most audit trails are garbage. Maybe they are human readable by some vague definition of readable (or even human). But they don’t have filters, or search or templates. So companies need to be (again based on a risk based approach) evaluating their audit trails system by system to see if they are up-to-the-task. You then end up with one or more solutions:

Rebuild the audit trail to make it human readable and give filters and search criteria. For example on a deviation record there is one view for “disposition” and another for “closure”
Add reports (such as a set of crystal reports) to make it human readable and give filters and search criteria. Probably end up with a report for “disposition” and another report for “closure.”
Utilize an export function to Excel (or similar program)and use Excel’s functions to filter and search. Remember to ensure you have a data verification process in place.
The best solution is to ensure the audit trail is a step in your workflow and the review is captured as part of the audit trail. Ideally this is part of an exception reporting process driven by the system.

What risk based questions should drive this?

Overall risk of the system
Capabilities of audit trail
Can the data be modified after entry? Can it be modified prior to approval?
Is the result qualitative or quantitative
Are changes to data visible on the record itself?

Data Integrity and the Role of Guidances

Interesting piece over at FDALawBlog on the new data integrity guidance “New Data Integrity Guidance Imposes Significant Burdens, Yet FDA Claims It Does Not Regulate by Guidance.”

I find it interesting to read a different perspective. I tend to be a big fan of guidances (they always need work) as they help lay down how we can get better and improve. Being on the front line of regulatory inspections probably more than a group of lawyers, I recognize the differences in how guidances are treated differently than regulations, and how the agencies apply very long lead times on how inspections treat this material. And frankly, the 483s and Warning Letters we are seeing coming out of data integrity scare the beejeezus out of me. There is also a need for the FDA to ensure it’s thinking on matters is aligned with our European and rest-of-world counterparts, especially in this day of mutual recognition agreements.

Regulatory and administrative law is definitely continually evolving. It is important to be aware of a variety of perspectives on the subject.

Quality and Ethics are Inseparable

There is a strong correlation between quality and ethics. Leadership’s demonstration of their philosophy and practice of ethical behavior impacts the whole organization in education, government or commercial enterprises
Dennis Sergent, The Ethics of Quality. The W. Edwards Deming Institute Blog

Quality is a management methodology, a set of ethics and a grab-bag of technical skills and tools (many of which are not unique to quality). Dennis Sergent does a good job riffing off of Deming’s Code of Professional Conduct, and in light of my recent post “Being a Quality Leader” I wanted to briefly talk about how leadership is perhaps the most effective lever in producing an ethical organization.

There are three major parts of ethical leadership:

Conscientiousness
Moral identity
Cognitive moral development, meaning how sophisticated one’s thinking is about ethical issues

Ethics and Quality are hand-in-hand. You cannot create a quality product if you do not have an ethical framework. I often think this is a part of Deming’s message that has been lost.

Being a Quality Leader

Domain Knowledge

Having recently said farewell to a leader in our quality organization, I have been reflecting on quality leaders and what makes one great. As I often do, I look to standards, in this case the American Society of Quality (ASQ).

The Certified Manager of Quality/Organizational Excellence (CMQ/OE)leads and champions process improvement initiatives—that can have regional or global focus—in various service and industrial settings. A CMQ/OE facilitates and leads team efforts to establish and monitor customer/supplier relations,supports strategic planning and deployment initiatives, and helps develop measurement systems to determine organizational improvement.
American Society of Quality

The ASQ’s Certified Manager of Quality/Operation Excellence (CMQ/OE) body of knowledge‘s first section is on leadership.

To be honest, the current body of knowledge (bok) is a hodge-podge collection of stuff that is sort of related but often misses a real thematic underpinning. The bok (and the exam) could use a healthy dose of structure when laying out the principles of roles and responsibilities, change management, leadership techniques and empowerment.

There are fundamental skills to being a leader:

Shape a vision that is exciting and challenging for your team (or division/unit/organization).
Translate that vision into a clear strategy about what actions to take, and what not to do.
Recruit, develop, and reward a team of great people to carry out the strategy.
Focus on measurable results.
Foster innovation and learning to sustain your team (or organization) and grow new leaders.
Lead yourself — know yourself, improve yourself, and manage the appropriate balance in your own life.

In order to do these things a leader needs to demonstrate skills in communication, critical thinking, problem solving, and skills motivating and leading teams (and self).

The best leaders know a lot about the domain in which they are leading, and part of what makes them successful in a management role is technical competence. A Quality leader needs to know quality as a domain AND the domain of the industry they are within.

Three domains necessary for a quality leader

In my industry it is just not enough to know quality (for now we’ll define that as the ASQ BoK) nor is it enough to know pharmaceuticals (with regulatory being a subdomain). It is not enough just to have leadership skills. It is critical to be able to operate in all three areas.

To excel as a leader in practice, you also need a lot of expertise in a particular domain.

As an example, take the skill of thinking critically in order to find the essence of a situation. To do that well, you must have specific, technical expertise. The critical information an engineer needs to design a purification system is different from the knowledge used to understand drug safety, and both of those differ in important ways from what is needed to negotiate a good business deal.

When you begin to look at any of the core skills that leaders have, it quickly becomes clear that domain-specific expertise is bound up in all of them. And the domains of expertise required may also be fairly specific. Even business is not really a single domain. Leadership in pharmaceuticals, transportation, and internet (for example) all require a lot of specific knowledge.

Similarly, with only leadership and technical, you are going to fumble. Quality brings a set of practices necessary for success. A domain filled with analytical and decision making capabilities that cross-over with leadership (critical thinking and problem-solving) but are deepened with that perspective.

There are also other smaller domains, or flavors of domains. If I was building this model out more seriously I would have an interesting cluster of Health and Safety with Quality (the wider bucket of compliance even). I’m simplifying for this post.

To go a step further. These three domains are critical for any quality professional. What changes is the development of wisdom and the widening of scope. This is why tenure is important. People need to be able to settle down and develop the skills they need to be successful in all three domains.

Good quality leaders recognize all this and look to build their organizations to reflect the growth of technical, quality and leadership domain.