Falsification and error

At the heart, data integrity is a lot about culture. There are technical requirements, but mostly we are returning to the same principles as quality culture and just keep coming back to Deming. A great example of this is the use of the fraud triangle and human error.

The fraud triangle was developed by Donald Cressey in the 1950s when investigating financial fraud and embezzlement. The principles Cressey identified are directly relevant to data integrity, and to quality culture as a whole.

Falsification Triangle
Element Exists When To Break
Incentive or Pressure Why commit falsification of data? Managerial pressure or financial gains are the two main drivers here to push people to commit fraud. Setting unrealistic objectives such as stretch goals, turnaround time or key performance indicators that are totally divorced from reality especially when these are linked to pay or advancement will only encourage staff to falsify data to receive rewards. These goals coupled with poor analytical instruments and methods will only ensure that corners will be cut to meet deadlines or targets. Management must lead by example – not through communication or establishing data governance structures but by ensuring the pressure to falsify data is removed. This means setting realistic expectations that are compatible with the organization’s capacity and process capability.
Rationalization or Incentive To commit fraud people must either have an incentive or can rationalize that this is an acceptable practice within an organization or department. Staff need to understand how their actions can impact the health of the patient. Ensure individuals know the importance of reliable and accurate data to the wellbeing of the patient as well as the business health of the company.
Opportunity The opportunity to falsify data can be due to encouragement by management as a means of keeping cost down or a combination of lax controls or poor oversight of activities that contribute to staff being able to commit fraud. Implement a process that is technically controlled so there is little, if any, opportunity to commit falsification of data.

Mistakes are human nature – we all have fat finger moments. This is why we build our processes and technologies to ensure we capture these errors and self-correct them. These errors should be tracked and trended, but only as a way to drive continuous improvement. It is important to have the capability in your quality systems to be able to evaluate mistakes up-to-and including fraud.

It helps to be able to classify issues and determine if there are changes to governance, management systems and behaviors necessary.

Events should be classified based on how intentional they are

Human error should be built into investigative systems. Yes, whenever possible we are looking for technical controls, but the human exists and needs to be fully taken into consideration.

The best way to ensure data integrity is the best way to build a quality culture.

System Model

Quality Management as a Program

Quality System Management should be viewed and governed as a program

Program management is commonly defined as “a group of projects that contribute to a common, higher order objective.” The projects in a program are related, and the intent of achieving benefits would not be realized if the projects were managed independently.

Program management includes the practices and processes of strategic alignment, benefits management, stakeholder management, governance, and lifecycle management. Program governance creates the control framework for delivering the programs’ change objectives and making benefit delivery visible to the organization’s control.

There are different styles of program management and what I am focusing on here is what is sometimes called “heartbeat”, which aims to achieve evolutionary improvement of existing systems and processes or organizational change. This program type creates value by reconciling contradicting views and demands for change from various organization actors in order to enhance existing systems and practices while sustaining operations.

Heartbeat program management is all about awareness of the contexts of the program and requires knowledge of strategy, competition, trends in the industry, and differences in management practices between the business units of the company. A good heartbeat program manager is highly concerned about their program’s long-term effects and implications for the company’s business.

Magic triangle of a program manager

Programs exist to create value by improving the management of projects and to create benefits through better organization of projects. The fundamental goals of program management are:

  • Efficiency and effectiveness: Aspects of management that a proficient project manager should address and benefit from coordination.
  • Business focus goal: The external alignment of projects with the requirements, goals, drivers and culture of the wider organization. These goals are associated with defining an appropriate direction for the constituent projects within a program as well as for the program as a whole.
GoalDescription
Efficiency and effectiveness goals
Improved co-ordinationAssist in identification and definition of project inter-dependencies and thereby reduce the incidence of work backlogs, rework and delays
Improved dependency managementReduce the amount of re-engineering required due to inadequate management of the interfaces between projects
More effective resource utilizationImprove the effectiveness and efficiency of the allocation of shared resources
Assist in providing justification for specialist resources that deliver an overall improvement to program delivery and/or business operations
More effective knowledge transferProvide a means to identify and improve upon transferable lessons.
Facilitate organizational learning
Greater senior management ‘visibility’Enable senior management to better monitor, direct and control the implementation process
Business focus goals
More coherent communicationImprove communication of overall goals and direction both internally and externally to the program
Target management attention clearly on the realization of benefits that are defined and understood at the outset and achieved through the lifetime of the program and beyond
Assist in keeping personal agendas in check
Improved project definitionEnsure that project definition is more systematic and objective, thereby reducing the prevalence of projects with a high risk of failure or obsolescence
Enable the unbundling of activities in a strategic project-set into specific projects
Enable the bundling of related projects together to create a greater leverage or achieve economies of scale
Better alignment with business drivers, goals and strategyImproves the linkage between the strategic direction of organizations and the management activities required to achieve these strategic objectives
Provide an enabling framework for the realization of strategic change and the ongoing alignment of strategy and projects in response to a changing business environment (via project addition/culling, etc.)

The Attributes of a Good Heartbeat Program Manager are the Attributes to a Good Quality Leader

As quality leaders we are often ambassadors to ensure that the quality program is progressing despite the conflicting requirements of the various stakeholders. We need to actively influence quality-related decisions of all stakeholders, including people holding superior positions. Having a well-developed personal network within the organization is particularly helpful.

It is critical to always be communicating about the quality program in a visionary way, to be seen as passionate ambassadors. Playing this role requires constant attention to differing expectations of the stakeholders and various ways to influence stakeholders for the benefit of the quality system. To always be striving to build quality, to advance quality.

As advocates for Quality, it is a core competency to be able to stand up and defend, or argue for, the quality program and team members. This ability to challenge others, including their superiors, in a productive way is a critical ability.

A key focus of the quality program should be on engagement with a conscious and sustained drive to secure buy-in from key stakeholders (including senior management) and win over the hearts and minds of those responsible for execution to make changes feel less painful and inflicted. As quality leaders our aim should always be to engender a climate of comprehension, inclusion and trust, and to draw upon expertise globally to create fit for purpose processes and systems

Effective quality leaders need to be “heavyweight” organizational players.

Core Competencies of the Heartbeat Manager

  • Contextual awareness
  • Scenario planning
  • Political skills
  • Courage
  • Networking

A note on program life

Many standard approaches perceive programs to have a finite life. This is constraining given that the strategies themselves, especially as applied to quality, have long lifetimes. I believe that program management has as much to learn from quality management,  and there is a lot of value in seeing an indefinite time horizon as beneficial.

Quality management is an evolutionary approach, and utilizing program management methodologies within it should be taken in the same light.

What is this quality profession all about?

A theme of this year for me has been focusing more and more on the difference between the product integrity focused approach to quality that folks in my profession normally focus in on and a more excellence focused approach. The two are not in opposition, but I can’t help feeling that the product integrity exclusiveness of many pharmaceutical quality professionals is holding us back.

This is especially on my mind coming back from ASQ WCQI and thinking about just how few of my pharma colleagues identify with that organization There are a whole host of reasons (including the fact that many people don’t associate with ANY professional association) but I can’t help but contemplate how do we make the excellence side of quality more relevant to not just pharmaceuticals but to wider questions of just what is quality anyway?

I’ve discussed the need to realize that we have different types of domain knowledges, but just what is this domain we call quality and is it truly its own discipline?

Disciplines can be modeled as a system comprising an “activity scope” that is enabled by a “knowledge base” but conditioned by a “guidance framework”.

From Rousseau, et al “A Typology for the Systems Field”
  • The guidance framework typically involves multiple worldviews. The same subject matter can be studied from different worldviews, and the theories around a given subject can be interpreted differently from different worldview perspectives. You can see this in the various flavors of continuous improvement or better yet, the presence of a sustainability push within the society.
  • The knowledge base is the data, theories and methodologies that drive the discipline
  • The activity scope describes the range of activities in a disciple, including the professional practice.

We’re probably truly multi-disciplinarian, in that the we draw from multiple other disciplines, a short list includes: Engineering, Computing, Control Theory, Mathematics, Information Theory, Operations research, system theory, Management sciences, a whole range of social sciences and more than I can think.

What does this mean?

I am more thinking aloud than anything at this point, but I think it’s important to work on developing the QBOK along a guidance framework, knowledge base and activity scope methodology. Then as we develop sub-body of knowledges we drill down from there, either in a very knowledge base way (such as the CMQ/OE) or in an activity scope (like the CPGP). I often feel that the way we develop these are more hit-and-miss and could do with some coherence – the biomedical auditor and hazop auditor are great examples of wanting to meet a very narrow need and thus being very very specific to a small set of the knowledge base.

I guess I’m striving towards applying theory to our practice a little more deliberately.

Some of the technical forums (Human Development and Leadership comes to mind) seem especially designed to pull information from one or two different originating disciplines and adapt it to the knowledge base. I think this process would be added by a coherent understanding of our guidance framework and just what the activity scope we are trying to address as discipline.

In short I am just thinking that a little more coherence, strategy and transparency would aid us as a profession. As I heard in many a conversation last week, we should probably as an organization be better at what we preach.

Sources

  • Bourke, J (2014). On Process Excellence vs. Operational Excellence vs. Business Excellence. BEX Institute.
  • Rousseau, D., Wilby, J., Billingham, J., & Blachfellner, S. (2016). A Typology for the Systems Field. Systema 4(1), 15-47
  • Wageeh, N. A. (2016). The Role of Organizational Agility in Enhancing Organizational Excellence: A Study on Telecommunications Sector in Egypt. International Journal of Business and Management, 11(4), 121

Bystander Effect, Open Communication a​nd Quality Culture

Our research suggests that the bystander effect can be real and strong in organizations, especially when problems linger out in the open to everyone’s knowledge. 

Insiya Hussain and Subra Tangirala (January 2019) “Why Open Secrets Exist in Organizations” Harvard Business Review

The bystander effect occurs when the presence of others discourages an individual from intervening in an emergency situation. When individuals relinquish responsibility for addressing a problem, the potential negative outcomes are wide-ranging. While a great deal of the research focuses on helping victims, the overcoming the bystander effect is very relevant to building a quality culture.

The literature on this often follows after social psychologists John M. Darley and Bibb Latané who identified the concept in the late ’60s. They defined five characteristics bystanders go through:

  1. Notice that something is going on
  2. Interpret the situation as being an emergency
  3. Degree of responsibility felt
  4. Form of assistance
  5. Implement the action choice

This is very similar to the 5 Cs of trouble-shooting: Concern (Notice), Cause (Interpret), Countermeasure (Form of Assistance and Implement), Check results.

What is critical here is that degree of responsibility felt. Without it we see people looking at a problem and shrugging, and then the problem goes on and on. It is also possible for people to just be so busy that the degree of responsibility is felt to the wrong aspect, such as “get the task done” or “do not slow down operations” and it leads to the wrong form of assistance – the wrong troubleshooting.

When building a quality culture, and making sure troubleshooting is an ingrained activity, it is important to work with employees so they understand that their voices are not redundant and that they need to share their opinions even if others have the same information. As the HBR article says: “If you see something, say something (even if others see the same thing).”

Building a quality culture is all about building norms which encourage detection of potential threats or problems and norms which encouraged improvements and innovation.

When troubleshooting causes trouble

I recently had a discussion with one of the best root cause investigators and problem solvers I know, Thor McRockhammer. Thor had concerns about a case where the expected conditions were not met and there were indications that individuals engaged in troubleshooting and as a result not only made the problem worse but led to a set of issues that seem rather systematic.

Our conversation (which I do not want to go into too much detail on) was a great example of troubleshooting going wrong.

Troubleshooting is defined as “Reactive problem solving based upon quick responses to immediate symptoms. Provides relief and immediate problem mitigation. But may fail to get at the real cause, which can lead to prolonged cycles of firefighting.” Troubleshooting usually goes wrong one of a few ways:

  1. Not knowing when troubleshooting shouldn’t be executed
  2. Using troubleshooting exclusively
  3. Not knowing when to go to other problem solving tools (usually “Gap from standard”) or to trigger other quality systems, such as change management.

Troubleshooting is a reactive process of fixing problems by rapid response and short-term corrective actions. It covers noticing the problem, stopping the damage and preventing spread of the problem.

So if our departure from expected conditions was a leaky gasket, then troubleshooting is to try to stop the leak. If our departure is a missing cart then troubleshooting usually involves finding the cart.

Troubleshooting puts things back into the expected condition without changing anything. It addresses the symptom and not the fundamental problems and their underlying causes. They are carried out directly by the people who experience the symptoms, relying upon thorough training, expertise and procedures designed explicitly for troubleshooting.

With out leaky gasket example, our operators are trained and have procedural guidance to tighten or even replace a gasket. They also know what not to do (for example don’t weld the pipe, don’t use a different gasket, etc). There is also a process for documenting the troubleshooting happened (work order, comment, etc).

To avoid the problems listed above troubleshooting needs a process that people can be thoroughly trained in. This process needs to cover what to do, how to communicate it, and where the boundaries are.

4 Cs of Trouble shooting from Art Smalley’s Four Types of Problems

Step

What we do

Things to be aware of

Concern

·       What do we known about the exact nature of the problem?

·       What do your standards say about how this concern should be documented?

o   For example, can be addressed as a comment or does it require a deviation or similar non-conformance

·       If the concern stems from a requirement it must be documented.

Cause

·       What do you know about the apparent (or root) cause of the problem?

·       Troubleshooting is really good at dealing with superficial cause-and-effect relationships. As the cause deepens, fixing it requires deeper problem-solving.

·       The cause can be a deficiency or departure from a standard

 

Countermeasure

·       What immediate or temporary countermeasures can be taken to reduce or eliminate the problem?

·       Are follow-up or more permanent countermeasures required to prevent recurrence?

o   If so, do you need to investigate more deeply?

·       Countermeasures need to be evaluated against change management

·       Countermeasures cannot ignore, replace or go around standards

·       Apply good knowledge management

Check results

·       Did the results of the action have any immediate effect on eliminating the concern or problem?

·       Does the problem repeat?

o   If so, do you need to investigate more deeply?

·       Recurrence should trigger deeper problem-solving and be recorded in the quality system.

·       Beware troubleshooting countermeasures becoming tribal knowledge and the new way of working

 

Trouble shooting is in a box

Think of your standards as a box. This box defines what should happen per our qualified/validated state, our procedures, and the like. We can troubleshoot as much as we want within the box. We cannot change the box in any way, nor can we leave the box without triggering our deviation/nonconformance system (reactive) or entering change management (proactive).

Communication is critical for troubleshooting

Troubleshooting processes need a mechanism for letting supervisors happen. Troubleshooting that happens in the dark is going to cause a disaster.

Operators need to be trained how to document troubleshooting. Sometimes this is as simple as a notation or comment, other-times you trigger a corrective action process.

Engaging in troubleshooting, and not documenting it starts to look a like fraud and is a data integrity concern.

Change Management

The change management process should be triggered as a result of troubleshooting. Operators should be trained to interpret it. This is often were concept of exact replacements and like-for-like come in.

It is trouble shooting to replace a part with an exact part. Anything else (including functional equivalency) is a higher order change management activity. It is important that everyone involved knows the difference.

CoversIs it troubleshooting?
Like-for- LikeSpare parts that are identical replacements (has the same the same manufacturer, part number, material of construction, version)

Existing contingency procedures (documented, verified, ideally part of qualification/validation)
Yes

This should be built into procedures like corrective maintenance, spare parts, operations and even contingency procedures.
Functionally equivalentEquivalent, for example, performance, specifications, physical characteristics, usability, maintainability, cleanability, safety

No

Need to understand root cause.
Need to undergo appropriate level of change management
NewAnything else No

Need to understand root cause.
Need to undergo appropriate level of change management

This applies to both administrative and technical controls.

ITIL Incident Management

ITIL Incident Management (or similar models) is just troubleshooting and has all the same hallmarks and concerns.

Conclusion

Trouble shooting is an integral process. And like all processes it should have a standard, be trained on, and go through continuous improvement.