Last month the World Health Organization published a draft guideline on Good practices for research and development facilities.
This arrow pretty much sums it up:
Seriously, not much surprising here. What this guide definitely does is place early research in the framework of Q10 and point out that there is one quality system to rule them all and that level of rigor is based on risk.
Give it a read.
Quality System Management should be viewed and governed as a program
Program management is commonly defined as “a group of projects that contribute to a common, higher order objective.” The projects in a program are related, and the intent of achieving benefits would not be realized if the projects were managed independently.
Program management includes the practices and processes of strategic alignment, benefits management, stakeholder management, governance, and lifecycle management. Program governance creates the control framework for delivering the programs’ change objectives and making benefit delivery visible to the organization’s control.
There are different styles of program management and what I am focusing on here is what is sometimes called “heartbeat”, which aims to achieve evolutionary improvement of existing systems and processes or organizational change. This program type creates value by reconciling contradicting views and demands for change from various organization actors in order to enhance existing systems and practices while sustaining operations.
Heartbeat program management is all about awareness of the contexts of the program and requires knowledge of strategy, competition, trends in the industry, and differences in management practices between the business units of the company. A good heartbeat program manager is highly concerned about their program’s long-term effects and implications for the company’s business.
Programs exist to create value by improving the management of projects and to create benefits through better organization of projects. The fundamental goals of program management are:
| Goal | Description |
| Efficiency and effectiveness goals | |
| Improved co-ordination | Assist in identification and definition of project inter-dependencies and thereby reduce the incidence of work backlogs, rework and delays |
| Improved dependency management | Reduce the amount of re-engineering required due to inadequate management of the interfaces between projects |
| More effective resource utilization | Improve the effectiveness and efficiency of the allocation of shared resources Assist in providing justification for specialist resources that deliver an overall improvement to program delivery and/or business operations |
| More effective knowledge transfer | Provide a means to identify and improve upon transferable lessons. Facilitate organizational learning |
| Greater senior management ‘visibility’ | Enable senior management to better monitor, direct and control the implementation process |
| Business focus goals | |
| More coherent communication | Improve communication of overall goals and direction both internally and externally to the program Target management attention clearly on the realization of benefits that are defined and understood at the outset and achieved through the lifetime of the program and beyond Assist in keeping personal agendas in check |
| Improved project definition | Ensure that project definition is more systematic and objective, thereby reducing the prevalence of projects with a high risk of failure or obsolescence Enable the unbundling of activities in a strategic project-set into specific projects Enable the bundling of related projects together to create a greater leverage or achieve economies of scale |
| Better alignment with business drivers, goals and strategy | Improves the linkage between the strategic direction of organizations and the management activities required to achieve these strategic objectives Provide an enabling framework for the realization of strategic change and the ongoing alignment of strategy and projects in response to a changing business environment (via project addition/culling, etc.) |
As quality leaders we are often ambassadors to ensure that the quality program is progressing despite the conflicting requirements of the various stakeholders. We need to actively influence quality-related decisions of all stakeholders, including people holding superior positions. Having a well-developed personal network within the organization is particularly helpful.
It is critical to always be communicating about the quality program in a visionary way, to be seen as passionate ambassadors. Playing this role requires constant attention to differing expectations of the stakeholders and various ways to influence stakeholders for the benefit of the quality system. To always be striving to build quality, to advance quality.
As advocates for Quality, it is a core competency to be able to stand up and defend, or argue for, the quality program and team members. This ability to challenge others, including their superiors, in a productive way is a critical ability.
A key focus of the quality program should be on engagement with a conscious and sustained drive to secure buy-in from key stakeholders (including senior management) and win over the hearts and minds of those responsible for execution to make changes feel less painful and inflicted. As quality leaders our aim should always be to engender a climate of comprehension, inclusion and trust, and to draw upon expertise globally to create fit for purpose processes and systems
Effective quality leaders need to be “heavyweight” organizational players.
Many standard approaches perceive programs to have a finite life. This is constraining given that the strategies themselves, especially as applied to quality, have long lifetimes. I believe that program management has as much to learn from quality management, and there is a lot of value in seeing an indefinite time horizon as beneficial.
Quality management is an evolutionary approach, and utilizing program management methodologies within it should be taken in the same light.
We have these quality systems with lots of levers, with interrelated components. And yet we select one or two metrics and realize that even if we meet them, we aren’t really measuring the right stuff nor are we driving continious improvement.
One solution is to create layered metrics, which basically means drill down your process and identify the metrics at each step.
Lots of ways to do this. An easy way to start is to use the 5-why process, a tool most folks are comfortable with.
So for example, CAPA. It is pretty much agreed upon that CAPAs should be completed in a timely manner. That makes this a top level goal. Unfortunately, in this hypothetical example, we are suffering a less than 100% closure goal (or whatever level is appropriate in your organization based on maturity)
| Why 1 | Why was CAPA closure not 100% Because CAPA tasks were not closed on time. Success factor needed for this step: CAPA tasks to be closed by due date. Metric for this step: CAPA closure task success rate |
| Why 2 | Why were CAPA tasks not closed on time? Because individuals did not have appropriate time to complete CAPA tasks. Metric for this step: Planned versus Actual time commitment |
| Why 3 | Why did individuals not have appropriate time to complete CAPA tasks? Because CAPA task due dates are guessed at. Metric for this step: CAPA task adherence to target dates based on activity (e.g. it takes 14 days to revise a document and another 14 days to train, the average document revision task should be 28 days) |
| Why 4 | Why are CAPA task due dates guessed at? Because appropriate project planning is not completed. Metric for this step: Adherence to Process Confirmation |
| Why 5 | Why is appropriate project planning not completed? Because CAPAs are always determined on the last day the deviation is due. Metric: Adherence to Root Cause Analysis process |
I might on report on the top CAPA closure rate and 1 or 2 of these, and keep the others in my process owner toolkit. Maybe we jump right to the last one as what we report on. Depends on what needs to be influenced in my organization and it will change over time.
It helps to compare this output against the 12 system leverage points.
These metrics go from 3 “goals of the system” with completing CAPA tasks effectively and on time, to 4 “self organize” and 5 “rules of the system.” It also has nice feedback loops based on the process confirmations. I’d view them as potentially pretty successful. Of course, we would test these and tinker and basically experiment until we find the right set of metrics that improves our top-level goal.
Effectiveness I recently had a bit of a wake-up call via Twitter. I asked the following question: “What’s the one thing /above all/ that makes for an effective organisation?” My thanks to all those who took the time to reply with their viewpoint. The wake-up call for me was the variety of these responses. All […]
via Effectiveness — Think Different
Great thought-piece over on “Think Different” on effectiveness, with a nice tie-in to Donnella Meadow’s “Twelve Leverage Points to Intervene in a System.”
In quality management systems, it is critical to look at effectiveness. If you do not measure, you do not know if the system is working the ways you expect and desire.
We often discuss lagging (output measurement) and leading (predictive) indicators, and this is a good way to start, but if we apply System Thinking and use Meadow’s twelve leverage points we can see that most metrics tend to be around 7-12, with the more effective levers being the least utilized.
I think there are a lot of value in finding metrics within these levers.
So for example, a few indicators on the effectiveness of lever 4 “The Power to Add, Change, Evolve, or Self-Organize System Structure”:
| Lagging | Leading |
| Effective CAPAs to the System | Number of changes initiated by level of organization and scale of change |
| Deviation Reduction |
We determined that ULA, SpaceX, and AR were not performing adequate quality assurance management for the EELV program as evidenced by the 181 nonconformities to the AS9100C at the EELV contractor production facilities. This inadequate quality assurance management could increase costs, delay launch schedules, and increase the risk of mission failure.
From ”
Evaluation of the Evolved Expendable Launch Vehicle Program Quality Management System DODIG-2018-045, Department of Defense Office of Inspector General
It is useful to read audit reports and inspection findings from multiple industries. From this we can see trends, make connections and learn.
I see a few things that stand out.
Our evaluation of the RIO database showed that 11 out of 26 risks related to either Atlas V or Delta IV launch vehicle were in “red” status, which indicates that risk mitigation was behind schedule.
It is not enough to identify risks (though that is a critical place to start). You just can’t track them (though again, if you don’t track it you don’t see it). You actually have to have clear plans to mitigate and eliminate the risks. And this is where the program seemed to fall short.
Not a surprise. I think a lot of companies are having these difficulties. In the pharma world the regulatory agencies have been signaling pretty strongly that this is an issue.
Make sure you identify risks, track them, and have plans that are actually carried out to remediate.
SpaceX failed to comply with AS9100C, section 7.1.3, which requires it to “establish, implement, and maintain a configuration management process.” Configuration management is a controlled process to establish the baseline configuration of a product and any changes to that product. This process should occur during the entire life cycle of a product to provide visibility and control of its physical, functional, and performance attributes.
First rule of reading inspection reports: Things probably went bad if a section starts with standard review 101 material.
That said, hello change management my dear friend.
This was the gist of my ASQ WCQI workshop last May, every industry needs good change management and change control.
ULA and AR failed to comply with AS9100C, section 8.3, which requires them to “ensure that product which does not conform to product requirements are identified and controlled to prevent its unintended use or delivery.”
At ULA, we found 18 expired limited-life material items that were between 32 and 992 days past their expiration dates, but available for use on EELV flight hardware. This material should have been impounded and dispositioned. The use of expired limited-life items, such as glues and bonding agents, could result in product that does not meet specifications and may require costly rework.
I find it hard to believe that these companies aren’t tracking inventory. If they are tracking inventory and have any sort of cycle count process then the mechanism exists to ensure expired material is removed from the possibility of use. And yet we still see these observations across the pharma industry as well.
Quality Management has it its core the same principles, no matter the industry. We use similar tools. Leverage the best practices out there. Read about stresses other companies are having, learn from them and remediate at your own organization.
