Change Management – Post Change Evaluation and Action

In “Change Management – Post Change Evaluation and Action” John Hunter (of CuriousCat) writes a nice post on the The W. Edwards Deming Institute Blog on linking change management to the PDSA (PDCA) lifecycle focusing on Act.

Post Change Evaluation is often called the effectiveness review, and is a critical part of change in the pharmaceutical quality system, and frankly is important no matter the industry.

An effectiveness review is the success criteria of the change viewed over enough data points based on a methodology informed by the nature of the change and risk.

 The success criteria should be achieved. If not, reasons why they have not been achieved should be assessed along with the mitigation steps to address the reasons why, including reverting to the previous operating state where appropriate. This may require the proposal of a subsequent change or amendment of the implementation plan to ensure success. Here we see the loop aspects of the PDSA lifecycle.

All changes should have a way back into knowledge management. The knowledge gathered from implementation of the change should be shared with the development function and other locations, as appropriate, to ensure that learning can be applied in products under development or to similar products manufactured at the same or other locations.

When choosing success criteria always strive for leading indicators that tell you how the change is working. Deviations are an awful way to judge the effectiveness of the change. Instead look for walkthroughs, checklists, audits, data gathering. Direct observation and real-time gathering and analysis of data of any sort is the best.

As mentioned above, ensure the change management/change control system is set up to deal with the inevitable change that does not work. Have a clear set of instructions on how to make that decision (returning to the success criteria), what steps to take to mitigate and what to do next. For example having guidance of when to create a deviation and on how to make a decision to rollback versus implement another change.

Risk Management leads to Change Management, Change Management contains Risk Management

We did an FMEA for the design of the room. Why do we need a risk assessment for the change control to implement the design features?

We have an environmental risk management plan, including a HAACP. Why does this change control require a new risk assessment?

If I received a nickel……

I want to expand on my earlier thoughts on risk management enabling change.

Risk Management is a key enabler of any quality by design, whether of product, facility or equipment. We do living risk assessments to understand the scope of our ongoing risk. Inevitably we either want to implement that new or improved design or we want to mitigate the ongoing risks in our operation. So we turn to change management. And as part of that change management we do a risk assessment. Our change management then informs ongoing risk review.

Risk Management Leads to Change Management

Design Implementation

Through your iterative design lifecycle there is a final design ready for introduction. Perhaps this is a totally new thing, perhaps it is a new set of equipment or processes, or just a modification.

All along through the iterative design lifecycle risk management has been applied to establish measurable, testable, unambiguous and traceable performance requirements. Now your process engages with change management to introduce the change.

And a new risk assessment is conducted.

This risk assessment is asking a different question. During the interative design lifecycle the risk question is some form of “What are the risks from this design on the patient/process.” As part of risk management, the question is “What are the risks to SISPQ/GMP from introducing the change.”

This risk assessment is narrower, in that it looks at the process of implementing. Broader that it looks at the entirety of your operations: facility, supply chain, quality system, etc.

The design risk assessment and risk management activities informs the change management risk assessment, but it cannot replace them. They also can serve to lower the rigor of the change management risk assessment, allowing the use of a less formal tool.

Living Risk Reviews

risk leads to change

In the third phase of risk management – risk review – we confirm that the risks identified and mitigated as planned and are functioning as intended. We also evaluate to see if any additional, previously unpredicted risks have appeared. Risk review is the living part of the lifecycle as we return to it on a periodic basis.

From this will come new mitigations, targeted to address the identified risks. These mitigations inevitably lead to change management.

We again do a new risk assessment focusing on the risk of implementing the change. Informed by the living risk assessment, we can often utilize a less formal tool to look at the full ramifications of introducing the mitigation (a change).

Change Controls contains Risk Management

risk and change management connections

Effective change management is enabled by risk management.

Each and every change requires a risk assessment to capture the risks of the change. This ICHQ10 requirement is the best way to determine if the change is acceptable.

This risk assessment evaluates the impact on the change on the facility, equipment, materials, supply chain, processes. testing, quality systems and everything else. It is one of the critical reasons it is crucial to involve the right experts.

From this risk assessment comes the appropriate actions before implementing the change, as well as appropriate follow-up activities and it can help define the effectiveness review.

What about grouped change controls?

Depends. Sometimes the risk management looks at the individual implementations. Othertimes you need to do separate ones. Many times the risk assessment lead you to breaking up one change control into many. Evaluate as follows:

  • Are the risks from the separate implementations appropriately captured
  • Are the risks from pauses between implementations appropriately captured
  • As the ripples appropriately understood

Change Management Leads back to Risk Management

Sometimes a change control requires a specific risk assessment to be updated, or requires specific risk management to happen.

What about HAACP?

Hazard Analysis Critical Control Point (HACCP) are great tools for risk assessments. They are often the catalyst for doing a change, they are often the artifact of a change. They should never be utilized for determining the impact of a change.

A hazard is any biological, chemical, or physical property that impacts human safety. The HAACP identifies and establishes critical limits. But a HAACP is not the tool to use to determine if a change should move forward and what actions to do. It is to static.

In Closing

Risk Management is an enabler for change, a tenet enshrined in the ICH guidances. We are engaging in risk management activities throughout our organizations. It is critical to understand how the various risk management activities fit together and how they should be separated.

Evolved Expendable Launch Vehicle (EELV) Quality Management

We determined that ULA, SpaceX, and AR were not performing adequate quality assurance management for the EELV program as evidenced by the 181 nonconformities to the AS9100C at the EELV contractor production facilities. This inadequate quality assurance management could increase costs, delay launch schedules, and increase the risk of mission failure.

From ”

Evaluation of the Evolved Expendable Launch Vehicle Program Quality Management System DODIG-2018-045, Department of Defense Office of Inspector General

It is useful to read audit reports and inspection findings from multiple industries. From this we can see trends, make connections and learn.

I see a few things that stand out.

DOD findings

Risk Register

Our evaluation of the RIO database showed that 11 out of 26 risks related to either Atlas V or Delta IV launch vehicle were in “red” status, which indicates that risk mitigation was behind schedule.

It is not enough to identify risks (though that is a critical place to start). You just can’t track them (though again, if you don’t track it you don’t see it). You actually have to have clear plans to mitigate and eliminate the risks. And this is where the program seemed to fall short.

Not a surprise. I think a lot of companies are having these difficulties. In the pharma world the regulatory agencies have been signaling pretty strongly that this is an issue.

Make sure you identify risks, track them, and have plans that are actually carried out to remediate.

Configuration Management

SpaceX failed to comply with AS9100C, section 7.1.3, which requires it to “establish, implement, and maintain a configuration management process.” Configuration management is a controlled process to establish the baseline configuration of a product and any changes to that product. This process should occur during the entire life cycle of a product to provide visibility and control of its physical, functional, and performance attributes.

First rule of reading inspection reports: Things probably went bad if a section starts with standard review 101 material.

That said, hello change management my dear friend.

This was the gist of my ASQ WCQI workshop last May, every industry needs good change management and change control.

Material Management

ULA and AR failed to comply with AS9100C, section 8.3, which requires them to “ensure that product which does not conform to product requirements are identified and controlled to prevent its unintended use or delivery.”

At ULA, we found 18 expired limited-life material items that were between 32 and 992 days past their expiration dates, but available for use on EELV flight hardware. This material should have been impounded and dispositioned. The use of expired limited-life items, such as glues and bonding agents, could result in product that does not meet specifications and may require costly rework.

I find it hard to believe that these companies aren’t tracking inventory. If they are tracking inventory and have any sort of cycle count process then the mechanism exists to ensure expired material is removed from the possibility of use. And yet we still see these observations across the pharma industry as well.

Concluding Thoughts

Quality Management has it its core the same principles, no matter the industry. We use similar tools. Leverage the best practices out there. Read about stresses other companies are having, learn from them and remediate at your own organization.

Contamination Control, Risk Management and Change Control

Microbiologists won’t be sequestered in the laboratory, running samples and conducting environmental testing, once the revisions proposed for Annex 1 of the EU and Pharmaceutical Inspection Cooperation Scheme (PIC/S) GMP guides take effect, Annex 1 rapporteur Andrew Hopkins said Oct. 15.

They will have a broader role that includes conducting risk assessments to ensure that sterile products are made as contamination-free as possible, said Hopkins, who is an inspector for the UK Medicines and Healthcare products Regulatory Agency.

Pink Sheet “EU GMP Annex 1 Would Give Microbiologists A Greater Role In Sterility Assurance, Rapporteur Says

Contamination Control is a fairly wide term used to mean “getting microbiologists out of the lab” and involved in risk management and compliance. Our organization splits that function off from the QC Microbiology organization but there are many models for making it work.

Risk Management is a major part of the new Annex 1, and what they are driving at are good risk assessments with good risk mitigation that involve the microbiologists.

living risk assessments

This is really what is meant by a contamination control strategy which considers the product and process knowledge and skills in pharmaceutical product manufacturing and GMP/ cGMP compliance under the auspices of a Pharmaceutical Quality System (Q10) together with initiatives of Quality by Design (Q8) and Quality Risk Management (Q9).

From this strategy comes:

  • Targeted/ risk based measures of contamination avoidance
  • Key performance indicators to assess status of contamination control
  • A defined strategy for deviation management (investigations) and CAPA

environmental monitoring

When it comes to change management, one of the easiest places to go wrong is to forget to bring the microbiologist in to changes. Based on your strategy you can determine change changes require their assessment and include it in the tool utilized to determine SMEs, for example:

Department Required if the change meets any of the following criteria:
Contamination Control The change impacts environment integrity, conditions or monitoring, including:

  • Changes to a controlled room or area that impact integrity
  • Changes in sampling methodology
  • Construction activities
  • Changes in personnel or material flow
  • The change will result in or modify exposure of product to the environment.

The change can impact microbiological control within a process stream, raw material or process equipment

The changes are to water systems

Don’t Just Tell Employees Organizational Changes Are Coming — Explain Why

To be successful, your story needs to start with the company’s core mission and then offer a compelling and inspiring future vision. You want to answer: How are the changes you make today helping you achieve your vision for tomorrow?
Don’t Just Tell Employees Organizational Changes Are Coming — Explain Why by Morgan Galbraith

I can’t stress enough the importance of proper communication around all changes, from the large transformations on down. Effective communication is effective change management.

I’ve discussed the need to be able to identify changes to strategic plans and use that to inspire, inform, empower, and engage.

changing business environment

Always spend the time on a good communication plan:

Information to Communicate
(What)
Objective
(Why)
Target Audience
(Who to)
Frequency
(When)
Start Date
(When)
End Date
(When)
Media
(How)
Responsible
(Who from)
Deliverable Comments
What to people need to know o Determine site readiness to start the project

o Define resource needs and availability

Tailor the communication to specific audiences. The same information is sometimes presented different ways How often? Start date End Date From face-to-face to all the other communication tools available in the modern workplace. Be creative Who is responsible for completing the communication What will execution look like  

 

Change Management of multi-site implementations

A colleague asks in response to my post Group change controls:

… deploying a Learning + documentation system … all around the word [as a global deployment]  … do we I initiate a GLOBAL CC or does each site created a local CC.

The answer is usually, in my experience, both.

Change management is about process, organization, technology and people. Any change control needs to capture the actions necessary to successful implement the change.

so at implementation I would do two sets of changes. A global to capture all the global level changes and to implement the new (hopefully) harmonized system And then a local change control at each site to capture all the site impact.

System Element Global Local
Process Introduce the new global process

Update all global standards, procedures, etc

How will local procedures change? How will local system interactions change – clean up all the local procedures to ensure the point to the new global procedures and are harmonized as necessary.
Technology Computer system validation

Global interfaces

Global migration strategy

Local interfaces (if any) and configurations

Are local technologies being replaced? Plan for decommissioning.

Local migration (tactical)

People What do people do on the global level?

How will people interact within the system in the future?

Global training

What will be different for people at each individual site?

Localized training

Organization Will there be new organizational structures in place? Is this system being run out of a global group? How will communication be run.

System governance and change management

Site organization changes

How will different organizations and sub organizations adopt, adapt and work with the system

If you just have a global change control you are at real risk of missing a ton of local uniqueness and leaving in place a bunch of old ways of thinking and doing things.

If you just do local change controls you will be at risk of not seeing the big picture and getting the full benefits of harmonization. You also will probably have way too many change controls that regurgitate the same content, and then are at risk of divergence – a compliance nightmare.

This structure allows you better capture the diversity of perspectives at the sites. A global change control tends to be dominated by the folks at each site who own the system (all your documents and training folks in this example), while a site change will hopefully include other functions, such as engineering and operations. Trust me, they will have all sorts of impact.

This structure also allows you to have rolling implementations. The global implements when the technology is validated and the core processes are effective. each site then can implement based on their site deliverables. useful when deploying a document management system and you have a lot of migration.

Multisite changes

As part of the deployment make sure to think through matters of governance, especially change management. Once deployed it is easy to imagine many changes just needing a central change control. But be sure to have thought through the criteria that will require site change controls – such as impact other interrelated systems, site validation or different implementation dates.

I’ve done a lot of changes and a lot of deployment of systems. This structure has always worked well. I’ve never done just a global and been happy with the final results, they always leave too much unchanged elements behind that come back to haunt you. In the last year I’ve done 2 major changes to great success with this model, and seen one where the decision not to use this model has left us with lots of little messes to clean up.

As a final comment, keep the questions coming and I would love to hear other folks perspectives on these matters. I’m perpetually learning and I know there are lots of permutations to explore.

Measures of success for changes

A colleague asks:

Is it a compliance risk to extend timelines on a change control?

I want to take a step back to an important fundamental of change management to answer this question. All changes are done to realize strategic purposes; a good change management system is all about accelerating change. From the big transformations to the emergency changes to keep product being made each and every change has a strategic goal.

changing business environment

From this alignment to the strategy, each change has success metrics. Success metrics include economic, quality, technical and organization (among others) and they drive the how and the when of our change.

For example, a change driven by a CAPA to prevent reoccurrence will potentially have a different timeline than a change tied to a strategic goal to leverage a new way of working. But both have timelines driven by strategic to the tactical needs, usually filtered through a risk based prioritization tool.

And sometimes these change. The compliance aspect is not so much did you extend, it’s did you know what was happening with the change control in enough time to influence it in such a way to assure meeting the how.

The KPIs and other measures built into your system should monitor and ensure your changes reach the intended benefits.

manage for success

To return to the original question. Unlike deviations/conformances where there is a specific requirements to complete in a timely way, and CAPAs where the root cause needs to be dealt with as soon as possible, change controls have their own internal timeline based on the drivers (which may be a CAPA). Extensions are not bad in a specific one-by-one change control approach. Instead they are indicative of larger troubles in the system and should be dealt with holistically to ensure you get the maximum benefit from your changes in the best possible time.