Change Management – Post Change Evaluation and Action

In “Change Management – Post Change Evaluation and Action” John Hunter (of CuriousCat) writes a nice post on the The W. Edwards Deming Institute Blog on linking change management to the PDSA (PDCA) lifecycle focusing on Act.

Post Change Evaluation is often called the effectiveness review, and is a critical part of change in the pharmaceutical quality system, and frankly is important no matter the industry.

An effectiveness review is the success criteria of the change viewed over enough data points based on a methodology informed by the nature of the change and risk.

 The success criteria should be achieved. If not, reasons why they have not been achieved should be assessed along with the mitigation steps to address the reasons why, including reverting to the previous operating state where appropriate. This may require the proposal of a subsequent change or amendment of the implementation plan to ensure success. Here we see the loop aspects of the PDSA lifecycle.

All changes should have a way back into knowledge management. The knowledge gathered from implementation of the change should be shared with the development function and other locations, as appropriate, to ensure that learning can be applied in products under development or to similar products manufactured at the same or other locations.

When choosing success criteria always strive for leading indicators that tell you how the change is working. Deviations are an awful way to judge the effectiveness of the change. Instead look for walkthroughs, checklists, audits, data gathering. Direct observation and real-time gathering and analysis of data of any sort is the best.

As mentioned above, ensure the change management/change control system is set up to deal with the inevitable change that does not work. Have a clear set of instructions on how to make that decision (returning to the success criteria), what steps to take to mitigate and what to do next. For example having guidance of when to create a deviation and on how to make a decision to rollback versus implement another change.

FDA signals – no such thing as a planned deviation

The subject of planned deviations made for a raucous “Breakfast with the FDA” session Sept. 25 at the Parenteral Drug Association/FDA conference in Washington.

These are the deviations from standard operating procedures that workers carry out on purpose, typically to keep a pharmaceutical plant operating when for one reason or another they won’t be doing it the way the company said they would.

FDA Compliance Experts Advise Against Treating Minor Changes As ‘Planned Deviations’” – Bowman Cox, Pink Sheet

I wish I had gone to the PDA/FDA conference this year, if for nothing else to have been able to stand up and cheer wildly when this was said:

Brooke Higgins, a senior policy advisor in the FDA drug center’s Office of Compliance, agreed that “it’s a very strange term, and it kind of makes your skin crawl a little bit.”

There is a whole lot more good stuff over at the Pink Sheet’s summary.

I am a firm believer that there are no such things as planned changes. There are change controls, some of which are temporary, occasionally even ones that are retroactive (deviation identifies a change which is formalized in a change control). But all are through the same system, with the same evaluations and assessments and the same sorts of actions.

Keep all changes together. Its a true best practice.

 

Change Management of multi-site implementations

A colleague asks in response to my post Group change controls:

… deploying a Learning + documentation system … all around the word [as a global deployment]  … do we I initiate a GLOBAL CC or does each site created a local CC.

The answer is usually, in my experience, both.

Change management is about process, organization, technology and people. Any change control needs to capture the actions necessary to successful implement the change.

so at implementation I would do two sets of changes. A global to capture all the global level changes and to implement the new (hopefully) harmonized system And then a local change control at each site to capture all the site impact.

System Element Global Local
Process Introduce the new global process

Update all global standards, procedures, etc

How will local procedures change? How will local system interactions change – clean up all the local procedures to ensure the point to the new global procedures and are harmonized as necessary.
Technology Computer system validation

Global interfaces

Global migration strategy

Local interfaces (if any) and configurations

Are local technologies being replaced? Plan for decommissioning.

Local migration (tactical)

People What do people do on the global level?

How will people interact within the system in the future?

Global training

What will be different for people at each individual site?

Localized training

Organization Will there be new organizational structures in place? Is this system being run out of a global group? How will communication be run.

System governance and change management

Site organization changes

How will different organizations and sub organizations adopt, adapt and work with the system

If you just have a global change control you are at real risk of missing a ton of local uniqueness and leaving in place a bunch of old ways of thinking and doing things.

If you just do local change controls you will be at risk of not seeing the big picture and getting the full benefits of harmonization. You also will probably have way too many change controls that regurgitate the same content, and then are at risk of divergence – a compliance nightmare.

This structure allows you better capture the diversity of perspectives at the sites. A global change control tends to be dominated by the folks at each site who own the system (all your documents and training folks in this example), while a site change will hopefully include other functions, such as engineering and operations. Trust me, they will have all sorts of impact.

This structure also allows you to have rolling implementations. The global implements when the technology is validated and the core processes are effective. each site then can implement based on their site deliverables. useful when deploying a document management system and you have a lot of migration.

Multisite changes

As part of the deployment make sure to think through matters of governance, especially change management. Once deployed it is easy to imagine many changes just needing a central change control. But be sure to have thought through the criteria that will require site change controls – such as impact other interrelated systems, site validation or different implementation dates.

I’ve done a lot of changes and a lot of deployment of systems. This structure has always worked well. I’ve never done just a global and been happy with the final results, they always leave too much unchanged elements behind that come back to haunt you. In the last year I’ve done 2 major changes to great success with this model, and seen one where the decision not to use this model has left us with lots of little messes to clean up.

As a final comment, keep the questions coming and I would love to hear other folks perspectives on these matters. I’m perpetually learning and I know there are lots of permutations to explore.

Measures of success for changes

A colleague asks:

Is it a compliance risk to extend timelines on a change control?

I want to take a step back to an important fundamental of change management to answer this question. All changes are done to realize strategic purposes; a good change management system is all about accelerating change. From the big transformations to the emergency changes to keep product being made each and every change has a strategic goal.

changing business environment

From this alignment to the strategy, each change has success metrics. Success metrics include economic, quality, technical and organization (among others) and they drive the how and the when of our change.

For example, a change driven by a CAPA to prevent reoccurrence will potentially have a different timeline than a change tied to a strategic goal to leverage a new way of working. But both have timelines driven by strategic to the tactical needs, usually filtered through a risk based prioritization tool.

And sometimes these change. The compliance aspect is not so much did you extend, it’s did you know what was happening with the change control in enough time to influence it in such a way to assure meeting the how.

The KPIs and other measures built into your system should monitor and ensure your changes reach the intended benefits.

manage for success

To return to the original question. Unlike deviations/conformances where there is a specific requirements to complete in a timely way, and CAPAs where the root cause needs to be dealt with as soon as possible, change controls have their own internal timeline based on the drivers (which may be a CAPA). Extensions are not bad in a specific one-by-one change control approach. Instead they are indicative of larger troubles in the system and should be dealt with holistically to ensure you get the maximum benefit from your changes in the best possible time.

Group change controls

A colleague asks:

hoping you can provide your perspective on “grouping changes”. That is, rules or guidance on when it is appropriate to lump several small changes into one change control.

There are lots of reason’s to want to group changes into one change control – ease of implementation, a perception of “reducing work” or just convenience. I apply three general rules for when this is a good idea:

  1. Implementation
  2. Release of product
  3. Concreteness of work package

Implementation

All changes should have the same  “Change in User” and “Regulatory Approval”

I talked about this is in “Changes become effective“:

Tell and Do Do and Tell Do and Report
Change in use After regulatory approval. When change is introduced to the ‘floor’ When change is introduced to the ‘floor’ When change is introduced to the ‘floor’
Regulatory approval Upon approvals After use, before send to market Upon reporting frequency (annual, within 6 months, within 1 year)
Sent to market After regulatory approval and change in use After regulatory approval and change in use After change in use

Ideally chose all changes that are “Do and Tell” or “Do and report”, I strongly do not recommend blending “Tell and Do,” there are just so many complexities here and it can lead to a constant revision of change controls and even increase risk of inadvertently sending products to the market.

Release of Product

All changes bundled together need to be releasable at the same time. Look at PQ/PV requirements and other testing that can be different and keep them in segregated change controls. Putting product on stability (or even needing stability data) is another good reason to separate changes.

Similarly, keep changes that have different internal product segregation requirements unbundled.

Concreteness of Work Bundles

The best reason to bundle changes is that they involve changing the same things: same documents, same automation, same piece of equipment. Once the bundling starts increasing scope of the change, it is time to have separate change controls.

If bundling changes starts changing your project management triangle (resources, time, scope)  then do a deep evaluation. You probably have separate change controls, even if they are under the same project.

Evaluate effectiveness reviews. Chances are totally separate effectiveness reviews for same impacted area are really separate changes.

What about risks?

You might notice that I do not call out risks as one of the criteria. Risks drive mitigations, which drive impact to the three criteria I gave. If a high risk’s mitigation does not trigger one of these criteria, it is usually not a problem to group the changes together. Though I’d pause and consider why the addition of a change that adds high risks does not change drive mitigations that change the amount of work I am doing and ensure that is appropriately documented.

Conclusion

Sometimes bundling changes make sense. other times you will curse the day you agreed to bundle. I find these three rules provide a framework for making the decision.

Change Control- Leveraging regulatory inspection data

The Pfizer McPherson site has been under a great deal of regulatory scrutiny, and as a result there is a lot we can learn from their findings.

In July the MHRA stated the following:

Hospira McPherson Changes

There is a lot to unpack here, and for most of it I can pull up some previous postings to start with:

Breaking down change controls is both a necessity and a difficulty. I talked about the need for a change strategy when breaking up changes. This connective tissue will help with issues like 2.4.1.1 above and can also serve as a good playbook for discussing the changes with an inspector. This is especially important when you find you need to implement related changes at different times. I talked about the various implementation dates in some detail.

Risk assessments are only getting more important, and for a company with international distribution it is important to consider the risks inherent to your regulatory strategy and distribution strategy and mitigate.

regulatory and change

If you have changes that will have long tails of regulatory approvals, then your change control needs to have the right controls to ensure appropriate and safe supply.

Build your actions to address all risks and impacts and ensure they are appropriately carried through.

action items

Finally ensure your change control process has a way to revise the plan and ensure all stakeholders are included in the decisions.

 

Computer system changes and the GAMP5 framework

Appropriate controls shall be exercised over computer or related systems to assure that changes in master production and control records or other records are instituted only by authorized personnel. Input to and output from the computer or related system of formulas or other records or data shall be checked for accuracy. The degree and frequency of input/output verification shall be based on the complexity and reliability of the computer or related system. A backup file of data entered into the computer or related system shall be maintained except where certain data, such as calculations performed in connection with laboratory analysis, are eliminated by computerization or other automated processes. In such instances a written record of the program shall be maintained along with appropriate validation data. Hard copy or alternative systems, such as duplicates, tapes, or microfilm, designed to assure that backup data are exact and complete and that it is secure from alteration, inadvertent erasures, or loss shall be maintained.

21 CFR 211.68(b)

Kris Kelly over at Advantu got me thinking about GAMP5 today.  As a result I went to the FDA’s Inspection Observations page and was quickly reminded me that in 2017 one of the top ten highest citations was against 211.68(b), with the largest frequency being “Appropriate controls are not exercised over computers or related systems to assure that changes in master production and control records or other records are instituted only by authorized personnel. ”

Similar requirements are found throughout the regulations of all major markets (for example EU 5.25) and data integrity is a big piece of this pie.

So yes, GAMP5 is probably one of your best tools for computer system validation. But this is also an argument for having one change management system/one change control process.

When building your change management system remember that your change is both a change to a validated change and a change to a process, and needs to go through the same appropriate rigor on both ends. Companies continue to get in a lot of trouble on this. Especially when you add in the impact of master data.

Make sure your IT organization is fully aligned. There’s a tendency at many companies (including mine) to build walls between an ITIL orientated change process and process changes. This needs to be driven by a risk based approach, and find the opportunities to tear down walls. I’m spending a lot of my time finding ways to do this, and to be honest, worry that there aren’t enough folks on the IT side of the fence willing to help tear down the fence.

So yes, GAMP5 is a great tool. Maybe one of the best frameworks we have available.

gamp5