ASQ Audit Conference – Day 1 Afternoon

I presented on change management and then I spent the afternoon focusing more on ASQ member leader stuff. So not much to report on sessions.

My session, Lessons on Change Management went well. I probably should have cut the slides way back instead of re-purposing slides from a longer presentation, but I think I hit a lot of key points and hopefully it was valuable for folks.

I ended up working the FDC Division table after that, so I skipped the final session of the day. Probably best, after presenting its always hard for me to focus for a little while.

Tomorrow is a full day, and I present on data integrity.

Lessons Learned and Change Management

One of the hallmarks of a quality culture is learning from our past experiences, to eliminate repeat mistakes and to reproduce success. The more times you do an activity, the more you learn, and the better you get (within limits for simple activities).  Knowledge management is an enabler of quality systems, in part, to focus on learning and thus accelerate learning across the organization as a whole, and not just one person or a team.

This is where the” lessons learned” process comes in.  There are a lot of definitions of lessons learned out there, but the definition I keep returning to is that a lessons learned is a change in personal or organizational behavior as a result from learning from experience. Ideally, this is a permanent, institutionalized change, and this is often where our quality systems can really drive continuous improvement.

Lessons learned is activity to lessons identified to updated processes
Lessons Learned

Part of Knowledge Management

The lessons learned process is an application of knowledge management.

Lessons identified is generate, assess, and share.

Updated processes (and documents) is contextualize, apply and update.

Lessons Learned in the Context of Knowledge Management

Identify Lessons Learned

Identifying lessons needs to be done regularly, the closer to actual change management and control activities the better. The formality of this exercise depends on the scale of the change. There are basically a few major forms:

  • After action reviews: held daily (or other regular cycle) for high intensity learning. Tends to be very focused on questions of the day.
  • Retrospective: Held at specific periods (for example project gates or change control status changes. Tends to have a specific focus on a single project.
  • Consistency discussions: Held periodically among a community of practice, such as quality reviewers or multiple site process owners. This form looks holistically at all changes over a period of time (weekly, monthly, quarterly). Very effective when linked to a set of leading and lagging indicators.
  • Incident and events: Deviations happen. Make sure you learn the lessons and implement solutions.

The chosen formality should be based on the level of change. A healthy organization will be utilizing all of these.

Level of ChangeForm of Lesson Learned
TransactionalConsistency discussion
After action (when things go wrong)
OrganizationalRetrospective
After action (weekly, daily as needed)
TransformationalRetrospective
After action (daily)

Successful lessons learned:

  • Are based on solid performance data: Based on facts and the analysis of facts.
  • Look at positive and negative experiences.
  • Refer back to the change management process, objectives of the change, and other success criteria
  • Separate experience from opinion as much as possible. A lesson arises from actual experience and is an objective reflection on the results.
  • Generate distinct lessons from which others can learn and take action. A good action avoids generalities.

In practice there are a lot of similarities between the techniques to facilitate a good lessons learned and a root cause analysis. Start with a good core of questions, starting with the what:

  • What were some of the key issues?
  • What were the success factors?
  • What worked well?
  • What did not work well?
  • What were the challenges and pitfalls?
  • What would you approach differently if you ever did this again?

From these what questions, we can continue to narrow in on the learnings by asking why and how questions. Ask open questions, and utilize all the techniques of root cause analysis here.

Then once you are at (or close) to a defined issue for the learning (a root cause), ask a future-tense question to make it actionable, such as:

  • What would your advice be for someone doing this in the future?
  • What would you do next time?

Press for specifics. if it is not actionable it is not really a learning.

Update the Process

Learning implies memory, and an organization’s memories usually require procedures, job aids and other tools to be updated and created. In short, lessons should evolve your process. This is often the responsibility of the change management process owner. You need to make sure the lesson actually takes hold.

Differences between effectiveness reviews and lesson’s learned

There are three things to answer in every change

  1. Was the change effective – did it meet the intended purposes
  2. Did the change have any unexpected effects
  3. What can we learn from this change for the next change?

Effectiveness reviews are 1 and 2 (based on a risk based approach) while lessons learned is 3. Lessons learned contributes to the health of the system and drives continuous improvements in the how we make changes.

Citations

  • Lesson learned management model for solving incidents. (2017). 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), Information Systems and Technologies (CISTI), 2017 12th Iberian Conference On, 1.
  • Fowlin, J. j & Cennamo, K. (2017). Approaching Knowledge Management Through the Lens of the Knowledge Life Cycle: a Case Study Investigation. TechTrends: Linking Research & Practice to Improve Learning61(1), 55–64. 
  • Michell, V., & McKenzie, J. (2017). Lessons learned: Structuring knowledge codification and abstraction to provide meaningful information for learning. VINE: The Journal of Information & Knowledge Management Systems47(3), 411–428.
  • Milton, N. J. (2010). The Lessons Learned Handbook : Practical Approaches to Learning From Experience. Burlington: Chandos Publishing.
  • Paul R. Carlile. (2004). Transferring, Translating, and Transforming: An Integrative Framework for Managing Knowledge across Boundaries. Organization Science, (5), 555.
  • Secchi, P. (Ed.) (1999). Proceedings of Alerts and Lessons Learned: An Effective way to prevent failures and problems. Technical Report WPP-167. Noordwijk, The Netherlands: ESTEC

Change Management – Post Change Evaluation and Action

In “Change Management – Post Change Evaluation and Action” John Hunter (of CuriousCat) writes a nice post on the The W. Edwards Deming Institute Blog on linking change management to the PDSA (PDCA) lifecycle focusing on Act.

Post Change Evaluation is often called the effectiveness review, and is a critical part of change in the pharmaceutical quality system, and frankly is important no matter the industry.

An effectiveness review is the success criteria of the change viewed over enough data points based on a methodology informed by the nature of the change and risk.

 The success criteria should be achieved. If not, reasons why they have not been achieved should be assessed along with the mitigation steps to address the reasons why, including reverting to the previous operating state where appropriate. This may require the proposal of a subsequent change or amendment of the implementation plan to ensure success. Here we see the loop aspects of the PDSA lifecycle.

All changes should have a way back into knowledge management. The knowledge gathered from implementation of the change should be shared with the development function and other locations, as appropriate, to ensure that learning can be applied in products under development or to similar products manufactured at the same or other locations.

When choosing success criteria always strive for leading indicators that tell you how the change is working. Deviations are an awful way to judge the effectiveness of the change. Instead look for walkthroughs, checklists, audits, data gathering. Direct observation and real-time gathering and analysis of data of any sort is the best.

As mentioned above, ensure the change management/change control system is set up to deal with the inevitable change that does not work. Have a clear set of instructions on how to make that decision (returning to the success criteria), what steps to take to mitigate and what to do next. For example having guidance of when to create a deviation and on how to make a decision to rollback versus implement another change.

FDA signals – no such thing as a planned deviation

The subject of planned deviations made for a raucous “Breakfast with the FDA” session Sept. 25 at the Parenteral Drug Association/FDA conference in Washington.

These are the deviations from standard operating procedures that workers carry out on purpose, typically to keep a pharmaceutical plant operating when for one reason or another they won’t be doing it the way the company said they would.

FDA Compliance Experts Advise Against Treating Minor Changes As ‘Planned Deviations’” – Bowman Cox, Pink Sheet

I wish I had gone to the PDA/FDA conference this year, if for nothing else to have been able to stand up and cheer wildly when this was said:

Brooke Higgins, a senior policy advisor in the FDA drug center’s Office of Compliance, agreed that “it’s a very strange term, and it kind of makes your skin crawl a little bit.”

There is a whole lot more good stuff over at the Pink Sheet’s summary.

I am a firm believer that there are no such things as planned changes. There are change controls, some of which are temporary, occasionally even ones that are retroactive (deviation identifies a change which is formalized in a change control). But all are through the same system, with the same evaluations and assessments and the same sorts of actions.

Keep all changes together. Its a true best practice.

 

Change Management of multi-site implementations

A colleague asks in response to my post Group change controls:

… deploying a Learning + documentation system … all around the word [as a global deployment]  … do we I initiate a GLOBAL CC or does each site created a local CC.

The answer is usually, in my experience, both.

Change management is about process, organization, technology and people. Any change control needs to capture the actions necessary to successful implement the change.

so at implementation I would do two sets of changes. A global to capture all the global level changes and to implement the new (hopefully) harmonized system And then a local change control at each site to capture all the site impact.

System Element Global Local
Process Introduce the new global process

Update all global standards, procedures, etc

How will local procedures change? How will local system interactions change – clean up all the local procedures to ensure the point to the new global procedures and are harmonized as necessary.
Technology Computer system validation

Global interfaces

Global migration strategy

Local interfaces (if any) and configurations

Are local technologies being replaced? Plan for decommissioning.

Local migration (tactical)

People What do people do on the global level?

How will people interact within the system in the future?

Global training

What will be different for people at each individual site?

Localized training

Organization Will there be new organizational structures in place? Is this system being run out of a global group? How will communication be run.

System governance and change management

Site organization changes

How will different organizations and sub organizations adopt, adapt and work with the system

If you just have a global change control you are at real risk of missing a ton of local uniqueness and leaving in place a bunch of old ways of thinking and doing things.

If you just do local change controls you will be at risk of not seeing the big picture and getting the full benefits of harmonization. You also will probably have way too many change controls that regurgitate the same content, and then are at risk of divergence – a compliance nightmare.

This structure allows you better capture the diversity of perspectives at the sites. A global change control tends to be dominated by the folks at each site who own the system (all your documents and training folks in this example), while a site change will hopefully include other functions, such as engineering and operations. Trust me, they will have all sorts of impact.

This structure also allows you to have rolling implementations. The global implements when the technology is validated and the core processes are effective. each site then can implement based on their site deliverables. useful when deploying a document management system and you have a lot of migration.

Multisite changes

As part of the deployment make sure to think through matters of governance, especially change management. Once deployed it is easy to imagine many changes just needing a central change control. But be sure to have thought through the criteria that will require site change controls – such as impact other interrelated systems, site validation or different implementation dates.

I’ve done a lot of changes and a lot of deployment of systems. This structure has always worked well. I’ve never done just a global and been happy with the final results, they always leave too much unchanged elements behind that come back to haunt you. In the last year I’ve done 2 major changes to great success with this model, and seen one where the decision not to use this model has left us with lots of little messes to clean up.

As a final comment, keep the questions coming and I would love to hear other folks perspectives on these matters. I’m perpetually learning and I know there are lots of permutations to explore.