Master and Transactional Data Management

Mylan’s 483 observation states that changes were being made to a LIMS system outside of the site’s change control process.

This should obviously be read in light of data integrity requirements. And it looks like in this case there was no way to produce a list of changes, which is a big audit trail no-no.

It’s also an area where I’ve seen a lot of folks make miss-steps, and frankly, I’m not sure I’ve always got it right.

There is a real tendency to look at the use of our enterprise systems and want all actions and approvals to happen within the system. This makes sense, we want to reduce our touch points, but there are some important items to consider before moving ahead with that approach.

Changes control is about assessing, handling and releasing the change. Most importantly it is in light the validated and regulatory impact. It serves disposition. As such, it is a good thing to streamline our changes into one system. To ensure every change gets assessed equally, and then gets the right level of handling it needs, and has a proper release.

Allowing a computer system to balkanize your changes, in the end, doesn’t really simplify. And in this day of master data management, of heavily aligned and talking systems, to be nimble requires us to know with a high degree of certainty that when we apply a change we are applying it thoroughly.

The day of separated computer systems is long over. It is important that our change management system takes that into account and offers single-stop shopping.

Mylan gets a 32 page 483

The FDA’s April 483 for Mylan Pharmaceuticals has been in the fore-front of a lot of conversations in the last week. Let’s be honest, the FDA posts a 32 page, 13 observation 483 report on any manufacturer and it will be news. One as prominent as Mylan and doubly so. On the same day, the FDA also posted a 2016 483 and 2017 warning letter against a Mylan facility in India.

The 483 is a hit parade of observations, like the 1st observation of failure of the quality unit, including a reference to lack of quality approval of change controls.

What everyone has been intensely focusing on is the strong emphasis on cleaning, with 11 pages dedicated to failures in cleaning validation.

Which to be frank, is a big deal in a multi-product facility.

Read the 483, and when doing so evaluate your site’s cleaning program. Ask yourself some of these questions:

Are there appropriate cleaning procedures in place for all products-contact equipment, product contact accessories?
Are there appropriate cleaning procedures in place for facility cleaning (dispensing, sampling room…)?
Do your procedures include the sequence of the cleaning activities? Is it significantly detailed?
Do the procedures address the different scenarios (cleaning between different batches of the same product, cleaning between products changes, holding time before and after cleaning…)?
Do the procedures address who is responsible for performing the cleaning?
Does the validation study, the acceptance criteria and when revalidation justification and keys documentation approved by Quality? Does it include a clear status on the cleaning process?
Is the strategy used for the cleaning validation clearly established? (matrix approach, dedicated equipment, worst case scenario, grouping equipment, equipment train…)
Are batches that come after the cleaning validation run, released after completion of the cleaning validation?
Are the acceptance criteria (products, detergents, cleaning agent, micro… ) scientifically established and followed? Do these acceptance criteria include a safety margin?

Approval of cleaning validation is a key responsibility of the quality unit that involves some very specific requirements. These requirements should be built into the quality systems, including validation, deviation, and change management.

Likelihood of occurrence in risk estimation

People use imprecise words to describe the chance of events all the time — “It’s likely to rain,” or “There’s a real possibility they’ll launch before us,” or “It’s doubtful the nurses will strike.” Not only are such probabilistic terms subjective, but they also can have widely different interpretations. One person’s “pretty likely” is another’s “far from certain.” Our research shows just how broad these gaps in understanding can be and the types of problems that can flow from these differences in interpretation.

“If You Say Something Is “Likely,” How Likely Do People Think It Is?” by by Andrew Mauboussin and Michael J. Mauboussin

Risk estimation is based on two components:

The probability of the occurrence of harm
The consequences of that harm

With a third element of detectability of the harm being used in many tools.

Often-times we simplify probability of the occurrence into likelihood. The quoted article above is a good simple primer on why we should be careful of that. It offers three recommendations that I want to talk about. Go read the article and then come back.

I. Use probabilities instead of words to avoid misinterpretation

Avoid the simplified quality probability levels, such as “likely to happen”, “frequent”, “can happen, but not frequently”, “rare”, “remote”, and “unlikely to happen.” Instead determine probability levels. even if you are heavily using expert opinion to drive probabilities, given ranges of numbers such as “<10% of the time”, “20-60% of the time” and “greater than 60% of the time.”

It helps to have several sets of scales.

The article has an awesome graph that really is telling for why we should avoid words.

W180614_MAUBOUSSIN_HOWPEOPLE

II. Use structured approaches to set probabilities

Ideally pressure test these using a Delphi approach, or something similar like paired comparisons or absolute probability judgments. Using the historic data, and expert opinion, spend the time to make sure your probabilities actually capture the realities.

Be aware that when using historical data that if there is a very low frequent of occurrence historically, then any estimate of probability will be uncertain. In these cases its important to use predicative techniques and simulations. Monte Carlo anyone?

III. Seek feedback to improve your forecasting

Risk management is a lifecycle approach, and you need to be applying good knowledge management to that lifecycle. Have a mechanism to learn from the risk assessments you conduct, and feed that back into your scales. These scales should never be a once and done.

In Conclusion

Risk Management is not new. It’s been around long enough that many companies have the elements in place. What we need to be doing to driving to consistency. Drive out the vague and build best practices that will give the best results. When it comes to likelihood there is a wide body of research on the subject and we should be drawing from it as we work to improve our risk management.

Move beyond setting your scales at the beginning of a risk assessment. Scales should exist as a library (living) that are drawn upon for specific risk evaluations. This will help to ensure that all participants in the risk assessment have a working vocabulary of the criteria, and will keep us honest and prevent any intentional or unintentional manipulation of the criteria based on an expected outcome.

FDA Repays Industry by Rushing Risky Drugs to Market — ProPublica

As pharma companies underwrite three-fourths of the FDA’s budget for scientific reviews, the agency is increasingly fast-tracking expensive drugs with…
— Read on www.propublica.org/article/fda-repays-industry-by-rushing-risky-drugs-to-market

This is worth reading. I remember when I first started it was easier to get European approvals before US, and have been surprised by the switch over the last few years.

I also watch all these companies struggle with QbD and wonder if these two trends go hand in hand.

No answers from me, but I do recommend reading this article.

Risk Filtering – A popular tool that is easy to abuse

An article titled “ICE Modified Its ‘Risk Assessment’ software So It Automatically Recommends Detention” is probably guaranteed to reach me, for a myriad of ways.

I believe strongly in professional codes of conduct, and the need to speak out. In this case, I am thinking of two charges:

Hold paramount the safety, health, and welfare of individuals, the public, and the environment.
Avoid conduct that unjustly harms or threatens the reputation of the Society, its members, or the Quality profession.

Reading this article, and doing some digging, tells me that the tools of quality that I hold dear have been abused and I believe it is appropriate to call that out.

Now, a caveat, risk assessment, and management have some flavors out there and I’ll be honest that I once made the mistake of getting into a discussion with a risk management expert from a bank and realizing we had very different ideas of risk management. But supposedly we’re all aligned (sort of) to ISO Guide 73:2009, “Risk management. Vocabulary.” And as such, I’ll try to stick pretty close to those shared commonalities. I also assume that ISO Guide 73:2009 is a shared point between me and whoever designed the ICE risk assessment software.

Risk assessment is one phase in risk management, and I’ll focus on that here. Risk assessment is about identifying risk scenarios. What we do is:

Establish the context and environment that could present a risk
Identify the hazards and considering the hazards these risks could present
Analyze the risks, including an assessment of the various contributing factors
Evaluate and prioritize the risks in terms of further action required
Identify the range of options available to tackle the risks and decide how to implement risk management strategies.

A look at the decision making around this found in the Reuters article, leads me to believe that what ICE is using meets these criteria and we can call it a risk assessment (why it is in quotes in the Motherboard article mystifies me).

There are a lot of risk assessment tools out there. it is important to know that risk assessment is not perfect, and as a result, we are constantly developing better tools and refining the ones we have.

My guess is we are seeing a computerized use of the risk ranking and filtering tool here. Very popular, and something I’ve spent a great deal of time developing. This tool involves breaking a basic risk question down into as many components as needed to capture factors involved in the risk. These factors are then combined into a relative risk score for ranking. Filters are weighting factors used to scale the risks to objectives.

And that is where this tool can often go wrong. It appears ICE under the Trump administration has determined its objective is to jail everyone. By adjusting the filters, the tool easily drives to that conclusion. And this is a problem. Here we see a quality tool being used to excuse inhumane policy choices. It is not the ICE agents separating families and jail people over a misdemeanor, it is the tool. And if that doesn’t strike to the heart of the banality of evil concept I’m not sure what does.

I could go deeper into the tool, how I would have built it, the ways you validate the effectiveness of it. And that all probably will make an excellent follow-up someday. But the reason I’m writing this post is primarily that I read this article and it dawned on me that someone very similar to me in skill set probably created this tool. Someone who maybe I’ve sat across the table at a professional conference, who has read the same articles, probably debates the same qualitative vs. quantitative debates. And this is a great example of when its necessary to speak up and criticize a tool of my profession being used for evil. I probably will never talk to the team who developed this tool, but we all see instances of companies around us being asked to build similar applications, using the tools of our profession, that will be used for the wrong results. And we owe it to our code of ethics to refuse.