Ensures that metric analysis and reporting meet all stakeholder needs
Ensures that adequate and appropriate resources (e.g., funding, personnel, tools) are available to properly perform metrics implementation, collection, and ongoing support.
Ensures that appropriate change management activities are undertaken
This is one of those that can be done at several levels, and it usually has several cuts, from a top-level strategic document to the process owner level to potentially deeper cuts lower in the organization. I am a big fan of each process owner owning their parts and it passing up.
A common and distorted application of psychological safety is that it is somehow a shield from accountability. Non-performing employees tend to invoke it as an excuse for poor performance, insisting that a focus on psychological safety means valuing people and building relationships. That’s true, but stretching the premise, they claim that we should give them a pass when they don’t perform.
The flawed logic seems to be along the lines of because we may have used fear and intimidation, command and control, and manipulative and coercive tactics with people in the past, in an attempt to hold folks accountable, we must shed the artifacts that drive accountability in order to have an environment of psychological safety.
In my experience, this is especially prevalent when discussing metrics around overdue quality systems records and training. How to discuss what is late, can you even publish a list of folks with overdue training?
I want to be very clear, psychological safety is not a kind of diplomatic immunity from having to deliver results. It is not a shield from accountability.
Being held accountable can be looked at as transparency in progress. Psychological safety allows us to be vulnerable and to trust that the organization will take the problems seriously and address them.
FDA’s Office of Pharmaceutical Quality (OPQ) in the Center for Drug Evaluation and Research (CDER) recently published a white paper proposal on the development of a rating system to measure a firm’s quality management maturity (QMM) as a way to mitigate drugs shortages and enhance the quality of finished drug products. These ratings would be publicly available. This is very aligned in thought to the recent NAS study recommendations in a paper commissioned by Congress.
This fits nicely within the recent draft guidance on metrics, and the two are definitely meant to fit together.
I am a big advocate of this work. I definitely want to see the particulars, but this is a long time coming and greatly needed. Frankly, the best way to make them happen is to require the QMM to be a factor in purchasing decisions for Medicaid/Medicare and the Veterans Hospitals (and more if possible).
The agency will be holding two workshops on quality management maturity, on May 24 and May 25. The first workshop will address CDER’s QMM program and the second will discuss quality ratings.
Quality ambassadors can influence their peers to prioritize quality, thereby strengthening the culture of quality in the organization. Quality leaders can use this guide to develop a quality ambassador program by identifying, training, and engaging ambassadors.
Create a strong sense of urgency around a big opportunity
Demonstrate the organizational value of Ambassadors by performing a needs analysis to assess the current state of employee engagement with quality.
Build and evolve a guiding coalition
Bring together key stakeholders from across the organization who will provide input in the program’s design and support its implementation.
Form a change vision and strategic initiatives
Identify the key objectives for implementing a Quality Ambassador program and outline the lines of effort required to successfully design and pilot it.
Enlist a volunteer army
Reach out and engage informal leaders at all levels of the organization. Find your current informal Ambassadors and draw them in.
Enable action by removing barriers
Be vigilant for factors that impede progress. Work with your Ambassadors and senior leaders to give teams the freedom and support to succeed.
Generate and celebrate short-term wins
Pilot the program. Create success stories by looking at the successful outcomes of teams that have Quality Ambassadors and by listening to team members and their customers for evidence that quality culture is improving. Your goal will be to create an environment where teams that do not have Quality Ambassadors are asking how they can participate.
Sustain acceleration
Scale the impact of your program by implementing it more broadly within the organization.
Define the Key Responsibilities of Quality Ambassadors
What activities should Quality Ambassadors focus on?
Example:Reinforce key quality messages with co-workers. Drive participation in quality improvement projects. Provide inputs to improve culture of quality. Provide inputs to improve and maintain data integroty
What will Quality Ambassadors need from their managers?
Example:Approval to participate, must be renewed annually
What will Quality Ambassadors receive from the Quality team?
Example:Training on ways to improve employee engagement with quality. Support for any questions/objections that ariseTraining on data integrity
What are Quality Ambassadors’ unique responsibilities?
Example:Acting as the point of contact for all quality-related queries. Reporting feedback from their teams to the Quality leadership. Conveying to employees the personal impact of quality on their effectiveness. Mitigating employee objections about pursuing quality improvement projects. Tackling obstacles to rolling out quality initiatives
What responsibilities do Quality Ambassadors share with other employees?
Example:Constantly prioritize quality in their day-to-day work
Expected time commitment
Example:8-10 hours/month, plus 6 hours of training at launch
We perform risk assessments; execute risk mitigations; and we end up with four types of inherent risks (parenthesis is opportunities) in our risk register:
Mitigated (or enhanced)
Avoided (or exploited)
Transferred (or shared)
Accepted
We’ve built a set of risk response plans to ensure we are continuing to treat these risks. And now we need to monitor the effectiveness of our risk plan and to ensure that the risks are behaving in the manner anticipated during risk treatment.
The living risk assessment is designed to conduct reassessment of risks after treatment and continuously throughout the life cycle. However, not all systems and risks need to be reassessed continually, and the organization should prioritize which systems should be reassessed based on a schedule.
Identify indicators that inform the organization about the status of the risk without having to conduct a full risk assessment every time. The trending status of these indicators can act as a flag for investigations, which may result in complete risk assessments.
This risk indicator is then a metric that indicates the state of the level of risk. It is important to note that not all indicators show the exact level of risk exposure, instead providing a trend of drivers, causes or intermediary effects of risk.
The most important risks can be categorized as key risks and the indicators for these key risks are known as key risk indicators (KRIs) which can be defined as: A metric that provides a leading or lagging indicator of the current state of risk exposure on key objectives. KRIs can be used to continually assess current and predict potential risk exposures.
These KRIs need to have a strong relationship with the key performance indicators of the organization.
A good rule of thumb is as you identify the key performance indicators to assess the performance of a specific process, product, system or function you then identify the risks and the KRIs for that objective.
Strive to have leading indicators that measure the elements that influences the risk performance. Lagging indicators will measure they actual performance of the risk controls.
These KRIs qualitatively or quantitatively present the risk exposure by having a strong relationship qirh the risk, its intermediate output or its drivers.
Let’s think in terms of a pharmaceutical supply chain. We’ve done our risk assessments and end up with a top level view like this:
For the risk column we should have some good probabilities and impacts and mitigations in place. We can then chose some KRIs to monitor, such as
Nonconformance rate
Supplier score card
Lab error rate
Product Complaints
As we develop, our KRIs can get more specific and focused. A good KRI is:
Quantifiable
Measurable (accurately and precisely)
Can be validated (have a high level of confidence)
Relevant (measuring the right thing associated with decisions)
In developing a KRI to serve as a leading indicator for potential future occurrences of a risk, it can be helpful to think through the chain of events that led to the event so that management can uncover the ultimate driver (i.e., root cause(s)) of the risk event. When KRIs for root cause events and intermediate events are monitored, we are in an enviable position to identify early mitigation strategies that can begin to reduce or eliminate the impact associated with an emerging risk event.
These KRIs will help us monitor and quantify our risk exposure. They help our organizations compare business objectives and strategy to actual performance to isolate changes, measure the effectiveness of processes or projects, and demonstrate changes in the frequency or impact of a specific risk event.
Effective KRIs can provide value to the organization in a variety of ways. Potential value may be derived from each of the following contributions:
Risk Appetite – KRIs require the determination of appropriate thresholds for action at different levels within the organization. By mapping KRI measures to identified risk appetite and tolerance levels, KRIs can be a useful tool for better articulating the risk appetite that best represents the organizational mindset.
Risk and Opportunity Identification – KRIs can be designed to alert management to trends that may adversely affect the achievement of organizational objectives or may indicate the presence of new opportunities.
Risk Treatment – KRIs can initiate action to mitigate developing risks by serving as triggering mechanisms. KRIs can serve as controls by defining limits to certain actions.
Investigations of a Dog 