ASQ Audit Conference – Day 2 Morning

Jay Arthur “The Future of Quality”

Starts with our “Heroes are gone” and “it is time to stand on our  two feet.”

Focuses on the time and effort to train people on lean and six sigma, and how many people do not actually do projects. Basic point is that we use the tools in old ways which are not nimble and aligned to today’s needs. The tools we use versus the tools we are taught.

Hacking lean six sigma is along a similar line to Art Smalley’s four problems.

Applying the spirit of hacking to quality.

Covers valuestream mapping and spaghetti diagrams with a focus on “they delays in between.” Talks about how control charts are not more standard. Basic point is people don’t spend enough time with the tools of quality. A point I have opinions on that will end up in another post.

Overcooked data versus raw data – summarized data has little or no nutritional value.

Brings this back to the issue of lack of problem diagnosis and not problem solving. Comes back to a need for a few easy tools and not the long-tail of six sigma.

This talk is very focused on LSS and the use of very specific tools, which seems like an odd choice at an Audit conference.

“Objectives and Process Measures: ISO 13485:2016 and ISO 9001:2015” by Nancy Pasquan

I appreciate it when the session manager (person who introduces the speaker and manages time) does a safety moment. Way to practice what we preach. Seriously, it should be a norm at all conferences.

Connects with the audience with a confession that the speaker is here to share her pain.

Objective – where we are going. Provide a flow chart of mission/vision (scope) ->establish process -> right direction? -> monitor and measure

Objectives should challenge the organization. Should not be too easy. References SMART. Covers objectives in very standard way. “Remember the purpose is to focus the effort of the entire organization toward these goals.” Links process objectives to the overall company objectives.

Process measures are harder. Uses training for an example. Which tells me adult learning practice is not as much as the QBOK way of thinking as I would like. Kilpatrick is a pretty well-known model.

Process measures will not tell us if we have the right process is a pretty loaded concept. Being careful of what you measure is good advice.

“Auditing Current Trends in Cleaning Validation” by Cathelene Compton

One of the trends in 2019 FDA Warning letters has been cleaning. While not one of the four big ones, cleaning validation always seems relevant and I’m looking forward to this presentation.

Starting with the fact that 15% if all observations on 483 forms related to leaning validation and documentation.

Reviews the three stages from the 2011 FDA Process Validation Guidance and then delvers into a deeper validation lifecycle flowchart.

Some highlights:

Stage 1 – choosing the right cleaning agent; different manufacturers of cleaning agents; long-term damage to equipment parts and cleaning agent compatibility. Vendor study for cleaning agent; concentration levels; challenge the cleaning process with different concentrations.

Delves more into cleaning acceptance limits and the importance of calculating in multiple ways. Stresses the importance of an involvement of a toxicologist. Stresses the use of Permitted Daily Exposure and how it can be difficult to get the F-factors.

Ensure that analytical methods meet ICHQ2(R1). Recovery studies on materials of construction. For cleaning agent look for target marker, check if other components in the laboratory also use this marker. Pitfall is the glassware washer not validated.

Trends around recovery factors, for example recoveries for stainless tell should be 90%.

Discusses matrix rationales from the Mylan 483 stressing the need to ensure all toxicity levels are determined and pharmaceological potency is there.

Stage 2 all studies should include visual inspection, micro and analytical. Materials of construction and surface area calculations and swabs on hard to clean or water hold up locations. Chromatography must be assessed for extraneous peaks.

Verification vs verification – validation always preferred.

Training – qualify the individuals who swab. Qualify visual inspectors.

Should see campaign studies, clean hold studies and dirty equipment hold studies.

Stage 3 – continuous is so critical, where folks fall flat. Do every 6 months, no more than a year or manual. CIP should be under a periodic review of mechanical aspects which means requal can be 2-3 years out.

SIPOC diagrams

I am a huge fan of a SIPOC which stands for suppliers-inputs-process-outputs-customers. A SIPOC diagram is a quick broad overview of all the elements of a process and serves as a great visual scope.

Blank SIPOC

Start with the process

Provide the key steps of the process in the middle column and briefly describe its key steps.  A SIPOC diagram is a high-level process map and is designed to get a birds-eye overview of the process. Do not include decision points or feedback loops.

Identify the outputs of the process

Focus on the key outputs of the process. In this step, write down the three or more main outputs. Use nouns for the most part and avoid categorizing your outputs into good or bad ones – that’s not the point of the diagram.

Identify the customers

List the people who benefit from the process. These don’t have to be the literal “customers.” E.g., if you are working on a diagram for an internal process, the “customers” are your coworkers. Think of who benefits from this process. Who would be upset if the process is not complete?

List the inputs for the process

List the inputs required for the process to function properly. Just like with every previous step, focus on the most important ones. Three to six main inputs should do.

Identify the suppliers of the inputs

List the suppliers based on what inputs the process uses. Be sure to mention any specific suppliers whose input has a direct influence on the output.

The template I use is here.

Examples of SIPOCs:

Personal Audits as part of team building for Projects

The personal audit is a tool used in change and project management (and such) to help team members and sponsors judge their strengths and weaknesses with respect to change leadership. It illustrates some skills from the full range necessary to introduce change into an organization.

This exercise is great to do at the beginning of the project, where it can help team members begin to understand some of the human issues applicable to all projects. As one mentor once told me – If this exercise strikes team members as inapplicable, then they really need to do it.

Domain What I do Well What I Need to Work On
Manage Attention: To what extent do I manage my time, energy, passion, focus and agenda?    
Adopt change roles? How much attention do I pay to smatters like: Creating a need, Shaping a vision, Mobilizing commitment,  Monitoring progress, Finishing the job,  Anchoring the change)    
Technical competence: To what extent to I demonstrate competence in technical abilities?    
Interpersonal competence: how skilled am I at interacting with others?    
Vision: How well can I articulate the desired outcome of the project and the benefits to others?    
Teamwork: How often do I recognize good work done by teammates?    
Diplomacy: How closely am I working with all the groups affected by this project?    
Conflict management: Can I deal with disagreement without avoiding it or blowing up?    
Summary: Overall strengths and weaknesses    

Environment, Health and Safety and the compliance domain

Benefits of Written Rules:
Capture important learnings and assumptions
Establish a standardized, organized and reproducible, method of conducting work safely
Ensure effective transfer of knowledge to new members of the group
Require disciplined thinking to formally document thus reducing errors in processes
Create a framework for delegation of decision-making
Demonstrate the organizations commitment to safety

Chet Brandon “Tried and True: Written Procedures are a Foundation of EHS Success

I don’t think there is a quality person who would read that list and not nod knowingly. Reading the excellent article quoted above reminded me that we all probably do EHS, Quality and compliance in general all wrong.

Yes, Health & Safety is about the employee; Quality is about the product (and legal is about following the law and finance does something about money) but what when you look at the tools we pretty much have a common tool-box. Root cause analysis, procedures, risk management, system thinking.

What is truly different is the question we ask:

  • Quality asks about the customer
  • Health and Safety asks about the employee
  • Environment asks about, well, the environment

I find it fascinating that it became environment, health and safety and most companies, as again, the question asked is rather different. In companies where care of the environment is separate (such as the energy industry) you will definitely see it as a separate entity.

I have only been at one company that was on the path of looking at quality, environment, health and safety were all similar disciplines and united them under a chief compliance officer (who was also head of legal). My current company is still struggling along the path of uniting standards and tools.

There is definitely a lot of different domain knowledge between the three, the same way quality is different between industries. However the commonalities that unite us are many and ones we should spend more time exploring.

Risk Filtering – A popular tool that is easy to abuse

An article titled “ICE Modified Its ‘Risk Assessment’ software So It Automatically Recommends Detention” is probably guaranteed to reach me, for a myriad of ways.

I believe strongly in professional codes of conduct, and the need to speak out. In this case, I am thinking of two charges:

  1. Hold paramount the safety, health, and welfare of individuals, the public, and the environment.
  2. Avoid conduct that unjustly harms or threatens the reputation of the Society, its members, or the Quality profession.

Reading this article, and doing some digging, tells me that the tools of quality that I hold dear have been abused and I believe it is appropriate to call that out.

Now, a caveat, risk assessment, and management have some flavors out there and I’ll be honest that I once made the mistake of getting into a discussion with a risk management expert from a bank and realizing we had very different ideas of risk management. But supposedly we’re all aligned (sort of) to ISO Guide 73:2009, “Risk management. Vocabulary.” And as such, I’ll try to stick pretty close to those shared commonalities. I also assume that ISO Guide 73:2009 is a shared point between me and whoever designed the ICE risk assessment software.

Risk assessment is one phase in risk management, and I’ll focus on that here. Risk assessment is about identifying risk scenarios. What we do is:

  1. Establish the context and environment that could present a risk
  2. Identify the hazards and considering the hazards these risks could present
  3. Analyze the risks, including an assessment of the various contributing factors
  4. Evaluate and prioritize the risks in terms of further action required
  5. Identify the range of options available to tackle the risks and decide how to implement risk management strategies.

A look at the decision making around this found in the Reuters article, leads me to believe that what ICE is using meets these criteria and we can call it a risk assessment (why it is in quotes in the Motherboard article mystifies me).

There are a lot of risk assessment tools out there. it is important to know that risk assessment is not perfect, and as a result, we are constantly developing better tools and refining the ones we have.

My guess is we are seeing a computerized use of the risk ranking and filtering tool here. Very popular, and something I’ve spent a great deal of time developing. This tool involves breaking a basic risk question down into as many components as needed to capture factors involved in the risk. These factors are then combined into a relative risk score for ranking. Filters are weighting factors used to scale the risks to objectives.

And that is where this tool can often go wrong. It appears ICE under the Trump administration has determined its objective is to jail everyone. By adjusting the filters, the tool easily drives to that conclusion. And this is a problem. Here we see a quality tool being used to excuse inhumane policy choices. It is not the ICE agents separating families and jail people over a misdemeanor, it is the tool. And if that doesn’t strike to the heart of the banality of evil concept I’m not sure what does.

I could go deeper into the tool, how I would have built it, the ways you validate the effectiveness of it. And that all probably will make an excellent follow-up someday. But the reason I’m writing this post is primarily that I read this article and it dawned on me that someone very similar to me in skill set probably created this tool. Someone who maybe I’ve sat across the table at a professional conference, who has read the same articles, probably debates the same qualitative vs. quantitative debates. And this is a great example of when its necessary to speak up and criticize a tool of my profession being used for evil. I probably will never talk to the team who developed this tool, but we all see instances of companies around us being asked to build similar applications, using the tools of our profession, that will be used for the wrong results. And we owe it to our code of ethics to refuse.