Evolved Expendable Launch Vehicle (EELV) Quality Management

We determined that ULA, SpaceX, and AR were not performing adequate quality assurance management for the EELV program as evidenced by the 181 nonconformities to the AS9100C at the EELV contractor production facilities. This inadequate quality assurance management could increase costs, delay launch schedules, and increase the risk of mission failure.

From ”

Evaluation of the Evolved Expendable Launch Vehicle Program Quality Management System DODIG-2018-045, Department of Defense Office of Inspector General

It is useful to read audit reports and inspection findings from multiple industries. From this we can see trends, make connections and learn.

I see a few things that stand out.

DOD findings

Risk Register

Our evaluation of the RIO database showed that 11 out of 26 risks related to either Atlas V or Delta IV launch vehicle were in “red” status, which indicates that risk mitigation was behind schedule.

It is not enough to identify risks (though that is a critical place to start). You just can’t track them (though again, if you don’t track it you don’t see it). You actually have to have clear plans to mitigate and eliminate the risks. And this is where the program seemed to fall short.

Not a surprise. I think a lot of companies are having these difficulties. In the pharma world the regulatory agencies have been signaling pretty strongly that this is an issue.

Make sure you identify risks, track them, and have plans that are actually carried out to remediate.

Configuration Management

SpaceX failed to comply with AS9100C, section 7.1.3, which requires it to “establish, implement, and maintain a configuration management process.” Configuration management is a controlled process to establish the baseline configuration of a product and any changes to that product. This process should occur during the entire life cycle of a product to provide visibility and control of its physical, functional, and performance attributes.

First rule of reading inspection reports: Things probably went bad if a section starts with standard review 101 material.

That said, hello change management my dear friend.

This was the gist of my ASQ WCQI workshop last May, every industry needs good change management and change control.

Material Management

ULA and AR failed to comply with AS9100C, section 8.3, which requires them to “ensure that product which does not conform to product requirements are identified and controlled to prevent its unintended use or delivery.”

At ULA, we found 18 expired limited-life material items that were between 32 and 992 days past their expiration dates, but available for use on EELV flight hardware. This material should have been impounded and dispositioned. The use of expired limited-life items, such as glues and bonding agents, could result in product that does not meet specifications and may require costly rework.

I find it hard to believe that these companies aren’t tracking inventory. If they are tracking inventory and have any sort of cycle count process then the mechanism exists to ensure expired material is removed from the possibility of use. And yet we still see these observations across the pharma industry as well.

Concluding Thoughts

Quality Management has it its core the same principles, no matter the industry. We use similar tools. Leverage the best practices out there. Read about stresses other companies are having, learn from them and remediate at your own organization.

Contamination Control, Risk Management and Change Control

Microbiologists won’t be sequestered in the laboratory, running samples and conducting environmental testing, once the revisions proposed for Annex 1 of the EU and Pharmaceutical Inspection Cooperation Scheme (PIC/S) GMP guides take effect, Annex 1 rapporteur Andrew Hopkins said Oct. 15.

They will have a broader role that includes conducting risk assessments to ensure that sterile products are made as contamination-free as possible, said Hopkins, who is an inspector for the UK Medicines and Healthcare products Regulatory Agency.

Pink Sheet “EU GMP Annex 1 Would Give Microbiologists A Greater Role In Sterility Assurance, Rapporteur Says“

Contamination Control is a fairly wide term used to mean “getting microbiologists out of the lab” and involved in risk management and compliance. Our organization splits that function off from the QC Microbiology organization but there are many models for making it work.

Risk Management is a major part of the new Annex 1, and what they are driving at are good risk assessments with good risk mitigation that involve the microbiologists.

living risk assessments

This is really what is meant by a contamination control strategy which considers the product and process knowledge and skills in pharmaceutical product manufacturing and GMP/ cGMP compliance under the auspices of a Pharmaceutical Quality System (Q10) together with initiatives of Quality by Design (Q8) and Quality Risk Management (Q9).

From this strategy comes:

Targeted/ risk based measures of contamination avoidance
Key performance indicators to assess status of contamination control
A defined strategy for deviation management (investigations) and CAPA

environmental monitoring

When it comes to change management, one of the easiest places to go wrong is to forget to bring the microbiologist in to changes. Based on your strategy you can determine change changes require their assessment and include it in the tool utilized to determine SMEs, for example:

Department	Required if the change meets any of the following criteria:
Contamination Control	The change impacts environment integrity, conditions or monitoring, including: Changes to a controlled room or area that impact integrity Changes in sampling methodology Construction activities Changes in personnel or material flow The change will result in or modify exposure of product to the environment. The change can impact microbiological control within a process stream, raw material or process equipment The changes are to water systems

Department

Required if the change meets any of the following criteria:

Contamination Control

The change impacts environment integrity, conditions or monitoring, including:

Changes to a controlled room or area that impact integrity
Changes in sampling methodology
Construction activities
Changes in personnel or material flow
The change will result in or modify exposure of product to the environment.

The change can impact microbiological control within a process stream, raw material or process equipment

The changes are to water systems

Don’t Just Tell Employees Organizational Changes Are Coming — Explain Why

To be successful, your story needs to start with the company’s core mission and then offer a compelling and inspiring future vision. You want to answer: How are the changes you make today helping you achieve your vision for tomorrow?
Don’t Just Tell Employees Organizational Changes Are Coming — Explain Why by Morgan Galbraith

I can’t stress enough the importance of proper communication around all changes, from the large transformations on down. Effective communication is effective change management.

I’ve discussed the need to be able to identify changes to strategic plans and use that to inspire, inform, empower, and engage.

changing business environment

Always spend the time on a good communication plan:

Information to Communicate
(What)

Objective
(Why)

Target Audience
(Who to)

Frequency
(When)

Start Date
(When)

End Date
(When)

Media
(How)

Responsible
(Who from)

Deliverable

Comments

What to people need to know

o Determine site readiness to start the project

o Define resource needs and availability

Tailor the communication to specific audiences. The same information is sometimes presented different ways

How often?

Start date

End Date

From face-to-face to all the other communication tools available in the modern workplace. Be creative

Who is responsible for completing the communication

What will execution look like

Improving But Not Learning by Doing

Alex Tarbok on Marginal revolution wrote “Improving But Not Learning by Doing” looking at a paper “Causal understanding is not necessary for the improvement of culturally evolving technology” — which has interesting things to say to those interested in knowledge management. In short it demonstrates that the complex problems we have today need to be approached multi-dimension approach, in short system thinking is required to do true knowledge management.

Knowledge management Circular_Process_6_Stages (for expansion)

Throughout the six stages we need to be evaluating for complexities and interfaces. It is very easy to think in a silo and then create bigger problems done the line.

The DIKW pyramid is a great resource to keep in mind here.

Data comprises facts, observations, or perceptions
Information is a subset of data, only including those data that possess context, relevance, and purpose
Knowledge is Information with direction, i.e., leads to appropriate actions
Wisdom is the understanding of the why

I know that in many knowledge management models wisdom is often discounted, but that is to our detriment. Quality is often all about the why, whether a regulatory commitment, or a deep understanding of history, or as is relevant here , the relationship between parts of a complex system (or the interrelationship between systems).