Building a Part 11/Annex 11 Course

I have realized I need to build a Part 11 and Annex 11 course. I’ve evaluated some external offerings and decided they really lack that applicability layer, which I am going to focus on.

Here are my draft learning objectives.

21 CFR Part 11 Learning Objectives

Understanding Regulatory Focus: Understand the current regulatory focus on data integrity and relevant regulatory observations.
FDA Requirements: Learn the detailed requirements within Part 11 for electronic records, electronic signatures, and open systems.
Implementation: Understand how to implement the principles of 21 CFR Part 11 in both computer hardware and software systems used in manufacturing, QA, regulatory, and process control.
Compliance: Learn to meet the 21 CFR Part 11 requirements, including the USFDA interpretation in the Scope and Application Guidance.
Risk Management: Apply the current industry risk-based good practice approach to compliant electronic records and signatures.
Practical Examples: Review practical examples covering the implementation of FDA requirements.
Data Integrity: Understand the need for data integrity throughout the system and data life cycles and how to maintain it.
Cloud Computing and Mobile Applications: Learn approaches to cloud computing and mobile applications in the GxP environment.

EMA Annex 11 Learning Objectives

General Guidance: Understand the general guidance on managing risks, personnel responsibilities, and working with third-party suppliers and service providers.
Validation: Learn best practices for validation and what should be included in validation documentation.
Operational Phase: During the operational phase, gain knowledge on data management, security, and risk minimization for computerized systems.
Electronic Signatures: Understand the requirements for electronic signatures and how they should be permanently linked to the respective record, including time and date.
Audit Trails: Learn about the implementation and review of audit trails to ensure data integrity.
Security Access: Understand the requirements for security access to protect electronic records and electronic signatures.
Data Governance: Evaluate the requirements for a robust data governance system.
Compliance with EU Regulations: Learn how to align with Annex 11 to ensure compliance with related EU regulations.

Course Outline: 21 CFR Part 11 and EMA Annex 11 for IT Professionals

Module 1: Introduction and Regulatory Overview

History and background of 21 CFR Part 11 and EMA Annex 11
Purpose and scope of the regulations
Applicability to electronic records and electronic signatures
Regulatory bodies and enforcement

Module 2: 21 CFR Part 11 Requirements

Subpart A: General Provisions
Definitions of key terms
Implementation and scope
Subpart B: Electronic Records
Controls for closed and open systems
Audit trails
Operational and device checks
Authority checks
Record retention and availability
Subpart C: Electronic Signatures
General requirements
Electronic signature components and controls
Identification codes and passwords

Module 3: EMA Annex 11 Requirements

General requirements
Risk management
Personnel roles and responsibilities
Suppliers and service providers
Project phase
User requirements and specifications
System design and development
System validation
Testing and release management
Operational phase
Data governance and integrity
Audit trails and change control
Periodic evaluations
Security measures
Electronic signatures
Business continuity planning

Module 4: PIC/S Data Integrity Requirements

Data Governance System
- Structure and control of the Quality Management System (QMS)
- Policies related to organizational values, quality, staff conduct, and ethics
Organizational Influences
- Roles and responsibilities for data integrity
- Training and awareness programs
General Data Integrity Principles
- ALCOA+ principles (Attributable, Legible, Contemporaneous, Original, Accurate, Complete, Consistent, Enduring, and Available)
- Data lifecycle management
Specific Considerations for Computerized Systems
- Qualification and validation of computerized systems
- System security and access controls
- Audit trails and data review
- Management of hybrid systems
Outsourced Activities
- Data integrity considerations for third-party suppliers
- Contractual agreements and oversight
Regulatory Actions and Remediation
- Responding to data integrity issues
- Remediation strategies and corrective actions
Periodic System Evaluation
- Regular reviews and re-validation
- Risk-based approach to system updates and maintenance

Module 5: Compliance Strategies and Best Practices

Interpreting regulatory guidance documents
Conducting risk assessments
Our validation approach
Leveraging suppliers and third-party service providers
Implementing audit trails and electronic signatures
Data integrity and security controls
Change and configuration management
Training and documentation requirements

Module 6: Case Studies and Industry Examples

Review of FDA warning letters and 483 observations
Lessons learned from industry compliance initiatives
Practical examples of system validation and audits

Module 7: Future Trends and Developments

Regulatory updates and revisions
Impact of new technologies (AI, cloud, etc.)
Harmonization efforts between global regulations
Continuous compliance monitoring

The course will include interactive elements such as hands-on exercises, quizzes, and group discussions to reinforce the learning objectives. The course will provide practical insights for IT professionals by focusing on real-world examples from our company.

The Audit Trail and Data Integrity

Requirement	Description
Attributable (Traceable)	Each audit trail entry must be attributable to the individual responsible for the direct data input so all changes or creation of data with the persons making those changes. When using a user’s unique ID, this must identify an individual pers on. Each audit trail must be linked to the relevant record throughout the data life cycle.
Legible	The system should be able to print or provide an electronic copy of the audit trail. The audit trail must be available in a meaningful format when. viewed in the system or as hardcopy.
Contemporaneous	Each audit trail entry must be date- and time-stamped according to a controlled clock which cannot be altered. The time should either be based on central server time or a local time, so long as it is clear in which time zone the entry was performed.
Original	The audit trail should retain the dynamic functionalities found in the computerized system, included search functionality to facilitate audit trail review activities.
Accurate	Audit trail functionality must be verified to ensure the data written to the audit trail equals the data entered or system generated. Audit trail data must be stored in a secure manner and users cannot have the ability to amend, delete, or switch off the audit trail. Where a system administrator amends, or switches off the audit trail, a record of that action must be retained.
Complete	The audit trail entries must be automatically captured by the computerized system whenever an electronic record is created, modified, or deleted. Audit trails, at minimum, must record all end user initiated processes related to critical data. The following parameters must be included: The identity of the person performing the action. In the case of a change or deletion, the detail of the change or deletion, and a record of the original entry. The reason for any GxP change or deletion. The time and date when the action was performed.
Consistent	Audit trails are used to review, detect, report, and address data integrity issues. Audit trail reviewers must have appropriate training, system knowledge and knowledge of the process to perform the audit trail review. The review of the relevant audit trails must be documented. Audit trail discrepancies must be addressed, investigated, and escalated to JEB management and national authorities, as necessary.
Enduring	The audit trail must be retained for the same duration as the associated electronic record.
Available	The audit trail must be available for review at any time by inspectors and auditors during the required retention period. The audit trail must be accessible in a human readable format.

21CFR Part 11 Requirements

Definition: An audit trail is a secure, computer-generated, time-stamped electronic record that allows for the reconstruction of events related to the creation, modification, and deletion of an electronic record.

Requirements:

Availability: Audit trails must be easily accessible for review and copying by the FDA during inspections.
Automation: Entries must be automatically captured by the system without manual intervention.
Components: Each entry must include a timestamp, user ID, original and new values, and reasons for changes where applicable.
Security: Audit trail data must be securely stored and not accessible for editing by users

EMA Annex 11 (Eudralex Volume 4) Requirements

Definition: Audit trails are records of all GMP-relevant changes and deletions, created by the system to ensure traceability and accountability.

Requirements:

Risk-Based Approach: Building audit trails into the system for all GMP-relevant changes and deletions should be considered based on a risk assessment.
Documentation: The reasons for changes or deletions must be documented.
Review: Audit trails must be available, convertible into a generally readable form, and regularly reviewed.
Validation: The audit trail functionality must be validated to ensure it captures all necessary data accurately and securely.

Requirements from PIC/S GMP Data Integrity Guidance

Definition: Audit trails are metadata recorded about critical information such as changes or deletions of GMP/GDP relevant data to enable the reconstruction of activities.

Requirements:

Review: Critical audit trails related to each operation should be independently reviewed with all other records related to the operation, especially before batch release.
Documentation: Significant deviations found during the audit trail review must be fully investigated and documented.

Leveraging Inspection Manuals for GMP Inspection Readiness

The various agency inspection manuals are critical tools for inspection readiness. I want to lay out where to find some of these manuals and then go deep into pre-approval inspections, focusing on data integrity.

European Medicines Agency

The European Medicines Agency (EMA) has established detailed procedures and work instructions for coordinating and conducting Good Clinical Practice (GCP), Good Manufacturing Practice (GMP), and pharmacovigilance inspections. Here are the key points regarding EMA’s inspection procedures:

GCP Inspection Procedures

EMA identifies applications for GCP inspections based on risk assessment criteria and exchanges information on shared applications with the FDA.
Inspections can be joint (conducted concurrently by EMA and FDA inspectors) or sequential (conducted separately by each agency).
EMA notifies the applicant/marketing authorization holder (MAH) and inspects sites about upcoming inspections through the IRIS industry portal instead of formal letters.
Applicants/MAHs must provide a signed statement accepting the inspection and granting direct access to documents and medical records.
Requested documents should be provided directly to inspectors in electronic format after consulting the reporting inspector.
After the inspection, EMA receives the draft inspection report, finalizes it with the inspectee’s responses, and publishes it in IRIS.

GMP Inspection Procedures

EMA coordinates GMP inspections based on risk assessment for marketing authorization applications, variations, and routine re-inspections.
Work instructions cover areas such as inspection announcement, fee calculation, product sampling/testing, and report circulation.

Pharmacovigilance Inspection Procedures

EMA has specific procedures for coordinating pharmacovigilance inspections and managing non-compliance notifications from MAHs.
Work instructions detail the inspection program creation, data entry in databases, and interactions with third-country inspectorates.

The EMA aims to harmonize inspection processes with the FDA and other regulatory bodies to streamline collaboration and information sharing while ensuring clinical trial subject protection and product quality.

FDA

The FDA Investigations Operations Manual (IOM) is the primary inspection manual used by FDA personnel when performing inspections and investigations.

The key points about the IOM are:

It provides comprehensive instructions, procedures, and policies for FDA investigators and inspectors to follow when conducting inspections, surveys, and investigations.
It covers inspectional activities for foods, drugs, medical devices, biologics, cosmetics, and other FDA-regulated products.
The manual details procedures for inspections of manufacturing facilities, sampling, import operations, recalls, consumer complaints, and other compliance activities.
It aims to ensure inspections are conducted consistently across FDA field offices and provide clear guidance to the industry on the FDA’s inspection approach.
The IOM is updated periodically to incorporate new laws, regulations, policies, and technological changes impacting FDA’s operations.
While not legally binding, the IOM represents the FDA’s current thinking and policies on inspections and investigations.

The FDA Investigations Operations Manual serves as the comprehensive inspection reference and procedure manual for FDA field staff carrying out the agency’s oversight and enforcement activities across all regulated product areas.

Pre-Approval Inspections

For new facilities, CPGM 7346.832, the FDA’s Compliance Program Guidance Manual for Pre-Approval Inspections (PAIs) of drug manufacturing facilities, is critical to spend time with. It outlines the objectives and procedures for FDA inspectors to evaluate a facility’s readiness for commercial manufacturing before approving a new drug application.

The key objectives of CPGM 7346.832 are:

Assess if the facility has a quality system capable of controlling commercial manufacturing operations.
Verify that the manufacturing processes, formulation, and analytical methods conform to the application details.
Audit raw data integrity to authenticate the data submitted in the application.
Evaluate the facility’s commitment to quality in pharmaceutical development (new objective added in 2022 revision).

The guidance instructs inspectors on evaluating the firm’s quality systems, process validation, data integrity, laboratory controls, change management, investigations, batch release procedures, and compliance with current Good Manufacturing Practices (cGMPs). It aims to ensure the facility can reliably produce the drug product described in the application.

Data Integrity

CPGM 7346.832 has specific requirements for data integrity audits during drug manufacturing facility pre-approval inspections (PAIs). Utilizing this document is an excellent way to evaluate your data integrity program.

The key points are:

Objective 3 of the guidance is “Data Integrity Audit”—auditing and verifying raw data associated with the product to authenticate the data submitted in the application.
Inspectors must audit the accuracy and completeness of data reported by the facility for the product. This involves verifying the factual integrity (data matches what was submitted) and contextual integrity (supporting data is complete).
Inspectors should examine raw data, such as chromatograms, analyst notebooks, electronic data, etc., and compare it to the summary data in the application’s Chemistry, Manufacturing, and Controls (CMC) section.
The data integrity audit should focus on finished product stability, dissolution, content uniformity, API impurities, etc.
Inspectors must identify any unreported relevant data, data falsification, improper invalidation of results, or unexplained data discrepancies.
Indications of data integrity issues include altered raw data, references to failing studies, discrepancies between samples, and missing records.

The data integrity audit aims to ensure the CMC data submitted to FDA is complete, reliable, and can be fully authenticated from the raw data at the manufacturing site. Robust data integrity is critical for the FDA to decide on the application’s approval.

Spreadsheets in a GxP Environment

I have them, you have them, and chances are they are used in more ways than you know. The spreadsheet is a powerful tool and really ubiquitous. As such, spreadsheets are used in many ways in the GxP environment, which means they need to meet their intended use and be appropriately controlled. Spreadsheets must perform accurately and consistently, maintain data integrity, and comply with regulatory standards such as health agency guidelines and the GxPs.

That said, it can also be really easy to over-control spreadsheets. It is important to recognize that there is no one-size-fits-all approach.

It is important to build a risk-based approach from a clear definition of the scope and purpose of an individual spreadsheet. This includes identifying the intended use, the type of data a spreadsheet will handle, and the specific calculations or data manipulations it will perform.

I recommend an approach that breaks the spreadsheet down into three major categories. This should also apply to similar tools, such as Jira, Smartsheet, or what-have-you.

Spreadsheet Functionality	Level of verification
Used like typewriters or simple calculators. They are intended to produce an approved document. Signatories should make any calculations or formulas visible or explicitly describe them and verify that they are correct. The paper printout or electronic version, managed through an electronic document management system, is the GxP record.	Control with appropriate procedural governance. The final output may be retained as a record or have an appropriate checked-by-step in another document.
A low level of complexity (few or no conditional statements, smaller number of cells) and do not use Visual Basic Application programs, macros, automation, or other forms of code.	Control through the document lifecycle. Each use is a record.
A high level of complexity (many conditional statements, external calls or writing to an external database, or linked to other spreadsheets, larger number of cells), using Visual Basic Application, macros, or automation, and multiple users and departments.	Treat under a GAMP5 approach for configuration or even customization (Category 4 or 5)

Requirements by Spreadsheet complexity

For spreadsheets, the GxP risk classification and GxP functional risk assessment should be performed to include both the spreadsheet functionality and the associated infrastructure components, as applicable (e.g., network drive/storage location).

For qualification, there should be a succinct template to drive activities. This should address the following parts.

1. Scope and Purpose

The validation process begins with a clear definition of the spreadsheet’s scope and purpose. This includes identifying its intended use, the type of data it will handle, and the specific calculations or data manipulations it will perform.

2. User Requirements and Functional Specifications

Develop detailed user requirements and functional specifications by outlining what the spreadsheet must do, ensuring that it meets all user needs and regulatory requirements. This step specifies the data inputs, outputs, formulas, and any macros or other automation the spreadsheet will utilize.

3. Design Qualification

Ensure that the spreadsheet design aligns with the user requirements and functional specifications. This includes setting up the spreadsheet layout, formulas, and any macros or scripts. The design should prevent common errors such as incorrect data entry and formula misapplication.

4. Risk Assessment

Conduct a risk assessment to identify and evaluate potential risks associated with the spreadsheet. This includes assessing the impact of spreadsheet errors on the final results and determining the likelihood of such errors occurring. Mitigation strategies should be developed for identified risks.

5. Data Integrity and Security

Implement measures to ensure data integrity and security. This includes setting up access controls, using data validation features to limit data entry errors, and ensuring that data storage and handling comply with regulatory requirements.

6. Testing (IQ, OQ, PQ)

IQ tests the proper installation and configuration of the spreadsheet.
OQ ensures the spreadsheet operates as designed under specified conditions.
PQ verifies that the spreadsheet consistently produces correct outputs under real-world conditions.

Remember, all one template; don’t get into multiple documents that each regurgitate all the same stuff.

Lifecycle Approach

Spreadsheets should have appropriate procedural guidance and training.

They should be under risk-based periodic review.

Good Scientific Practices as Phase Appropriate

There has been increasing evidence in recent years that research in life sciences is lacking in reproducibility and data quality. This raises the need for effective systems to improve data integrity in the evolving non-GxP research environment. Reproducibility is a defining principle of scientific research, and broadly refers to the ability of researchers, other than the original researchers, to achieve the same findings using the same data and analysis data reproducibility is key to the reinforcement and credibility of scientific evidence. All results should be replicable by different investigators in varied geographical settings, using independent data, instruments, and analytical methods.

Some examples:

In 2022 there were 11 Federal Register notices with ORI findings of research misconduct that involved Public Health Service support or funding. These cases included falsified data submitted in National Institutes of Health grant applications and PHS-supported publications. These cases resulted debarment periods of up to four years and supervision periods of up to 12 years.
Novartis “data manipulation” involving its Zolgensma gene therapy
Leen Kawas Resigned as CEO of Athira in 2021 following an investigation into her doctoral work.

Without a doubt it is critical to build a quality culture within our research organizations. Through educating our scientific staff we can continue to innovate and discover new pathways, new drugs and new treatments. Efficient processes enhance research effectiveness and lead to scientific discoveries. Data integrity supports good science, drug safety, products and treatment development for patients and customers. While this looks similar in research as in later phases there are 4 primary pillars:

Train researchers on basic documentation processes and good scientific practices to ensure data integrity and quality. Targeted training should be added on new guidelines, processes and regulations applied to their specific activities.
Empower for change and to speak up
Incentives for Behaviours Which Support Research Quality
Promote a Positive Error Culture

I’m a huge fan of the EQIPD approach:

https://eqipd181379605.wordpress.com/.

Bespalov, A., Bernard, R., Gilis, A., Gerlach, B., Guillén, J., Castagné, V., Lefevre, I. A., Ducrey, F., Monk, L., Bongiovanni, S., Altevogt, B., Arroyo-Araujo, M., Bikovski, L., Bruin, N. de, Castaños-Vélez, E., Dityatev, A., Emmerich, C. H., Fares, R., Ferland-Beckham, C., … Steckler, T. (2021, May 24). Introduction to the EQIPD Quality System. eLife. https://elifesciences.org/articles/63294