PCPC Data Integrity Case Studies

www.Ceruleanllc.com

© 2017 Cerulean Associates LLC 1

FDA Data Integrity Case

Studies for Cosmeticswith enforcement examples and steps to consider

John AvellanetCerulean Associates LLC

www.CeruleanLLC.com

Personal Care Products Council

October 2017

Data Integrity Case Study training approach

by:

Robert D. TollefsenNational Expert Investigator- Drugs/ComputersUSFDA/[email protected]

Appreciation also goes out to the Data Integrity CAG members for developing this agreed upon approach: Phil Pontikos-NE Devices, Thomas Arista-NE Drugs, Kip Hanks-NE Biologics, Robert Tollefsen-NE Drugs/Computers, Justin Boyd-CSO DMPTI, Barbara Wright-CSO BIMO Specialist, Nancy Rolli-Director PQIBI, Mary Kennelly-Regulatory Council OPRM, and Monica Caphart-OTED

Based upon:

www.Ceruleanllc.com


Agenda

cosmetic data integrity requirements

FDA’s inspection approach

steps for you to consider

© 2017 Cerulean Associates LLC

This is not legal advice. Information in this presentation draws upon a variety of sources, including published warning letters, form

FDA-483s, personal experiences, interviews and research, all or any of which may or may not have been prepared or conducted by

Cerulean Associates LLC. Cerulean Associates LLC does not provide a warranty concerning the accuracy of the information

contained in this presentation. The contents of this presentation are intended for general information only and should not be

construed as legal advice. Cerulean Associates LLC assumes no liability for actions taken or not taken as a result of the information

in this presentation. This presentation is copyrighted 2017 by Cerulean Associates LLC, all rights reserved.

3www.Ceruleanllc.com

FDA Data Integrity Definition

Data have integrity “if they are fit for their intended uses in operations, decision making and planning” (J.M. Juran)

- http://en.wikipedia.org/wiki/Data_quality

*Note: In the final 21 CFR 11 rule and preamble, “integrity” is referenced 50+ times

4© 2017 Cerulean Associates LLC www.Ceruleanllc.com

Are the data “fit for use” in making product safety

and/or quality decisions…?

www.Ceruleanllc.com


“Part 11 is not an IT issue, but rather a

way to manage the additional

risks associated with electronic data.”- George Smith, Jr., Chair of FDA Part 11 Working Committee, June 2011


FDA cGMP Cosmetic Guidance

§ III.C Records

“Records should be retained in either paper or electronic format. Records should capture in detail the operations, procedures, deviations from procedures, justifications, protocols, reports, methods, precautions, corrections and other measures, and other appropriate information related to GMPs.”

© 2017 Cerulean Associates LLC 6www.Ceruleanllc.com

1) Phrase “…related to GMPs”directly points to FDA’s (pharmaceutical) cGMPs for the 21st century – and thus data integrity expectations for cGMPs(e.g., characterized by ALCOA+ throughout the data lifespan)

2) And “…other appropriate information” is currently interpreted by FDA to include raw data not just results

www.Ceruleanllc.com


FDA cGMP Cosmetic Guidance

§ 7 Records

“Check whether control records are maintained of:

a) Raw materials and primary packaging materials, documenting disposition of rejected materials.

b) Manufacturing of batches, documenting the:i. Kinds, lots and quantities of material used.

ii. Processing, handling, transferring, holding and filling.

iii. Sampling, controlling, adjusting and reworking.

iv. Code marks of batches and finished products.

c) Finished products, documenting sampling, individual laboratory controls, test results and control status.

d) Distribution, documenting initial interstate shipment, code marks and consignees.”

© 2017 Cerulean Associates LLC www.Ceruleanllc.com 7

Source:FDA Cosmetics Good Manufacturing Practice (GMP) Guidelines/Inspection Checklist, 2008http://www.fda.gov/Cosmetics/GuidanceRegulation/GuidanceDocuments/ucm2005190.htm

Inspectional Approach for Data Integrity

1. Awareness Trigger

2. Verification

3. Confirmation

4. Impact

www.fda.gov 8

http://www.fda.gov/Cosmetics/GuidanceRegulation/GuidanceDocuments/ucm2005190.htm

www.Ceruleanllc.com


1. Awareness

• We must be mindful of data integrity throughout our inspection

• During a routine inspection:

– Observing operations

– Inspecting equipment

– Reviewing records

– Reviewing procedures (SOPs, policies, etc.)

www.fda.gov 9

1. Awareness – tools & techniques

Example tools available to evaluate data integrity:

– Audit trails

– Entry/exit logs

– Video recording systems

– Networked systems access control lists (ACLs)

– Inventory lists

– Statements/claims in procedures

– File system permissions

– Match controls against ALCOA+ characteristics

– Firm’s overall data integrity compliance plan


www.Ceruleanllc.com


FDA Investigator Instructions

“If a firm is keeping electronic records, determine if they are in compliance with 21 CFR Part 11. At a minimum, ensure that:

(1) the firm has prepared a plan for achieving full compliance with part 11 requirements and is making progress toward completing that plan in a timely manner

(2) accurate and complete electronic and human readable copies of electronic records, suitable for review, are made available

(3) employees are held accountable and responsible for actions.

If initial findings indicate the firm’s electronic records may not be trustworthy and reliable, or when electronic recordkeeping systems inhibit

meaningful FDA inspection, a more detailed evaluation may be warranted.”

- FDA Enforcement Compliance Policy Manual, Attachment A

http://www.fda.gov/ICECI/EnforcementActions/BioresearchMonitoring/ucm133927.htm


Example Health Canada Citation


“The creation, maintenance, processing and/or review of laboratory data was inadequate. Management of electronic data was deficient in that the systems did not ensure traceability and integrity through the data lifecycle. For example…Quality

Control lack a Data Integrity & Compliance masterplan….”- Health Canada Inspection Exit Notice to Knowlton Development Corporation, November 2016

http://www.fda.gov/ICECI/EnforcementActions/BioresearchMonitoring/ucm133927.htm

www.Ceruleanllc.com



• You come across something that seems unusual or odd

• Data appear to be “too good to be true”

• Steps in the procedure are not logical or happen to fast

• After repeated analyses, suddenly something passes with no explanation for the previous failures

• Information you’ve observed/gathered conflicts with information from your inspection preparation

• Systems scattered throughout the facility “are not used for cGXP work” (or firm claims to rely solely upon paper printouts for decision-making)

• Repeated, uniform data patterns


FDA Data Integrity Worries

• Fraud

• Erroneous values

• Missing values

• Deviations from data collection/review protocol or procedures

• “Unconscious bias”

• Imputation (substitution of data for missing values)

• Lack of consistency of results within/across trials

• Data entry errors (abandoned entries, cancelled entries, etc.)

• Extensive corrections (including CYA-based “Memos to the File”)

• Missing source data (or originals if did not make a true copy)

• Exclusion of data points (batch QC results, lab test outliers, etc.) from analyses

• Reliance on unplanned subset analyses

• Re-testing after initial QC sampling and testing failed (e.g., testing into compliance)


Did the firm have effective controls in place to either [A] prevent or [B] detect

and mitigate?

www.Ceruleanllc.com


Is This a Red Flag…?

During your inspection of a manufacturing facility, you uncover that data in the laboratories can be deleted, edited, etc. by any laboratory personnel. There are no written procedures that admonish users not to manipulate or delete data during data reviews, testing reviews, etc.

However, the firm has qualified all of its laboratory instrumentation, has laboratory notebooks for each analyst (which they use appropriately), conducts routine training on cGLP & cGMP requirements, and holds regular internal quality audits of its labs.

Is this acceptable? Why or why not?


Example FDA-483 Observation


“Appropriate controls are not exercised over computers or related systems to ensure that changes in master production and control records or other records are instituted only by authorized personnel. Specifically, your firm has failed to develop written procedures to govern use of…a stability data software and…for

chromatographic data processing….”- FDA-483 to Estee Lauder, August 2015SOPs are part of

acceptable (and expected) controls for data integrity

www.Ceruleanllc.com


2. Verification

• Evaluate the trigger event :

– Do the records represent the facts as stated?

– Are there other reasonably plausible scenarios that explain the situation?

• Cross check:

– review other relevant records and/or systems

– make direct observations of other activities

– sample additional data and systems to determine if this is a one-off issue or is systemic

www.fda.gov 17

Example Inspection Questions

• Who is authorized to access the system and enter/change data? How is this tracked and reviewed? How was this tested and verified?

• Are there written procedures for system validation, data collection, and computerized system security?

• Are original data entered directly into an electronic record at the time of collection or are data transcribed from paper into an electronic record?

• How is recorded data reviewed? By whom? Are they qualified? Is there a conflict of interest? If yes, how is this conflict managed?

• In databases, how are missing information and unexplained inconsistencies documented and corrected?

• Are there edit checks and data logic checks for acceptable ranges of values (such as in database systems and spreadsheets)?

• Are there controls in place to prevent, detect, and mitigate effects of computer viruses (and malware) on data?


www.Ceruleanllc.com



During your inspection, you note that a firm does conduct regular data reviews prior to releasing every single product. This review is documented in an SOP.

The data review consists of using a printout of testing results and matching the data values on the print out with the corresponding approved product quality attributes (ingredient percentages, color, etc.) and ranges.

Is this acceptable? Why or why not?


Example FDA-483 Observation


“Failure to maintain a backup file of data entered into the computer or related system. Electronic records are used, but they do not meet retention, system access limitation, audit trail, and authority check requirements to ensure that they are trustworthy, reliable and generally equivalent to paper. Specifically…There is no management oversight to review the audit trail and electronic data after the completion of an analysis. During peer review, the reviewer only looks at the

printout….”- FDA-483 to DLC Laboratories, December 2015

No electronic data review during lab data reviews

www.Ceruleanllc.com


3. Confirmation

• Ask direct questions to the responsible personnel about data integrity.

– Confirm what the facts are showing

– Who knew or didn’t know?

– When did they know?

– Why was this done?

– Were they aware of the regulatory expectations around this?

• Does your evidence document the findings?

www.fda.gov

Consider any CAPAs already in progress

as mitigating factors

21


During your inspection, you find a portable hard drive sitting atop an HPLC in the firm’s quality control laboratory. Upon questioning, none of the personnel in the lab know what the drive is for nor who put it there.

After talking with the firm’s IT department, it becomes clear that the drive is used to back up all the data on the HPLC on a weekly basis. The backups are set to recycle at the same frequency as the network to ensure that the drive never becomes full (and backups fail).

Is this acceptable? Why or why not?© 2017 Cerulean Associates LLC 22www.Ceruleanllc.com

www.Ceruleanllc.com


Example FDA-483 Citation

“Observation 3. Data from computerized systems is not backed up at appropriate intervals and appropriately stored. Specifically, in the QC Raw Material and Stability laboratory, HPLC units are tied to a single standalone PC, upon which data is collected and stored. A back-up of the data transferred to a portable hard drive every …, which is not stored remotely from the equipment.”

FDA-483, L’Oreal, December 9, 2015


NOTE:

This FDA-483 cites L’Oreal for ALL laboratory data under the

“…other appropriate information” phrase, not just

laboratory “test results”

4. Impact

• What is the impact on product quality? on product safety?

– Identify if there are specific products or lots distributed within the US that are impacted

• How long has this occurred?

• Where else may similar data integrity issues exist?

– Does this expand to other departments, facilities, or upper management?

– Do the same personnel have other responsibilities?

– Are there other record sets that may also be untrustworthy?

www.fda.gov 24

www.Ceruleanllc.com


Example Areas of Concern


improper calibration

incomplete record keeping

overwriting files

peak shaving or enhancing

failure to retain original source if not

making true copy

deleting test results or other data

failure to record data accurately

archived data loss

rounding data values

eliminating outliers

mislabeled samples

failure to follow SOPs

unqualified personnel

making up data


During your inspection, you find 17 CAPA investigations into data integrity related issues throughout a facility –in manufacturing, in the quality control release lab, in the building management system, etc. Only 5 of the CAPAs are still unresolved after nearly a year.

When discussing your findings with the Quality Unit, it becomes clear that these 5 open CAPAs cannot be closed due to lack of budget, but Quality is going to ask again for the resources to address next year.

Is this acceptable? Why or why not?© 2017 Cerulean Associates LLC 26www.Ceruleanllc.com

www.Ceruleanllc.com


Example Warning Letter Excerpt


“Failure to have computerized systems with sufficient controls to prevent unauthorized access or changes to data. Our inspection documented that your firm’s quality unit was aware of the lack of controls in your computerized systems to prevent the manipulation and deletion of quality-related data.

Your site’s senior management failed to take sufficient corrective action and prevent the recurrence of these

problems.”Warning Letter to Ipca Laboratories, January 2016

www.fda.gov/ICECI/EnforcementActions/WarningLetters/2016/ucm484910.htm

www.Ceruleanllc.com

Inspectional Approach for Data Integrity


2. Verification

3. Confirmation

4. Impact

www.fda.gov 28

www.Ceruleanllc.com


True or False?

___ 1. Cosmetic firms do not have to worry about data integrityunless they make OTCs

___ 3. Data with integrity can be characterized as informationthat is accurate, legible, contemporaneous, original, attributable, complete, consistent, enduring, and available

___ 4. For data to be suitable for usage to make decisions aroundproduct safety and/or product quality, that data musthave controls around it that ensure its trustworthiness


Draft an Overall Plan

Data Integrity Compliance Plan

• summarize your site’s regulated activities for

data handling (four data lifecycle stages)

• summarize organizational roles

• summarize vendor roles

• list ongoing, overall controls (SOPs, policies,

training, auditing, etc.)

• summarize validation by system type

• explain risk-based approach

• layout overall timeline (and progress to-date)

• include pointers to audit plan, SOPs, policies,

completed validations, etc.

• site senior management should sign-off on


Cerulean

Quality Management System

Site Data Integrity ComplianceMaster Plan

www.Ceruleanllc.com

www.Ceruleanllc.com


Identify & Prioritize SOPs

• Good data integrity practices

• Conducting internal data quality audits

• Determining Part 11/Annex 11 applicability

• Validating a commercial-off-the-shelf (COTS) system

• Validating spreadsheets and macros

• Validating hosted computerized systems

• Managing computerized system configurations and changes

• Ensuring Part 11/Annex 11 compliance at suppliers

• Scanning paper records (to make true copies)

• Archiving digital records

• Verifying and auditing digital record archives

• Reviewing and destroying expired archived records


Potential examples to consider

Identify & Prioritize Data Systems


Processes (and systems) that generate, manipulate, transmit, control, provide or store data used to:

• directly support regulatory submissions and reporting

• manage and control and test for critical parameters in manufacturing

• test and release finished product

• used to carry out product recalls

• adverse event and complaint handling, investigation and management

• support post-market surveillance

Lab systems and factory floor systems

And don’t forget – under NIPP, data controls here are likely to be area of scrutiny

www.Ceruleanllc.com


Regulator’s Computerized System

Inventory Format*Type Area/Site Product Name,

Purpose & Supplier

Version or Model

Last Validation Date

Most RecentChanges (within past year)

Networked(onsite)

Labs (all) ChromeleonChromotographyData System(ThermoScientific)

v 6.9 Dec. 2014 Change controls#73, 76, 81

Hosted SaaS

Corporate(all sites)

TrackWise EQMS(Sparta)

v 8.1 Nov. 2015 Change controls #81, 111

Stand-alone

QC Lab Excel SampleTracking Worksheet(Microsoft with custom macros)

v Office2013

April 2016 n/a


*Source:MHRA Inspection Notification Letter

Identify & Prioritize Ongoing

Controls

• data integrity and cGXP training

• system validation projects – including supplemental validations

• QA internal audits of IT controls, documentation, and SOPs

• periodic audits of critical suppliers (CMOs, contract sterilizers, contract labs, etc.)

• periodic “fire drills” for data archives

• formal periodic progress checks on data integrity compliance implementation plan (site by site, corporate wide)

• periodic independent gap assessments and/or training


www.Ceruleanllc.com


Periodic Independent Data

Integrity Tasks to Consider

• Run a training workshop on data integrity (combine with a gap assessment of your site)

• Help you create dataflow maps matched to critical business systems and workflows

• Conduct a gap analysis of data integrity controls

– across labs or plants or suppliers

– create a baseline across multiple sites (e.g., 4 different types of sites)


questions…?


www.Ceruleanllc.com


Request Expert Training

at Your Site

Visit Cerulean’s website to learn more about our

private, highly interactive workshops

held onsite at your location

Sample agendas and Request form at

www.ceruleanllc.com/private-training/


About Your PresenterJohn Avellanet

Trainer for FDA and Health Canada inspectors on advanced

data integrity inspection techniques and detecting data fraud in

clinical, laboratory, and manufacturing operations

Served on behalf of the US Department of Justice as the

independent overseer for the five-year, multi-million dollar Dr.

Comfort Corporate Integrity Agreement

Industry reviewer for the international standard, BS 10008

Evidential Weight and Legal Admissibility of Electronic

Information (2015)

Lead expert for the ISPE GAMP Data Integrity Working Group

Author of Get to Market Now! Turn FDA Compliance into a

Competitive Edge in the Era of Personalized Medicine (2010);

co-author of Pharmaceutical Regulatory Inspections (2014)

Prior to founding Cerulean, John spent more than 15 years

designing, implementing, and being accountable for quality

systems and data compliance programs for FDA, DEA, BIS,

ICH, IMDRF, and ISO

[email protected]

www.ceruleanllc.com


www.Ceruleanllc.com


thank you


Picture Credits

Photos, images and clip art that appear on these slides have been used to enhance this presentation and may NOT

be used for commercial or promotional purposes without permission from copyright holders.

Do not remove or copy from this presentation.

Contact:

iStockphoto

Cerulean Associates LLC


PCPC Data Integrity Case Studies

Documents

Transcript of PCPC Data Integrity Case Studies