Periodic Audit Trail Review - Scope, Content & Frequency

v9991

Trusted Information Resource
#1
'audit trail review' is required to reflect the state of 'control' and usually carried out at broadly two levels.
1. at the end and specific to the unit-operation / analysis (batch or unit operation or analysis for specific equipment)...there is broad clarity on the audit trail review criteria (contents) for specific to the batch


2. my query is about the second one, "predefined/pre determined frequency - WHEN - HOW/WHAT"

note :-
I was looking for any templates (could only find certain guidances...enclosed); there will be more iterations if we start building from these references., hence looking for some template/advanced starting point.
 

Attachments

Elsmar Forum Sponsor

yodon

Staff member
Super Moderator
#2
Sorry, I got lost... from what document are you quoting (regarding frequency)?

Audit trail information should be part of the computer system validation. Unless the software (or configuration) changes, the audit trail data will be consistently generated so after validation, checking doesn't make much sense. In fact, the GAMP annex you attached states:

Audit trails should be regarded primarily as a tool to be used for investigation, as and when required, rather than for continuous routine review. Routine review of all audit trail content is not required, and is not consistent with a risk-based approach. The cost and effort is not justified by any likely benefit.

I guess it wouldn't hurt to do a routine check to ensure, for example, that the environment hasn't changed (especially true for SaaS applications / cloud-based / delivered applications) but otherwise, I don't see value.
 

v9991

Trusted Information Resource
#3
Sorry, I got lost... from what document are you quoting (regarding frequency)?
here we go...
GUIDANCE ON GOOD DATA AND RECORD MANAGEMENT
PRACTICES

Systems typically include many metadata fields and audit trails. It is expected that during validation of the system the organization will
establish
based upon a documented and justified risk assessment the frequency, roles and responsibilities, and approach to review of
the various types of meaningful metadata, such as audit trials. For example, under some circumstances, an organization may justify
periodic review of audit trails that track system maintenance activities, whereas audit trails that track changes to critical GxP data with
direct impact on patient safety or product quality would be expected to be reviewed each and every time the associated data set is being
[/QUOTE]

reviewed and approved
and prior to decision-making.


Data Integrity and Compliance With CGMP
7. How often should audit trails be reviewed?.................................................................................6
226 each record and before final approval of the record. Audit trails subject to regular review 227 should include, but are not limited to, the following: the change history of finished 228 product test results, changes to sample run sequences, changes to sample identification, 229 and changes to critical process parameters. 230
231
FDA recommends routine scheduled audit trail review based on the complexity of the 232 system and its intended use.

 
MHRA GMP Data Integrity Definitions and Guidance for Industry March 2015
Audit trail review should be part of the routine data review / approval process, usually performed by the operational area which has generated the data (e.g. laboratory). There should be evidence available to confirm that review of the relevant audit trails have taken place. When designing a system for review of audit trails, this may be limited to those with GMP relevance (e.g. relating to data creation, processing, modification and deletion etc). Audit trails may be reviewed as a list of relevant data, or by a validated ‘exception reporting’ process. QA should also review a sample of relevant audit trails, raw data and metadata as part of self inspection to ensure on-going compliance with the data governance policy / procedures.
coming to the real part....I agree with the emphasis on the "intent/value" of the exercise. (frequency is mere part of the it_)

Let me try...
a. details specific/relevant to the decision ought to be reviewed before releasing the cgxp data. (activity driven, viz., before completing an unit operation viz., drying, its appropriate to have reviewed relevant data/records from the system, viz., excursions, recipe, etc)

b. others such as changes (periodic and routine (not the daily ones)) need to be reconciled.
For example, under some circumstances, an organization may justify
periodic review of audit trails that track system maintenance activities,
 
Last edited:

Ajit Basrur

Staff member
Admin
#4
It is already there in one of the FDA documents that you have attached ....

FDA recommends that audit trails that capture changes to critical data be reviewed with each record and before final approval of the record.

Audit trails subject to regular review should include, but are not limited to, the following: the change history of finished product test results, changes to sample run sequences, changes to sample identification, and changes to critical process parameters.

FDA recommends routine scheduled audit trail review based on the complexity of the system and its intended use.
 

v9991

Trusted Information Resource
#5
It is already there in one of the FDA documents that you have attached ....[/B]
:agree1: :yes:

v9991
2. my query is about the second one, "predefined/pre determined frequency - WHEN - HOW/WHAT"

note :-
I was looking for any templates (could only find certain guidances...enclosed); there will be more iterations if we start building from these references., hence looking for some template/advanced starting point.
 

Ajit Basrur

Staff member
Admin
#6
I do not have a checklist but preparing one just by listing the requirements from EU GMP Annex 11 and FDA 21 CFR Part 11 would be a great start.

Also refer THIS to get more questions.
 

v9991

Trusted Information Resource
#7
we ended up covering following in periodic review checklist for audit trails.

* Data
review of 'reasons' captured for each activity.

* System
any standby/backup system triggered
review of time syn in the lab.
any system/policy updates.

* user ids and privileges changes if any.
No. of instances where user id is locked or pwd reset .
No. of ids activated / deactivated.
any change in the admin/qa pwds.

* backup & archival
compliance as per the changes
status of any data 'restored'
backup status of the log for audit trail.

of course, there will be other application/lab specific requirements.
 

v9991

Trusted Information Resource
#8
https://www.agilent.com/cs/library/...our_lab_for_a_data_integrity_audit_Wright.pdf
Data Integrity (DI) questions:
• Is electronic data available?​
• Is electronic data reviewed?​
• Is meta data (audit trails) reviewed regularly?​
• Are there clear segregation of duties?​
• Has the system been validated for its intended use?​
Internal Data Review External (Auditor)
Data Review
• Analysis performed as per the monograph.​
• Sequence information correct.​
• Chromatography is typical.​
• SST acceptance criteria achieved.​
• NO “conditioning” or “test” injections using the sample (use a standard or control sample if specified by your procedures and monograph).​
• Correct integration (pay attention to MANUAL integration).​
• Chromatography appropriately scaled.​
• Individual results duplicate and meet specification.​
• Check the sequence and individual injection audit trail - any atypical / suspect activity?​
• Data processing: - Do the audit trail comments provide traceability? - Can the reprocessing be justified?​
• Check electronic results within the CDS match results reported on hard copy chromatography or in LIMS / SAP systems.​
Administration control
• Individual user profiles and passwords.​
• Clear segregation of duties within user profiles.​
• Restricted privileges for user (cant delete / over-write / move).​
• Audit trail functionality switched ON.​
• Date / time functionality locked by IT.​
• Lab Demo – User log-on (multiple), date / time locked, cant delete data.​
• Data recall – Electronic sequence / data file recall in lab using staff member. Data recall needs to be fast and efficient.​
• Data review – Chromatography scaling, integration and electronic results.​
• Audit trail review – looking for suspicious activity, justification of processing.​
• Training – assess staff competency with CDS in lab. Make sure staff are trained to interact with the auditor. Have a CDS superuser present during the lab inspection.​
• Query search –assurance that batch hasn’t been analysed multiple times as part of an investigation.​
https://www.fda.gov/downloads/About...fMedicalProductsandTobacco/CDER/UCM561491.pdf
Current expectations and guidance, including data integrity and compliance with CGMP

Regular Review Scheduled Review
• Overwriting​
• Aborting runs​
• Testing into compliance​
• Deleting​
• Backdating​
• Altering data​
• (not an all-inclusive list)​

How are you fulfilling the FDA's Audit Trail expectations for Data Integrity?
Regular Review Scheduled Review

the change history of finished product test results,
changes to sample run sequences,
changes to sample identification, and
changes to critical process parameters
• Look for unusual login activity​
• Monitor record deletion (if such an activity is not permissible)​
• Monitor changes to critical system configuration records​
• Monitor user role changes​
• Monitor abnormal, disallowed or unusual record state changes​
• Monitor system logs for critical application errors and correlate them with user activity​
• And much more….​

http://www.cbinet.com/sites/default/files/files/Longden_Heather_pres.pdf
 Audit trails tell us WHO did WHAT, WHEN automatically
 Audit trails tell us WHY as defined by the user
 They have two primary purposes:
– Give a history to the data, to help decide if it can be trusted​
– They should deter wrongdoing (think of CCTV)​
o Without review, they are not a deterrent

Audit trail record…at least the following information
Name of the person who made – the change to the data; – Description of the change; – Time and date of the change; – Justification for the change;

• the change history of finished product test results,​
• changes to sample run sequences,​
• changes to sample identification,​
• changes to critical process parameters. ( not “processing” parameters)​
– routine scheduled audit trail review based on the complexity of the system and its intended use • include discrete event logs, history files, database queries or reports​
• require specific training in evaluating the configuration settings and reviewing electronic data and metadata, such as audit trails, for individual computerized systems​
• correct use of Admin functionalities​
• determine if any retesting or additional testing of new functionality is required​
• Deleting Data only by designated administrators and WHY​
•​
•  Creating projects only by designated administrators​
•  Regular archiving of projects / altering access or status of projects​
•  Altering System Policies  User creation patterns​
•  Password resetting activity​
•  Alteration of systems​
•  Changes to roles​

How are you fulfilling the FDA's Audit Trail expectations for Data Integrity?

• Look for unusual login activity​
• Monitor record deletion (if such an activity is not permissible)​
• Monitor changes to critical system configuration records​
• Monitor user role changes​
• Monitor abnormal, disallowed or unusual record state changes​
• Monitor system logs for critical application errors and correlate them with user activity​
• And much more...​
http://microsep.co.za/wp-content/up...-Frylinck-Empower-Electroninc-Data-Review.pdf

SYSTEM AUDIT TRAIL
• Deleting data only by designated administrators and WHY​
• Creating projects only by designated administrators​
• Regular archiving of projects / altering access or status of projects​
• Altering System Policies​
• User creation patterns​
• Password resetting activity​
• Unauthorised access to system​
• Alteration of systems​
• Changes to roles​
• Access to system at non working time​
• Restore of Projects and Project Integrity​
• Check on performance of IQ (Warning, Error)​
• Archive and Removal of Audit Trail​
 Unsuccesful Attempt to Confirm Identity​
PERIODIC AUDIT TRAIL
It’s like an internal audit on the compliance of the system – Find concerns BEFORE the audit –​
Find ways to improve the efficiency of systems and processes –​
Documented evidence of actively searching for data integrity issues –​
Eg Review System Audit Trail for correct use of Admin functionalities​
Review major and minor changes to determine if any retesting or additional testing of new functionality is required
– Has it significantly expanded or changed use​
– Is the system still in control and in a validated state?​
http://www.who.int/medicines/areas/...a-management-practices_QAS15-624_16092015.pdf

Management reviews and regular reporting of quality metrics
• tracking and trending the occurrence of invalid and aberrant data may reveal unforeseen variability in processes and procedures previously believed to be robust, opportunities to enhance analytical procedures and their validation, validation of processes, training of personnel or sourcing of raw materials and components; Working document QAS/15.624 page 13​
• regular review of audit trails may reveal incorrect processing of data and help prevent incorrect results from being reported and identify the need for additional training of personnel;​
• routine inspections of computerized systems may reveal gaps in security controls that inadvertently allow personnel to access and potentially alter time/date stamps. These findings help raise awareness to management of need to allocate resources to improve computerized systems validation controls;​
• monitoring of contract acceptors and tracking and trending of associated quality metrics for these sites help to better identify risks that may indicate the need for more active engagement and allocation of additional resources by the contract giver to ensure quality standards are met.​
Read pg.18 onwards

Data Integrity ALCOA+
A Attributable Who acquired the data or performed an action​
L Legible Can you read and understand the data entries?​
C Contemporaneous Was it documented at the time of the activity​
O Original Is is the first recorded observation (or a verified true copy)?​
A Accurate Is it scientifically valid with no errors?​
+​
Complete All data including any repeat or reanalysis performed​
Consistent All elements of the analysis are dated/time stamped and in the expected sequence​
Enduring Recorded in a permanent, maintainable form for the useful life​
Available For review, audit or inspection over the lifetime of the record

http://www.formpipe.com/Global/Life Science/Demo 2017/Data Integrity 2017.pdf
Common Data Integrity Issues

Common passwords :- Analysts share passwords, unable to identify who created or changed a record​
User privileges :- System configuration does not adequately define or segregate user levels Users have access to unauthorised functions​
Computer System Operational Controls :- Inadequate controls over data Unauthorised access to modify or delete files No automatic saving of files, records not accurate or complete​
Processing methods :- Integration parameters not controlled, chromatograms may be re-integrated without correct change process​
Audit trails :- Functionality turned off, no complete record of the data life cycle – who modified a file and why​
Conflict of interest :- Business process owners granted enhanced security access e.g. system administrator “Unofficial” documentation Recording data first on a scrap of paper then transferring to the official document (e.g. the laboratory notebook)​
Failure to review “original data” :-Data and metadata not reviewed together to ensure context is maintained Errors or omissions may be undetected​
Inadequate data retention arrangements :- Failure to avoid inadvertent or deliberate alteration or loss throughout the retention period​

http://www.pharmacy.tcd.ie/assets/pdf/QPFORUM-Data Integrity-BBuhlmann.pdf

Data Integrity Continuum
System Error (ignore)
Individual Mistake (Sloppiness)
Individual Malfeasance (sleaziness)
Institutional Malfeasance (fraud )

Data Review
Good Documentation Practices - Legible, Contemporaneous, Permanent, Attributable, Traceable, Time/Date Stamped​
System Audit Trail Tracks actions of System Administrator Reviewed periodically based on risk Defined in Administrators SOPs​
Data Audit Trail Tracks actions of users, reviewers, and approvers Is reviewed when the data is reviewed Defined in User Operational SOPs​

https://www.agilent.com/cs/library/...our_lab_for_a_data_integrity_audit_Wright.pdf

http://media.firabcn.es/content/S109015/Presentaciones/coso_anna.pdf

1. Results discarded without explanation​
2. Overwriting electronic raw data files for on-going sequences​
3. EM plates without evidence of contact (finger prints)​
4. Operators with several profiles in a system​
5. Dates of # print outs without appropriate correlation​
1. Which profiles and privileges are defined?​
2. Who could change the data?​
3. Is the e-data reviewed, or only paper data?​
4. How do you manage your automated IPC controls?​
5. Where do you keep your back-ups?​

Important Questions and Answers concerning the Audit Trail Review

Important Questions and Answers concerning the Audit Trail Review - ECA Academy

Important Questions and Answers concerning the Audit Trail Review - Part 2 - ECA Academy

Understanding Audit Trail Requirements in Electronic GxP Systems
The audit trail must be:

Automated The audit trail entries must be automatically captured by the computer system whenever an electronic record is created, modified or deleted.​
Secure Audit trail data must be stored in a secure manner and must not be editable by any user.​
Contemporaneous Each audit trail entry must be time stamped according to a controlled clock which cannot be altered. The time should either be based on central server time or a local time, so long as it is clear in which time zone the entry was performed.​
Traceable Each audit trail entry must be attributable to the individual responsible for the direct data input. Updates made to data records must not obscure previous values and where required by regulation the reason for changing the data must also be recorded.​
Archived The audit trail must be retained as long as the electronic record is required to be stored.​
Available The audit trail must be available for agency review and copying.​
Audit trail content and reason it is required:
Identification of the User making the entry This is needed to ensure traceability. This could be a user’s unique ID, however there should be a way of correlating this ID to the person.​
Date and Time Stamp This is a critical element in documenting a sequence of events and vital to establishing an electronic record’s trustworthiness and reliability. It can also be effective deterrent to records falsification.​
Link to Record This is needed to ensure traceability. This could be the record’s unique ID.​
Original Value This is needed in order to be able to have a complete history and to be able reconstruct the sequence of events​
New Value​
Reason for Change This is only required if stipulated by the regulations pertaining to the audit trailed record. (See below)​
Are Digital Signatures Accepted by Lab Regulators? Best Practice ELNs Ensure Validity

http://www.formpipe.com/Global/Life Science/Demo 2017/Data Integrity 2017.pdf

DATA INTEGRITY CHECKLIST

 

Attachments

Last edited by a moderator:
Thread starter Similar threads Forum Replies Date
S Auditing our own activity - Periodic audit of our documentation Internal Auditing 11
S ISO 14001 - Periodic External Audit by Consultant Requirement? ISO 14001:2015 Specific Discussions 99
eule del ayre Documented Information - Periodic Review of Documents? IATF 16949:2016 / ISO 9001:2015 IATF 16949 - Automotive Quality Systems Standard 34
D CSV - Periodic Review Qualification and Validation (including 21 CFR Part 11) 1
Marc Interesting Discussion The periodic table is 150 years old - March 2019 Coffee Break and Water Cooler Discussions 3
P Periodic Review - Deviations/incidents that are reviewed Qualification and Validation (including 21 CFR Part 11) 5
M FDA or CE requirements for periodic checks of data backups and retrievals EU Medical Device Regulations 3
T Regulation (EU) 2017/745 and PSUR (periodic safety update report) EU Medical Device Regulations 9
V Periodic review criteria for reviewing/updating SOPs US Food and Drug Administration (FDA) 1
P EU MDR and PSUR (Periodic Safety Update Report) EU Medical Device Regulations 31
B IATF 16949 Cl. 8.5.1.5 - Requirement for Periodic Overhaul IATF 16949 - Automotive Quality Systems Standard 10
S How to document revisions after Periodic Review of documents ? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
I Does ISO 13485 have a Periodic Document Review Requirement? ISO 13485:2016 - Medical Device Quality Management Systems 7
A Current Control Periodic Document Reviews - Process Updates in the FMEA FMEA and Control Plans 3
F Periodic Review Requirements for Software - Medical DeviceS Software Quality Assurance 2
T Periodic Quality Manual and Procedure Review Requirements General Auditing Discussions 5
S Is hiring a CB for certification and periodic audits an outsourced activity? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 20
V Where & How to Capture Details of Periodic Reviews of Change Control & CAPAs Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 8
T Clause for Maintenance - Periodic Maintenance Activity (Oil change) Internal Auditing 4
C API Q1/ISO-TS 29001 Periodic Assessment of Stock Oil and Gas Industry Standards and Regulations 3
B Can I Claim That Periodic Calibration is Not Required? ISO 13485:2016 - Medical Device Quality Management Systems 6
Q Computerized System Periodic Review Requirement - Pharma Company Qualification and Validation (including 21 CFR Part 11) 7
R Periodic evaluation of suppliers - ISO 17025 4.6 Purchasing services and supplies ISO 17025 related Discussions 3
R Do Engineering Standards require periodic review? ISO 9001 4.2.4 (b) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 21
M Documents of External Origin - Periodic or annual verification of revision status ISO 13485:2016 - Medical Device Quality Management Systems 10
M Surface plates - Questioning the need for periodic calibration of surface plates General Measurement Device and Calibration Topics 12
I Periodic Review of Process and Equipment Validation Qualification and Validation (including 21 CFR Part 11) 4
R Periodic / Maintenance Audits Schedule ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
D Periodic Product Requalification Process IATF 16949 - Automotive Quality Systems Standard 1
tony wardle Periodic Evaluation of Legal Compliance - ISO 14001 - 4.5.2.1 - Legal Requirements ISO 14001:2015 Specific Discussions 22
K Periodic validation of raw material test reports (7.4.3) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
K Review complete, no changes needed - When a Procedure is due for a periodic review? Document Control Systems, Procedures, Forms and Templates 10
T QP-5.6.0 - Management Review should be both periodic and continual Management Review Meetings and related Processes 14
A Meeting the ISO 14001 Periodic Audits Requirement - Internal vs. External Auditing ISO 14001:2015 Specific Discussions 11
D ISO 9004 6.2.2.2 has listed "periodic refresher programs for people already trained" General Auditing Discussions 6
M What does the APQP manual mean by the term 'Periodic Requirements' FMEA and Control Plans 2
J Document Control NCR - Periodic Reviews Document Control Systems, Procedures, Forms and Templates 3
M Doubt about the element 15.3 (VDA) - Periodic cross-check and repeated inspections VDA Standards - Germany's Automotive Standards 3
M Audit Criteria Training Materials Internal Auditing 1
K New supplier audit as per V3.1 by French Automotive OEM General Auditing Discussions 2
S Complexity Rating - CB adding another audit day for "high complexity" AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
B Looking for 10 Internal Audit Online Training Participants ISO 17025 related Discussions 2
K MDSAP Audit Approach 2020 for Brazil Other Medical Device Regulations World-Wide 1
K MDSAP Audit Approach 2020 ISO 13485:2016 - Medical Device Quality Management Systems 1
B IATF16949 audit requirement - Auditor request UCL and LCL must be show Xbar-R, IATF 16949 - Automotive Quality Systems Standard 7
T COVID, Furlough and ISO9001 Surveillance Audit Coffee Break and Water Cooler Discussions 2
R External Audit and Certificate prorogation due to the pandemic General Auditing Discussions 10
Dean Bell Implementation of Controls as per SOA for Stage 2 Audit IEC 27001 - Information Security Management Systems (ISMS) 0
G Logistic organization and controls - IATF/ISO 9001 audit Nonconformance and Corrective Action 2

Similar threads

Top Bottom