Measureables for Document Control and Internal Audits

C

chaosweary

#31
Re: Measurables for Document Control and Internal Audits

How about just using an (potential)FMEA type approach and adding some cost columns aside from just the ISO compliance audits? Probably too simple of an approach...:2cents:
 
Elsmar Forum Sponsor

pondo

Registered Visitor
#32
Re: Measurables for Document Control and Internal Audits

Just measure time to closure.

If you discovered something non-conforming then it should be in your best interest to fix is ASAP - right?????
 

Jim Wynne

Staff member
Admin
#33
Re: Measurables for Document Control and Internal Audits

Just measure time to closure.

If you discovered something non-conforming then it should be in your best interest to fix is ASAP - right?????
Yes, but ASAP means "as soon as possible," and that might mean tomorrow, next week, or next year.
 

pondo

Registered Visitor
#34
Re: Measurables for Document Control and Internal Audits

Yes, but ASAP means "as soon as possible," and that might mean tomorrow, next week, or next year.
There is always a "minimum" amount of time.

It's also a reflection as to the robustness of the QMS and the priorities given to it.
 

Jim Wynne

Staff member
Admin
#35
Re: Measurables for Document Control and Internal Audits

There is always a "minimum" amount of time.

It's also a reflection as to the robustness of the QMS and the priorities given to it.
My point was that setting arbitrary time limits isn't a good idea. Some take longer than others, and if you set a limit that's too restrictive, you'll wind up with (A) a lot of half-arsed CAs, or (B) people having to explain needlessly why they didn't get their CAs done in time.
 
V

vanputten

#36
Re: Measurables for Document Control and Internal Audits

Measuring the number of findings is a terrible metric. How do you know if it is due to the ability of the auditor or changes in the business or both? And if you make a goal of X number of findings, you can be assured that the individual will find at least that number of things wrong whether vaild or not. Do not have a goal or metric related to the number of finding. Understand the impact of target setting.

[A little off topic, there are many, many things that cannot be measured or at least not measured accurately and realiably.]


From the Audit Guy, Dennis Arter, here are some measurables for the internal audit program:

Here are some suggested metrics for the audit boss to consider:

- Percentage of audits started on time
- Number of disputed audit findings
- Percentage of audit plans accepted with no modifications
- Percentage of audit reports completed on time (e.g. within x days of
audit)
- Percentage of audit reports accepted by audit boss on first pass (complete and accurate)
- Time period between the exit meeting and delivery of the written (paper or
e-mail) report
- Number of overdue responses to corrective action requests generated from adverse audit findings
- Turnover of the audit staff

Cutting through the fog, there are two fundamental questions for how well an individual auditor is performing:

a) Were adverse conclusions accepted by the audited party and actions taken to correct the condition?
b) Did the auditor find problems before they got huge or outsiders (regulators and customers) made a fuss over those problems?

Dennis Arter

Regards,

Dirk
 

Stijloor

Staff member
Super Moderator
#37
Re: Measurables for Document Control and Internal Audits

From the Audit Guy, Dennis Arter, here are some measurables for the internal audit program:

Here are some suggested metrics for the audit boss to consider:

- Percentage of audits started on time
- Number of disputed audit findings
- Percentage of audit plans accepted with no modifications
- Percentage of audit reports completed on time (e.g. within x days of
audit)
- Percentage of audit reports accepted by audit boss on first pass (complete and accurate)
- Time period between the exit meeting and delivery of the written (paper or
e-mail) report
- Number of overdue responses to corrective action requests generated from adverse audit findings
- Turnover of the audit staff

Cutting through the fog, there are two fundamental questions for how well an individual auditor is performing:

a) Were adverse conclusions accepted by the audited party and actions taken to correct the condition?
b) Did the auditor find problems before they got huge or outsiders (regulators and customers) made a fuss over those problems?

Dennis Arter

Regards,

Dirk
Hello Dirk,

Thank you so much for posting this. I am encouraged!

Stijloor.
 

pondo

Registered Visitor
#38
Re: Measurables for Document Control and Internal Audits

Measuring the number of findings is a terrible metric. How do you know if it is due to the ability of the auditor or changes in the business or both? And if you make a goal of X number of findings, you can be assured that the individual will find at least that number of things wrong whether vaild or not. Do not have a goal or metric related to the number of finding. Understand the impact of target setting.

[A little off topic, there are many, many things that cannot be measured or at least not measured accurately and realiably.]


From the Audit Guy, Dennis Arter, here are some measurables for the internal audit program:

Here are some suggested metrics for the audit boss to consider:

- Percentage of audits started on time
- Number of disputed audit findings
- Percentage of audit plans accepted with no modifications
- Percentage of audit reports completed on time (e.g. within x days of
audit)
- Percentage of audit reports accepted by audit boss on first pass (complete and accurate)
- Time period between the exit meeting and delivery of the written (paper or
e-mail) report
- Number of overdue responses to corrective action requests generated from adverse audit findings
- Turnover of the audit staff

Cutting through the fog, there are two fundamental questions for how well an individual auditor is performing:

a) Were adverse conclusions accepted by the audited party and actions taken to correct the condition?
b) Did the auditor find problems before they got huge or outsiders (regulators and customers) made a fuss over those problems?

Dennis Arter

Regards,

Dirk

:mg: After reading this I have to strongly agree with AndyN.

I just use time to closure just to have a measurement. The point is to continually improve, not set your quality system up for failure.
 
#39
Re: Measurables for Document Control and Internal Audits

Measuring the number of findings is a terrible metric. How do you know if it is due to the ability of the auditor or changes in the business or both? And if you make a goal of X number of findings, you can be assured that the individual will find at least that number of things wrong whether vaild or not. Do not have a goal or metric related to the number of finding. Understand the impact of target setting.

[A little off topic, there are many, many things that cannot be measured or at least not measured accurately and realiably.]


From the Audit Guy, Dennis Arter, here are some measurables for the internal audit program:

Here are some suggested metrics for the audit boss to consider:

- Percentage of audits started on time
- Number of disputed audit findings
- Percentage of audit plans accepted with no modifications
- Percentage of audit reports completed on time (e.g. within x days of
audit)
- Percentage of audit reports accepted by audit boss on first pass (complete and accurate)
- Time period between the exit meeting and delivery of the written (paper or
e-mail) report
- Number of overdue responses to corrective action requests generated from adverse audit findings
- Turnover of the audit staff

Cutting through the fog, there are two fundamental questions for how well an individual auditor is performing:

a) Were adverse conclusions accepted by the audited party and actions taken to correct the condition?
b) Did the auditor find problems before they got huge or outsiders (regulators and customers) made a fuss over those problems?

Dennis Arter

Regards,

Dirk
Dirk:
Dennis is a great guy with humugous experience of audits - external audits. Like many luminaries in the (ASQ) auditing world, there is a mushing together of the tools and techniques of external and internal audits. It can be seen all over the place. In principle they're very similar, but in practice, the audits are poles apart or should be.

Most of these metrics are for audit/auditors managers who have nothing better to do - and we have already established that in the main, most of these folks have a real daytime job.........
 
P

potdar

#40
Re: Measurables for Document Control and Internal Audits

Dirk:
Dennis is a great guy with humugous experience of audits - external audits. Like many luminaries in the (ASQ) auditing world, there is a mushing together of the tools and techniques of external and internal audits. It can be seen all over the place. In principle they're very similar, but in practice, the audits are poles apart or should be.

Most of these metrics are for audit/auditors managers who have nothing better to do - and we have already established that in the main, most of these folks have a real daytime job.........
OK. Third party audits, measurements, managemnt concerns and fudged results. I'll just quote my experience to show how the metrics get affected as many earlier have said.

I was working for one of the largest and revered multinational CBs. We issued Observations, Non-critical NCRs and Critical NCRs as an feedback of our audits. Non-critical said ok, commit action, set it right . We will be back after a year and check it during surveillance. Critical said sorry, no certification till you set it right. Correct these and call us for a re-audit.

When I joined, auditors used to issue critical NCRs. Then the focus of the management (of the CB) shifted to increasing the customer base. Metric of the branch managers was 'how many new certificates did you issue this quarter?' So, the critical NCRs dwindled and finally came to zero. They were now reworded and disguised as non-critical.

Next, the system of issuing non-critical NCRs was dropped by the CB (the foreign parent). Now we either had an Observation or an NCR. NCR said re-audit, which the branch manager's metric won't allow. So, all NCRs got reworded as Observations. Even without assigning any metrics, these were inefficient audits.

Again, I shall repeat my contention that if the measure for audit effectiveness is going to work at cross purposes with the measure for branch effectiveness, having both the measures is an invitation to chaos. In such a situation, branch effectiveness should prevail.

Thats what happened and many good auditors quit.:mg: Possibly the metric amongst the list given by Dennis.

Internal auditors are subjected to much larger pulls and pressures. Why complicate their jobs by putting metrics on a part time function to be done by them? I can write separately on how they are alredy subject to many criss-crossing targets and doing a constant balancing act. If a simpler way out is found to measure the effectiveness and take the load off their head, it is for the good of the organisation.
 
Thread starter Similar threads Forum Replies Date
P Received a minor for not having good measureables/goals. Need help with KPIs. IATF 16949 - Automotive Quality Systems Standard 52
A Measureables to Monitor an Integrated Management System Capability IATF 16949 - Automotive Quality Systems Standard 3
A Corrective Action Tracking - What Measureables and How to Track Status ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
J Lack of Measureables in our Quality Objectives ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
J Quality Objective Measureables - Please Review ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
N Quality Objective - Scrap Measureables ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
J Measureables to determine efficiency and effectiveness of the COP Processes IATF 16949 - Automotive Quality Systems Standard 4
H Ergonomics Metrics (Measureables) - Human factors engineering Lean in Manufacturing and Service Industries 39
D Design and Development Measureables - ISO9001:2000 Clause 7.3 Design and Development of Products and Processes 5
G Statistical Techniques Measureables for Application Development Company Statistical Analysis Tools, Techniques and SPC 3
1 Made it through Phase One - Measureables for Management Review - Need list or form Management Review Meetings and related Processes 9
K Business Plan - Quality Objectives and Measureables - TS 16949 Para 5.4.4 IATF 16949 - Automotive Quality Systems Standard 8
A Internal Audits - Design Process Measurement (Measureables) Design and Development of Products and Processes 6
Icy Mountain Quality Objectives Redux - Setting Actual Numbers (Measureables) IATF 16949 - Automotive Quality Systems Standard 15
L Measureables: Efficiency - Effectiveness of COP (Customer Oriented Process) and KSP ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
7 ISO 14001 Measureables - Plastics related industry Miscellaneous Environmental Standards and EMS Related Discussions 1
Marc Measuring and Improving Quality - Choosing Measureables Quality Tools, Improvement and Analysis 3
J Document Control Metrics Document Control Systems, Procedures, Forms and Templates 3
A Document Review and Document Approval --- 2 Signatures needed acc. §820.40? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
I Document Control Software Document Control Systems, Procedures, Forms and Templates 2
K 25-year lifetime of medical device - document storage period EU Medical Device Regulations 1
J UDI - Where to document it? EU Medical Device Regulations 8
T Controlling Expandable Forms in Paper-Based Document Control System Document Control Systems, Procedures, Forms and Templates 10
S Changing revision status of a reviewed document Manufacturing and Related Processes 4
shimonv Document Control Procedure Header Content Document Control Systems, Procedures, Forms and Templates 3
A Document release vs its related training. Which should come first? ISO 13485:2016 - Medical Device Quality Management Systems 18
Q Document ownerships Document Control Systems, Procedures, Forms and Templates 14
E Document Change Request forms (or no forms)? ISO 13485:2016 - Medical Device Quality Management Systems 9
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
B Document Approval Matrix Benchmarking Document Control Systems, Procedures, Forms and Templates 3
J Document Approval Signature Order Document Control Systems, Procedures, Forms and Templates 10
L How to add exemption or statement to control of document procedure? ISO 13485:2016 - Medical Device Quality Management Systems 5
M What is the proper way to document measurements Measurement Uncertainty (MU) 1
P Equipment 21 CFR 820.70(g) - User Requirements Document for Off the shelf equipment 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
B Do IFU designs have to be document controlled under ISO 13485? Document Control Systems, Procedures, Forms and Templates 2
C Certificate of Conformance Form - COC for each customer a controlled document? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
A Document "Correspondence IATF 16949 vs ISO13485" available? IATF 16949 - Automotive Quality Systems Standard 0
R Document Retention - Discard hard-copies after scanning? ISO 13485:2016 - Medical Device Quality Management Systems 2
P Mylar plot suppliers in accordance with D6-51991 document AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
A Document Change Notice vs complete System re-write Manufacturing and Related Processes 4
Q Document approval through SharePoint (without signature) Records and Data - Quality, Legal and Other Evidence 4
S What is considered a "core algorithm"? (From an FDA guidance document) Medical Information Technology, Medical Software and Health Informatics 4
U Document Approval - Software company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
O Please, help with document on ISO/ TS 17758 and (IDF 87: 1979). Thank you. Manufacturing and Related Processes 1
K Proper document of SMPS used in infant warmer for IEC 60601-1 testing IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
I Nitpicking on document released dates ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
I Document Control on Log Files ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
Q Refusal to Document Complaints Customer Complaints 39
Q Tracking Procedure Revisions (Document Control) Document Control Systems, Procedures, Forms and Templates 9
C Document Control Stamps - Does anyone still stamp their documents? Document Control Systems, Procedures, Forms and Templates 24

Similar threads

Top Bottom