Measureables for Document Control and Internal Audits

J

JaneB

#61
I never said you should not monitor things like internal audits or document control. Measuring them is a waste of time unless they have been identified as critical areas. In an earlier post I suggested a way of monitoring these processes...I'm sure there are others
There's a difference in my mind between monitoring something (is it going OK?) and assessing something (is it delivering value? is this a useful activity?).

Perhaps it's the words themselves, or their application. For while I quite see the difficulty associated with measuring, say, internal audit - I still think it's valuable to have a way of assessing whether one is getting that nebulous 'value' from internal audit. And presumably we need to be able to tie down what 'value' is a bit more closely to see if we are/are not getting it.
 
Elsmar Forum Sponsor
C

CliffK

#62
There's a difference in my mind between monitoring something (is it going OK?) and assessing something (is it delivering value? is this a useful activity?).

Perhaps it's the words themselves, or their application. For while I quite see the difficulty associated with measuring, say, internal audit - I still think it's valuable to have a way of assessing whether one is getting that nebulous 'value' from internal audit. And presumably we need to be able to tie down what 'value' is a bit more closely to see if we are/are not getting it.
AndyN has already answered this question.

I would only add that the percentage of managers who request an audit will always be low, because internal audits are grossly inefficient ways to monitor and measure quality system effectiveness.

It all comes back to this question: who in his or her right mind would have an internal audit program if it were not required by some standard?
 

Helmut Jilling

Auditor / Consultant
#63
...It all comes back to this question: who in his or her right mind would have an internal audit program if it were not required by some standard?
I am in the middle of giving a 3 day training on Internal "Improvement Auditing." I would completely disagree with that comment - the whole point of the training is to find value and improvements (including compliabce, but not limited to that). If your audit program does that, you will not want to quit. If your audit programs don't do that, you are missing a powerful opportunity.

A couple thoughts, processes must be measured (numbers, data, charts), or monitored. We consider monitoring to be observational, but not necessarily a numerical metric. I monitor the weather, but I don't turn it into a formal metric. The standard uses both terms, so they must infering a difference between them.

An audit can be used for monitoring, but it is a bit clumsy. I would look for better metrics when possible.

The best way to determine what to measure is to do a good job of defining the criteria for effectiveness. If you define that exception skills is a criteria for the Training Process, and you can tell me what you want to know about that, we can figure out a way to measure that criteria. Usually, the weak link is we have not defined the important criteria, so we are guessing at what should be measured.
 
Likes: db
C

CliffK

#64
I am in the middle of giving a 3 day training on Internal "Improvement Auditing." I would completely disagree with that comment - the whole point of the training is to find value and improvements (including compliabce, but not limited to that). If your audit program does that, you will not want to quit. If your audit programs don't do that, you are missing a powerful opportunity.
Maybe I should take your training.:)

I'm just skeptical about your statement. In 2+ decades at this, I have yet to see a positive ROI on internal audit activity.

Perhaps someone has some hard numbers? Anecdotal evidence?

If you want to make the process work or improve it, get the performers involved. That's where you get the bang for your buck.
 
#65
It all comes back to this question: who in his or her right mind would have an internal audit program if it were not required by some standard?
I too, want to disagree. I have several clients who are not registered. We can expand the question to who in his or her right mind would go through all of this if it were not a mandate? The bottom line is that we need ways to find value in all of our processes. That is why 4.1 e) exists.

Part of monitoring, measuring and analyzing processes is to make sure they are still effective (but how can you tell, if you have no measurables?). But a better reason is to find ways to improve the processes.

I have seen internal audit OFIs result in substantial savings in a company. I have also seen where they have uncovered unnecessary risks to customer satisfaction.
 

Sidney Vianna

Post Responsibly
Staff member
Admin
#66
It all comes back to this question: who in his or her right mind would have an internal audit program if it were not required by some standard?
Good question. Only those who have experienced internal audits that brought up real business benefits would consider doing it (and more of it) without a standard mandate. I reckon that those that benefit from internal audits are the minority, though. Highly effective internal audits are the exception, not the rule. But we all know that.
 
#67
It occurs to me that your observation is true, Sidney, partly due to the fact that auditor training and qualification is compliance based. I believe it's high time the powers that be should overhaul the internal auditor training requirements completely and make a differentiation, not just by name, but in the entire qualification process/personnel requirements/course providers methods etc.

Internal audit training has always been based on external audit methods. This thread has shown how much variation there is in defining what an effective audit 'looks like', and it's my long held belief that internal auditors should be part of the solution when they find things which need action, not simply report back to management "we're not following procedures".........in order to give the value of the audits back to the organization.

We could all sit around waiting for our senior management to 'get it', about audits - it won't happen and my understanding of behavioural psychology is that if you want to change something, you have to change first.........

Even ISO 19011 doesn't really address the needs of an effective internal audit program. It has always read like a book for managing external (i.e CB) audits.

Is anyone in agreement? How about a small revolution...............;)
 

Helmut Jilling

Auditor / Consultant
#68
...
I'm just skeptical about your statement. In 2+ decades at this, I have yet to see a positive ROI on internal audit activity.

Perhaps someone has some hard numbers? Anecdotal evidence?
Lot's of anecdotal...little really hard numbers...

If you want to make the process work or improve it, get the performers involved. That's where you get the bang for your buck.
Absolutely agree...

Maybe I should take your training.:)
Absolutely agree... :D
 
J

JaneB

#69
It all comes back to this question: who in his or her right mind would have an internal audit program if it were not required by some standard?
Someone in a field where it is useful, and with the intelligence and/or experience to know that a well planned and implemented audit program is of great value.

The argument of 'this wouldn't be done except for it being mandated in a Standard' puts the cart before the horse (an English expression meaning, 'wrong way around'). Why is it in a Standard? Because it's good practice.
 
J

JaneB

#70
I believe it's high time the powers that be should overhaul the internal auditor training requirements completely and make a differentiation, not just by name, but in the entire qualification process/personnel requirements/course providers methods etc.
Me too.

Internal audit training has always been based on external audit methods. ... Even ISO 19011 doesn't really address the needs of an effective internal audit program. It has always read like a book for managing external (i.e CB) audits.
You're right. It has - which makes for much confusion, & doesn't effectively address the needs of internal auditors. I've occasionally had experience of people coming back from training courses run by certifying bodies who have been taught to think that their sole role is to get the procedures & just look for compliance/non with them. Or again, who needs a formal 'opening' meeting in a company where you say hello to the people 10 times a day?

Is anyone in agreement? How about a small revolution...............;)
Count me in.
 
Thread starter Similar threads Forum Replies Date
P Received a minor for not having good measureables/goals. Need help with KPIs. IATF 16949 - Automotive Quality Systems Standard 52
A Measureables to Monitor an Integrated Management System Capability IATF 16949 - Automotive Quality Systems Standard 3
A Corrective Action Tracking - What Measureables and How to Track Status ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
J Lack of Measureables in our Quality Objectives ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
J Quality Objective Measureables - Please Review ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
N Quality Objective - Scrap Measureables ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
J Measureables to determine efficiency and effectiveness of the COP Processes IATF 16949 - Automotive Quality Systems Standard 4
H Ergonomics Metrics (Measureables) - Human factors engineering Lean in Manufacturing and Service Industries 39
D Design and Development Measureables - ISO9001:2000 Clause 7.3 Design and Development of Products and Processes 5
G Statistical Techniques Measureables for Application Development Company Statistical Analysis Tools, Techniques and SPC 3
1 Made it through Phase One - Measureables for Management Review - Need list or form Management Review Meetings and related Processes 9
K Business Plan - Quality Objectives and Measureables - TS 16949 Para 5.4.4 IATF 16949 - Automotive Quality Systems Standard 8
A Internal Audits - Design Process Measurement (Measureables) Design and Development of Products and Processes 6
Icy Mountain Quality Objectives Redux - Setting Actual Numbers (Measureables) IATF 16949 - Automotive Quality Systems Standard 15
L Measureables: Efficiency - Effectiveness of COP (Customer Oriented Process) and KSP ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
7 ISO 14001 Measureables - Plastics related industry Miscellaneous Environmental Standards and EMS Related Discussions 1
Marc Measuring and Improving Quality - Choosing Measureables Quality Tools, Improvement and Analysis 3
J Document Control Metrics Document Control Systems, Procedures, Forms and Templates 3
A Document Review and Document Approval --- 2 Signatures needed acc. §820.40? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
I Document Control Software Document Control Systems, Procedures, Forms and Templates 2
K 25-year lifetime of medical device - document storage period EU Medical Device Regulations 1
J UDI - Where to document it? EU Medical Device Regulations 8
T Controlling Expandable Forms in Paper-Based Document Control System Document Control Systems, Procedures, Forms and Templates 10
S Changing revision status of a reviewed document Manufacturing and Related Processes 4
shimonv Document Control Procedure Header Content Document Control Systems, Procedures, Forms and Templates 3
A Document release vs its related training. Which should come first? ISO 13485:2016 - Medical Device Quality Management Systems 18
Q Document ownerships Document Control Systems, Procedures, Forms and Templates 14
E Document Change Request forms (or no forms)? ISO 13485:2016 - Medical Device Quality Management Systems 9
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
B Document Approval Matrix Benchmarking Document Control Systems, Procedures, Forms and Templates 3
J Document Approval Signature Order Document Control Systems, Procedures, Forms and Templates 10
L How to add exemption or statement to control of document procedure? ISO 13485:2016 - Medical Device Quality Management Systems 5
M What is the proper way to document measurements Measurement Uncertainty (MU) 1
P Equipment 21 CFR 820.70(g) - User Requirements Document for Off the shelf equipment 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
B Do IFU designs have to be document controlled under ISO 13485? Document Control Systems, Procedures, Forms and Templates 2
C Certificate of Conformance Form - COC for each customer a controlled document? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
A Document "Correspondence IATF 16949 vs ISO13485" available? IATF 16949 - Automotive Quality Systems Standard 0
R Document Retention - Discard hard-copies after scanning? ISO 13485:2016 - Medical Device Quality Management Systems 2
P Mylar plot suppliers in accordance with D6-51991 document AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
A Document Change Notice vs complete System re-write Manufacturing and Related Processes 4
Q Document approval through SharePoint (without signature) Records and Data - Quality, Legal and Other Evidence 4
S What is considered a "core algorithm"? (From an FDA guidance document) Medical Information Technology, Medical Software and Health Informatics 4
U Document Approval - Software company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
O Please, help with document on ISO/ TS 17758 and (IDF 87: 1979). Thank you. Manufacturing and Related Processes 1
K Proper document of SMPS used in infant warmer for IEC 60601-1 testing IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
I Nitpicking on document released dates ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
I Document Control on Log Files ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
Q Refusal to Document Complaints Customer Complaints 39
Q Tracking Procedure Revisions (Document Control) Document Control Systems, Procedures, Forms and Templates 9
C Document Control Stamps - Does anyone still stamp their documents? Document Control Systems, Procedures, Forms and Templates 24

Similar threads

Top Bottom