Measureables for Document Control and Internal Audits

J

JaneB

#11
Re: Measurables for Document Control and Internal Audits

Interesting question, and discussion, with some very good points made both on the 'for' measuring and 'against' sides.

I'd be unhappy about not doing any measurement or assessment at all, and most particularly of internal audit. Though I don't take the view that one has to measure 'everything'. But then, I'm not an engineer, either. Which also means that I am quite willing to accept subjective measures where they are appropriate - I'd argue that subjective assessment in certain areas is very appropriate, eg. 'satisfaction' or 'perceived value'. I don't share a desire to drive intelligent, subjective analysis/feedback from people out of everything.

If no one is doing any assessing of whether the customers of IA are satisfied with it (for example), I think there's strong risk of having a process that isn't perceived to be of much value, and consequencies include low (if any) drive to improve.

For document control, depends a lot on how important it is. Sounds like the OP is in a situation where it is - but time to change perhaps not, or not easily assessable. What instead? Um - 'perceived timeliness'? Accuracy? 'Documents I need are available when & where I need them'?

I think Duke's given excellent advice - I've taken the liberty of reformatting it a bit:
Ask yourself (the organization) who the customers are of those two processes. Then go talk to the customers and ask them
1. what is important about the outcomes of those processes to them
2. which measurable outcomes are most important, and
3. what improvements they perceive would be most useful.

Then establish metrics around the ones that make more sense to measure and set targets for improvement.
We all tend to think of our own customers/clients/situations when responding - do you know what yours want and if they're getting it?

PS: Oops - I just realised (sorry) I strayed into the automotive forum without noticing, so apologies. My comments were intended for 9001 environments.
 
Last edited by a moderator:
Elsmar Forum Sponsor

Stijloor

Staff member
Super Moderator
#12
Re: Measurables for Document Control and Internal Audits

Friends,

If the benefit of any process to the organization cannot be demonstrated through clear objectives and associated measurements, that process should be eliminated because it does not add value. What's so wrong with my reasoning? In my many years of experience, I have seen this work very well including internal auditing and control of documents. Now I'll rest my case.

Stijloor.
 
P

potdar

#13
Re: Measurables for Document Control and Internal Audits

Friends,

If the benefit of any process to the organization cannot be demonstrated through clear objectives and associated measurements, that process should be eliminated because it does not add value. What's so wrong with my reasoning? In my many years of experience, I have seen this work very well including internal auditing and control of documents. Now I'll rest my case.

Stijloor.
Nothing wrong. But then nothing is wrong with evaluating the benefits of these processes as a part of a larger prcess called say 'MR's functions'. What matters is that the processes get evaluated. Not the name under which they get evaluated.

'MR's functions' in my opinions has certain advatages over evaluating the sub processes individually.
 

Stijloor

Staff member
Super Moderator
#14
Re: Measurables for Document Control and Internal Audits

Metrics/objectives should be focused on business processes, and only on support processes where their impact on business processes is significant.
When a support process fails to deliver the expected results, any business process supported by it will be negatively affected. That's why it is called a "support process." Whether you call that impact "significant" or otherwise is subject to interpretation.
 
#15
Re: Measurables for Document Control and Internal Audits

My original response was intended to describe a process that can be used to identify metrics/objectives for anything. However, I'd have to agree with AndyN that measures/objectives for document control and auditing are probably pretty useless. Almost no one in the organization would be interested.

Metrics/objectives should be focused on business processes, and only on support processes where their impact on business processes is significant.
I agree. I often use the analogy of the car dashboard when helping an organization's management to agree on what processes and measurements are important to focus on. If you overwhelm top management with too many measurements, they will lose sight of the important things and blame the QMS for being too complex. Well, yes, because we made it so, not because it needs to be.

The whole idea of measurement and monitoring can be totally overblown at the risk of turning people off. Sure you can measure all sorts of things, but what really gives an indication the car is going to get you where you want to be?

To me, measuring document control and internal audits is like filling up the dashboard with a ton of dials and gauges. Knowing the oil temperature of the transmission or the battery charging is all very well, but contributes very little to getting down the road, to your destination, safetly and so on.

So let's keep it relevant to the business and useful to management.
 

Stijloor

Staff member
Super Moderator
#16
Re: Measurables for Document Control and Internal Audits

So let's keep it relevant to the business and useful to management.
How do you determine what's relevant? By what criteria?
How do you determine what's useful? By what criteria?

"If you can not measure it, you can not improve it." Lord Kelvin.
 
Last edited:
P

pldey42

#17
Re: Measurables for Document Control and Internal Audits

The reason I think it's worth measuring the value of internal audits is this: Many internal audit programmes are staffed by "voluntary" auditors who also have "real" work to do. Every year at budget time the quality manager faces the question from top management, "Why do we gotta do this?"

Imagine the answers from two different quality managers in two different organizations, one doesn't measure internal audits and one does ...

Here's Gnome Eshure's answer: "Cos we gotta do it, boss. ISO 9001 says we gotta do internal audits and our auditors, Audit and Stampit, ain't gonna give us our ISO 9001 cert if we don't. So give. Please?"

And here's Elf Orlether's answer: "Last year our internal audits and corrective actions cost us $100,000 annualized and we saved $500,000 annualized as a result. Hands up those who'd like to do Internal Audit again this year?"

Whom do you think management teams will more likely listen to? Gnome Eshure or Elf Orlether?

Pat
 

Stijloor

Staff member
Super Moderator
#18
Re: Measurables for Document Control and Internal Audits

'MR's functions' in my opinions has certain advantages over evaluating the sub processes individually.
Let's say that a "main process" has been reviewed during Management Review (and let's hope that this is conducted on a fairly regular basis instead of the "yearly ISO review event"), and the conclusion is that the "main process" is deemed ineffective.... Now, would you perhaps look at audit results? Let's now assume (based on previous posters' suggestions) that no objectives and no measurements were established for the internal audit process, where would you begin with correcting this situation? Has the main process failed? Has the internal audit process failed?
 
#19
Re: Measurables for Document Control and Internal Audits

The reason I think it's worth measuring the value of internal audits is this: Many internal audit programmes are staffed by "voluntary" auditors who also have "real" work to do. Every year at budget time the quality manager faces the question from top management, "Why do we gotta do this?"

Imagine the answers from two different quality managers in two different organizations, one doesn't measure internal audits and one does ...

Here's Gnome Eshure's answer: "Cos we gotta do it, boss. ISO 9001 says we gotta do internal audits and our auditors, Audit and Stampit, ain't gonna give us our ISO 9001 cert if we don't. So give. Please?"

And here's Elf Orlether's answer: "Last year our internal audits and corrective actions cost us $100,000 annualized and we saved $500,000 annualized as a result. Hands up those who'd like to do Internal Audit again this year?"

Whom do you think management teams will more likely listen to? Gnome Eshure or Elf Orlether?

Pat
Patrick:

How are you?

With respect, IMHO your example is unreal. If an organization is registered, the audits become part of that budget - if it's actually budgeted for. Often there's no money set aside for audits, just for the reason you stated - the people doing them have other jobs, audits get squeezed in, on top of regular work. So I rather doubt if anyone has a true handle on the costs of doing audits.

I agree wholeheartedly that cost savings can be reported, but others here have advocated telling management about pure numbers of nc's, comparisons to external audit finsings etc. which would bore the pants off most top management I know, and they mean nothing.

BTW - my criteria for relevant and useful are what top management's are......and I've never advocated measuring audits and document control. There are bigger fish to fry, IMHO.
 

Stijloor

Staff member
Super Moderator
#20
Re: Measurables for Document Control and Internal Audits

Andy,

You are an auditor for a CB. When you conduct an audit to verify the effectiveness of the internal audit process, what do you want to see? What are you looking for? I look forward to a few examples.

Stijloor.
 
Thread starter Similar threads Forum Replies Date
P Received a minor for not having good measureables/goals. Need help with KPIs. IATF 16949 - Automotive Quality Systems Standard 52
A Measureables to Monitor an Integrated Management System Capability IATF 16949 - Automotive Quality Systems Standard 3
A Corrective Action Tracking - What Measureables and How to Track Status ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
J Lack of Measureables in our Quality Objectives ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
J Quality Objective Measureables - Please Review ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
N Quality Objective - Scrap Measureables ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
J Measureables to determine efficiency and effectiveness of the COP Processes IATF 16949 - Automotive Quality Systems Standard 4
H Ergonomics Metrics (Measureables) - Human factors engineering Lean in Manufacturing and Service Industries 39
D Design and Development Measureables - ISO9001:2000 Clause 7.3 Design and Development of Products and Processes 5
G Statistical Techniques Measureables for Application Development Company Statistical Analysis Tools, Techniques and SPC 3
1 Made it through Phase One - Measureables for Management Review - Need list or form Management Review Meetings and related Processes 9
K Business Plan - Quality Objectives and Measureables - TS 16949 Para 5.4.4 IATF 16949 - Automotive Quality Systems Standard 8
A Internal Audits - Design Process Measurement (Measureables) Design and Development of Products and Processes 6
Icy Mountain Quality Objectives Redux - Setting Actual Numbers (Measureables) IATF 16949 - Automotive Quality Systems Standard 15
L Measureables: Efficiency - Effectiveness of COP (Customer Oriented Process) and KSP ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
7 ISO 14001 Measureables - Plastics related industry Miscellaneous Environmental Standards and EMS Related Discussions 1
Marc Measuring and Improving Quality - Choosing Measureables Quality Tools, Improvement and Analysis 3
J Document Control Metrics Document Control Systems, Procedures, Forms and Templates 3
A Document Review and Document Approval --- 2 Signatures needed acc. §820.40? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
I Document Control Software Document Control Systems, Procedures, Forms and Templates 2
K 25-year lifetime of medical device - document storage period EU Medical Device Regulations 1
J UDI - Where to document it? EU Medical Device Regulations 8
T Controlling Expandable Forms in Paper-Based Document Control System Document Control Systems, Procedures, Forms and Templates 10
S Changing revision status of a reviewed document Manufacturing and Related Processes 4
shimonv Document Control Procedure Header Content Document Control Systems, Procedures, Forms and Templates 3
A Document release vs its related training. Which should come first? ISO 13485:2016 - Medical Device Quality Management Systems 18
Q Document ownerships Document Control Systems, Procedures, Forms and Templates 14
E Document Change Request forms (or no forms)? ISO 13485:2016 - Medical Device Quality Management Systems 9
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
B Document Approval Matrix Benchmarking Document Control Systems, Procedures, Forms and Templates 3
J Document Approval Signature Order Document Control Systems, Procedures, Forms and Templates 10
L How to add exemption or statement to control of document procedure? ISO 13485:2016 - Medical Device Quality Management Systems 5
M What is the proper way to document measurements Measurement Uncertainty (MU) 1
P Equipment 21 CFR 820.70(g) - User Requirements Document for Off the shelf equipment 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
B Do IFU designs have to be document controlled under ISO 13485? Document Control Systems, Procedures, Forms and Templates 2
C Certificate of Conformance Form - COC for each customer a controlled document? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
A Document "Correspondence IATF 16949 vs ISO13485" available? IATF 16949 - Automotive Quality Systems Standard 0
R Document Retention - Discard hard-copies after scanning? ISO 13485:2016 - Medical Device Quality Management Systems 2
P Mylar plot suppliers in accordance with D6-51991 document AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
A Document Change Notice vs complete System re-write Manufacturing and Related Processes 4
Q Document approval through SharePoint (without signature) Records and Data - Quality, Legal and Other Evidence 4
S What is considered a "core algorithm"? (From an FDA guidance document) Medical Information Technology, Medical Software and Health Informatics 4
U Document Approval - Software company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
O Please, help with document on ISO/ TS 17758 and (IDF 87: 1979). Thank you. Manufacturing and Related Processes 1
K Proper document of SMPS used in infant warmer for IEC 60601-1 testing IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
I Nitpicking on document released dates ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
I Document Control on Log Files ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
Q Refusal to Document Complaints Customer Complaints 39
Q Tracking Procedure Revisions (Document Control) Document Control Systems, Procedures, Forms and Templates 9
C Document Control Stamps - Does anyone still stamp their documents? Document Control Systems, Procedures, Forms and Templates 24

Similar threads

Top Bottom