Measureables for Document Control and Internal Audits

J

Jan T

#91
Yes, it is an interesting read......

What you suggest are not related to the measurables of the audit process, IMHO, Jan, simply indications of the state of the QMS, or symptoms. It occurs to me that the 'product' of auditing is often measured, rather than the 'service' which is provided and has been debated here.

You're correct to state that trends should be evaluated, as part of overall audit management and reporting, but I personally don't see that that has anything to do with how effective the audit process is, however.
I agree that other measurables may need to analyzed to fully "audit" the audit process.

We can measure the tangibles such as:

- are the audits are done on time
- are they done by fully trained and independant auditors
- are the findings recorded
- are corrective actions done in a timely manner
- are management made aware of audit findings is this recorded

These will give us objective evidence that the process of auditing is effective.

But I would like more from the audit than the standard cookie cutter info. I would like to use the data from it to obtain more insite into the various process that are being audited to find trends and establish corrective/preventive actions that will help eliminate these trends in the future.

Good Heavens!! I just read my answer back and I think I am on my soap box!
Sorry guys guess I just got a little over zealous. :eek:

Jan
 
Elsmar Forum Sponsor

Sidney Vianna

Post Responsibly
Staff member
Admin
#92
This can be trended yearly and will easily show the effectiveness of the internal audits in helping to pinpoint problems or potential problems.
When it comes to auditing, either internal or external, one must be very careful about the conclusions of data trending, because (and sorry for keep repeating myself) audit results are influenced much more by the people performing the audit, rather than the system, product, process, etc. being audited.

For example, in 2004 during an AAQG meeting, one of the Aerospace CB's, at the request of a large Aerospace OEM, presented a statistical analysis of the AS9100 audits done under the "new process approach". If you remember, process approach audits were even less understood then than they are now. The data showed that the vast majority of the findings were still being written against document/data control and record keeping. Immediately, someone jumped at the conclusion that the aerospace suppliers could not progress beyond basic QC development if they were still failing miserably in such basic areas.

I raised the question if the results could be attributed to the fact that auditors tend to converge to their comfort zone, low hanging fruits, black and white "doc control" issues, instead of delving into much more significant, albeit sensitive areas such as competence and resource management, top management involvement, root cause analysis, etc...

Just because you train an auditor in process based audits, it does not mean that s/he becomes competent nor will perform audits in such a fashion, when unsupervised. So, be careful about trending audit results because auditor variation is, most of the time, many times larger than variation of the subject being audited.
 
#93
Sidney, I agree totally and I'm amazed that, with all the experiences to date, any CB would still be trotting out the same kinds of data.

Jan, many of the things you posted are not true measurements of an effective audit program. Doing audits on time is like airlines who claim "on time departure" - the flight was delayed two hours, but they left when they said they would! Closure of corrective actions is just that, not auditing per se.

Let me help with this example. Last week an audit was being done on a system that was years old, hardly upgraded from the 94 standards. Using newly (that week) trained auditors we found so many issues that management were reeling from the effects. They had been audited by an accredited registrar every 6 months for a week at a time, their corporate auditors visited too and they had Lead Auditor qualified people in plant. They did their audits on time, they closed out corrective actions etc. etc. - but the system was nearly 13 years out of date! The auditors were trained, the management were informed in writing of any nc's, etc., the company was registered for a long while.

But it was the 'newbies' who found basic 'blocking and tackling' wasn't being done, so all the whole thing was a sham.......D'you know why?

Because it was all in the planning of the audit and we used auditors who know the process they are auditing, just that they didn't audit their work. so they were already 'up the ladder' on the audit. Since there were three groups, I had to runbetween them and do not claim any responsibility for the results (with the exception that I'm one of the best trainers there is, they told me so);)
 

Stijloor

Staff member
Super Moderator
#94
Somebody please make this post go away!:frust:
Hello The Man,

As you have noticed, this thread will not go away. :nope::nope:
We Covers are very persistent and passionate about our profession. See other threads....
That's what makes this Forum the BEST!

I look forward to your posts.

Stijloor.
 

Jim Wynne

Staff member
Admin
#96
Hello The Man,

As you have noticed, this thread will not go away. :nope::nope:
We Covers are very persistent and passionate about our profession. See other threads....
That's what makes this Forum the BEST!

I look forward to your posts.

Stijloor.
"The Man" asked for the post to go away, not the thread. There's a current discussion about the Cove TOS as it pertains to people registering under more than one user name; I think that perhaps The Man Wes was just testing.
 

Stijloor

Staff member
Super Moderator
#97
"The Man" asked for the post to go away, not the thread. There's a current discussion about the Cove TOS as it pertains to people registering under more than one user name; I think that perhaps The Man Wes was just testing.
Jim,

Yes, I saw the word "post", and I interpreted "thread", because this thread is quite active for a long while.... Was this a test? Ignorance on my part. :eek: Now I feel like a "guinea pig." :confused:

Stijloor.
 

Jim Wynne

Staff member
Admin
#98
Jim,

Yes, I saw the word "post", and I interpreted "thread", because this thread is quite active for a long while.... Was this a test? Ignorance on my part. :eek: Now I feel like a "guinea pig." :confused:

Stijloor.
Actually, I thought the same thing at first, but I had just come from the TOS thread, saw Wes's picture but not his name, and made a wild guess. On the other hand, maybe Wes is The Man, in which case I'll have to remember to be suitably deferential in the future. :tg:
 
H

HSSE Auditor

#99
Actually, I thought the same thing at first, but I had just come from the TOS thread, saw Wes's picture but not his name, and made a wild guess. On the other hand, maybe Wes is The Man, in which case I'll have to remember to be suitably deferential in the future. :tg:
Do you mean? "The man behind the curtain"? :mg:
 
Thread starter Similar threads Forum Replies Date
P Received a minor for not having good measureables/goals. Need help with KPIs. IATF 16949 - Automotive Quality Systems Standard 52
A Measureables to Monitor an Integrated Management System Capability IATF 16949 - Automotive Quality Systems Standard 3
A Corrective Action Tracking - What Measureables and How to Track Status ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
J Lack of Measureables in our Quality Objectives ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
J Quality Objective Measureables - Please Review ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
N Quality Objective - Scrap Measureables ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
J Measureables to determine efficiency and effectiveness of the COP Processes IATF 16949 - Automotive Quality Systems Standard 4
H Ergonomics Metrics (Measureables) - Human factors engineering Lean in Manufacturing and Service Industries 39
D Design and Development Measureables - ISO9001:2000 Clause 7.3 Design and Development of Products and Processes 5
G Statistical Techniques Measureables for Application Development Company Statistical Analysis Tools, Techniques and SPC 3
1 Made it through Phase One - Measureables for Management Review - Need list or form Management Review Meetings and related Processes 9
K Business Plan - Quality Objectives and Measureables - TS 16949 Para 5.4.4 IATF 16949 - Automotive Quality Systems Standard 8
A Internal Audits - Design Process Measurement (Measureables) Design and Development of Products and Processes 6
Icy Mountain Quality Objectives Redux - Setting Actual Numbers (Measureables) IATF 16949 - Automotive Quality Systems Standard 15
L Measureables: Efficiency - Effectiveness of COP (Customer Oriented Process) and KSP ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
7 ISO 14001 Measureables - Plastics related industry Miscellaneous Environmental Standards and EMS Related Discussions 1
Marc Measuring and Improving Quality - Choosing Measureables Quality Tools, Improvement and Analysis 3
J Document Control Metrics Document Control Systems, Procedures, Forms and Templates 3
A Document Review and Document Approval --- 2 Signatures needed acc. §820.40? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
I Document Control Software Document Control Systems, Procedures, Forms and Templates 2
K 25-year lifetime of medical device - document storage period EU Medical Device Regulations 1
J UDI - Where to document it? EU Medical Device Regulations 8
T Controlling Expandable Forms in Paper-Based Document Control System Document Control Systems, Procedures, Forms and Templates 10
S Changing revision status of a reviewed document Manufacturing and Related Processes 4
shimonv Document Control Procedure Header Content Document Control Systems, Procedures, Forms and Templates 3
A Document release vs its related training. Which should come first? ISO 13485:2016 - Medical Device Quality Management Systems 18
Q Document ownerships Document Control Systems, Procedures, Forms and Templates 14
E Document Change Request forms (or no forms)? ISO 13485:2016 - Medical Device Quality Management Systems 9
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
B Document Approval Matrix Benchmarking Document Control Systems, Procedures, Forms and Templates 3
J Document Approval Signature Order Document Control Systems, Procedures, Forms and Templates 10
L How to add exemption or statement to control of document procedure? ISO 13485:2016 - Medical Device Quality Management Systems 5
M What is the proper way to document measurements Measurement Uncertainty (MU) 1
P Equipment 21 CFR 820.70(g) - User Requirements Document for Off the shelf equipment 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 7
B Do IFU designs have to be document controlled under ISO 13485? Document Control Systems, Procedures, Forms and Templates 2
C Certificate of Conformance Form - COC for each customer a controlled document? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
A Document "Correspondence IATF 16949 vs ISO13485" available? IATF 16949 - Automotive Quality Systems Standard 0
R Document Retention - Discard hard-copies after scanning? ISO 13485:2016 - Medical Device Quality Management Systems 2
P Mylar plot suppliers in accordance with D6-51991 document AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
A Document Change Notice vs complete System re-write Manufacturing and Related Processes 4
Q Document approval through SharePoint (without signature) Records and Data - Quality, Legal and Other Evidence 4
S What is considered a "core algorithm"? (From an FDA guidance document) Medical Information Technology, Medical Software and Health Informatics 4
U Document Approval - Software company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
O Please, help with document on ISO/ TS 17758 and (IDF 87: 1979). Thank you. Manufacturing and Related Processes 1
K Proper document of SMPS used in infant warmer for IEC 60601-1 testing IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
I Nitpicking on document released dates ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
I Document Control on Log Files ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
Q Refusal to Document Complaints Customer Complaints 39
Q Tracking Procedure Revisions (Document Control) Document Control Systems, Procedures, Forms and Templates 9
C Document Control Stamps - Does anyone still stamp their documents? Document Control Systems, Procedures, Forms and Templates 24

Similar threads

Top Bottom