Audit Findings - Evidence that a System is Ineffective

#1
Does anyone have some examples they can share, of audit findings, non-conformities, audit reports etc., which give evidence that the situation is (quantitatively or qualititatively) ineffective?

I don't mind if it's an internal audit or external audit, and please be sure to 'clean up' any sensitive data within the reports.

Thanks!
 
Last edited:
Elsmar Forum Sponsor

Randy

Super Moderator
#2
Re: Audit Findings - including effectiveness

Here's 2 excamples. NC #1 led to NC#2. This isn't the actual format we use. I use these as examples during training occasionally





Organization: ABC
Incident Number: XX-1
Area: Maintenance Department
18001:2007 Clause Number: 4.4.6
Category: X MAJOR

Requirement:
ABC procedure XYZ111, Section 3.11 requires that the OEM of equipment being repaired or modified beyond original equipment guidelines be contacted prior to the repair or modification. Section 3.2 requires that all repairs or modification performed on equipment that exceed original equipment guidelines be assessed for Risk

Nonconformity:
•Maintenance records for Taylor Forklift repairs indicated that a modification to OEM assemblies had been made contrary to ABC procedure XYZ111.
•The effectiveness and safety of the repair methods on the Taylor’s could not be verified at time of assessment
•There is no evidence to indicate any communication with Taylor about the repairs made to their equipment
•Risk assessment for modifications to the wheel hubs of the Taylor Forklifts have not been performed




Organization: ABC
Incident Number: XX-2
Area: Maintenance Department
18001:2007 Clause Number: 4.4.2
Category: X MAJOR

Requirement:
Personnel shall be competent to perform tasks that may impact on OH&S in the workplace.


Nonconformity:
There is no evidence available to demonstrate that maintenance personnel are competent to safely perform the level of vehicle maintenance required of them.

Placement of personnel into the Maintenance Department is determined solely by bid and seniority within the company and does not take into consideration any training, education or experience.

During the review of personnel training and competence development for the Maintenance Department it was identified that there is no supporting evidence to show that minimum competencies for personnel in the department has been established, additionally there is no evaluation for competence in place for maintenance department personnel.
 
#3
Thanks, Randy.

Do you (or anyone else) have anything which indicates the performance of a process was affected?

The examples provided tend to be a lack of evidence, which isn't quite the same (IMHO) as having a quantitative or qualititative effect - by way of explanation of what I mean, using one of your examples of competencies, what did having a group of unqualified maintenance people running around actually cause in terms of the process?

Was it not having the required up-time and availability of equipment because the company didn't have sufficient techs to get to the scheduled maintenance? Did they have call backs to the machinery because the person didn't fix it correctly? This is the type of audit finding I'm looking for.
 

Randy

Super Moderator
#4
The group contributed to a loss of nearly $250K of equipment related to accidents/failures caused by improper and unapproved maintenance. The risk level of these failures could have been catastrophic related to loss of life if the failures had occurred while handling the product. (There would have been no remains to recover)

The loss of the equipment (downtime) contributed to loss of production (not within my scope I just had the OHS) and potential loss of customer satisfaction.

The second NC I gave clearly stated that the maintenance techs lacked the competency to perform effective work (this was related to all aspects of the facility because they only became maintenance because of union seniority and not ability). We found evidence of this in other areas as well and that was noted in the QMS side (not in my scope).

Therefore the Competence, Awareness and Training piece (18k, 14K 4.4.2 & 9K 6.6.2) was not effective in the QMS, EHS and OHS of this facility and the results were found to be serious and systemic
 
#5
So, without me seeming to be too controversial, why didn't you mention the $250K of failures etc. actually in the report..........?

The lack of competency could be that the company simply didn't follow the standard or their procedure - which is a compliance issue and (while it will undoubtedly lead to problems) there was nothing more than that.

What I'm looking for are some examples where the auditor (basically) reported that; "Because of 'X' a whole lot of 'Y' was lost/wasted.........."
 
Last edited:

BradM

Staff member
Admin
#6
So, without me seeming to be too controversial, why didn't you mention the $250K of failures etc. actually in the report..........?

The lack of competency could be that the company simply didn't follow the standard or their procedure - which is a compliance issue and (while it will undoubtedly lead to problems) there was nothing more than that.

What I'm looking for are some examples where the auditor (basically) reported that; "Because of 'X' a whole lot of 'Y' was lost/wasted.........."
Interesting question. So, I'm inquisitive to your situation.

Why would an auditor be reporting the above?

1. Would that not be outside the scope of the audit?
2. How would an auditor have sufficient knowledge to assess the cost of a non-compliance?

It seems to be that any such reporting would be speculation on the part of the auditor. Auditors would be about investigating if the organization is in line with the system upper management wants followed. Whether that system is effective would be determined by an entirely different set of measurements.
 

Randy

Super Moderator
#7
So, without me seeming to be too controversial, why didn't you mention the $250K of failures etc. actually in the report..........?
Yes (at that time 18001 was concerned with property as well) our greatest concern was assessment of risk, competence and not following operational controls.

The lack of competency could be that the company sinmpy didn't follow the standard or their procedure - which is a compliance issue and (while it will undoubtedly lead to problems) there was nothing more than that.
They followed their procedure, the procedure didn't meet the requirements of the standard. This was a conformance issue realted to a management system requirement. Andy, there is more out there than 9001


What I'm looking for are some examples where the auditor (basically) reported that; "Because of 'X' a whole lot of 'Y' was lost/wasted.........."
Because of "X" some people could have been killed, equipment permanently destroyed and product really messed up.
 
#8
Randy..........I'm not interested in any specific standard, I'm merely asking why not include in the report that those things were at risk. To me, being in non-compliance with the standard, doesn't mean that the situation is at risk of anything - other than meeting the standard......

I'm at a bit of a loss to understand your comments, since it all seems to be about compliance - which is an external auditor's role, I know.

I'm looking for examples where someone has actually included actual details of cost/waste/performance etc.
 

Wes Bucey

Quite Involved in Discussions
#9
About 30 years ago (1975), My firm was in the midst of a due diligence evaluation of a deal where a manufacturer was seeking to sell and leaseback its entire facility, process equipment as well as real estate. Our client was an investor consortium willing to put up $50,000,000 for the deal, subject to my firm's "fairness opinion" attesting the deal was fair for both investors and the stockholders of the manufacturer. In effect, we were in the position of a third party auditor/registrar, since we were paid a flat fee, not contingent on whether the deal closed or not.

For those unfamiliar with the concept, due diligence absolutely considers factors such as conformance to government regulations (including zoning and life, health, safety items), dollar and cents equivalents of real estate and equipment, projected financial risks as to whether the manufacturer has good prospects of remaining in profitable business at the location, determining the funds will be used for the avowed purpose and not siphoned off to the detriment of the company that will be the tenant, labor relations, and a multitude of other factors.

The process on this deal took a team of 12, including experts in several fields, putting together individual reports which would be assembled into a final omnibus report for the parties to the deal.

One major hangup obsessed one of our investigators - the front and sides of the building were immaculately landscaped and maintained, but a large empty lot behind the employee parking lot in the rear of the building was a very scruffy wasteland with large patches of bare, hard-packed earth, very ugly in comparison with the remainder of the property.

He commented about it to our partner-in-charge and said, "Gee! that ugly lot is an eyesore and detracts from the value of the property as a going business. One of our recommendations should be that they clean up and landscape that lot as part of the deal."

The partner opined that, "Maybe there are some plans to develop that lot." and proceeded to dig further.

The upshot after some digging
(literally - we hired a contractor to do some test bores when we couldn't get a straight answer from executives of the manufacturer)
:
we discovered the entire site was polluted with toxic waste which the manufacturer had tried to hide with new landscaping. The waste in the rear was so toxic, the landscaping efforts were unsuccessful and the plants all died within the six months we were performing the due diligence! The idiots in the manufacturing company were trying to browbeat their landscape contractor into replanting for free instead of spending another $20,000 or $30,000 for a cosmetic cleanup BEFORE our due diligence got to that stage. They were so full of hubris, they thought we'd miss the problem.

We collected our fee and were out of the deal. Our client cited the report as excuse for killing the deal and THEY were out of it as well. We later learned the manufacturer had been illegally dumping waste for years to save the cost of proper disposal. The litigation with governments, investors, workers, neighboring property owners went on for years. Miraculously, none of the guys responsible were ever indicted.

Off hand, I'd say our finding constituted prima facie evidence the SYSTEM was ineffective, since all internal controls which should have detected and stopped the pollution were bypassed. Dozens of people had to have either advance knowledge or "after the fact" knowledge of the dumping, but no mention was ever made in internal audits. Certainly, no corrective action was recorded as being suggested or implemented.
 

Randy

Super Moderator
#10
Here's where common language seperates us....When you say compliance Ithink something mandatory legal or regulatory.....I say conformance when talking about meeting the requirements of a standard.

You also didn't ask for a report, you asked for examples of NC's which I provided.....
 
Thread starter Similar threads Forum Replies Date
Ajit Basrur Track audit findings on Excel tracker Excel .xls Spreadsheet Templates and Tools 9
I What kind of wine best complements the Friday that you close out your external audit findings? Opinions are welcome. Coffee Break and Water Cooler Discussions 12
shimonv Rigid rules for handling supplier audit findings ISO 13485:2016 - Medical Device Quality Management Systems 11
J Any diverging opinions regarding audit findings or conclusions must be resolved no later than the closing meeting ISO 13485:2016 - Medical Device Quality Management Systems 13
XRAY_3121 Informational MDSAP Audit Findings - Document change orders Other Medical Device Regulations World-Wide 12
G Web based tool for benchmarking audit findings Benchmarking 6
Albert G. What are general examples of audit findings with ISO 9001:2015? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 15
A Internal Audit - Findings - Recent Internet Audit (Prior to Certification) IEC 27001 - Information Security Management Systems (ISMS) 7
M Increase in Major TS16949 Audit Findings IATF 16949 - Automotive Quality Systems Standard 14
dubrizo Are you documenting Internal Audit findings as NCRs? Internal Auditing 18
dubrizo Audit Findings: Writing against Internal an ISO Clause Internal Auditing 10
K Definition of Repeat Audit Findings Internal Auditing 11
Q Writing appropriate ISO 13485 Audit Report Findings Internal Auditing 15
M Do all findings (nonconformities) in an internal audit require a corrective action? Quality Management System (QMS) Manuals 55
Mikey324 Questionable Audit Findings after Rules 4th Edition Release IATF 16949 - Automotive Quality Systems Standard 4
R Root Cause for "Simple" External Audit Findings ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
S Internal Audit Findings Summary Rewrite by an Auditee ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
M Help with Internal Audit Findings ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
M Internal Audit Findings and Issuance Problem Internal Auditing 4
E Corrective Actions Root Causes from Surveillance Audit Findings Nonconformance and Corrective Action 3
A Audit Report Findings for an audit conducted yesterday General Auditing Discussions 10
M External Audit Findings - TS 16949 IATF 16949 - Automotive Quality Systems Standard 3
R Root Cause Analysis of Audit Findings - New Quality Manager Quality Manager and Management Related Issues 20
I Types of Internal Audit findings based on ISO 9001 Clause 8.2.2 Quality Manager and Management Related Issues 10
C No findings from External Audit - Should I be worried? General Auditing Discussions 7
K External Auditor asking for other Audit Findings ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 36
6 ISO 9001 Audit Findings Sept 2011 - Critique and Comments ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Management Review Analysis of Internal Audit Findings Quality Manager and Management Related Issues 8
X Audit Findings - The Process/Clause Matrix does not identify all the processes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 30
B Mining Nonconformance Data for Potential Missed Audit Findings Misc. Quality Assurance and Business Systems Related Topics 2
A How to Address Internal Quality Audit Findings Internal Auditing 8
B Critical Action Limits (CAL) for All Internal and External Audit Findings General Auditing Discussions 17
X ISO 9001:2008 Internal Audit - Classification of Audit Findings Internal Auditing 5
AnaMariaVR2 Pros & Cons of FDA viewing QAU Audit Report Findings US Food and Drug Administration (FDA) 7
S External Auditor Findings when an Internal Audit found a Nonconformance Yesterday Document Control Systems, Procedures, Forms and Templates 11
D ISO 10001:2007 - Critical Audit Findings Other ISO and International Standards and European Regulations 6
A ISO 17020 Audit Findings - Calibration from Accredited Laboratory required Other ISO and International Standards and European Regulations 8
L Categorizing Internal Audit Findings Internal Auditing 10
B Writing Techniques for Audit Findings, Reports, etc. Internal Auditing 15
Michael Malis For Internal Audit findings, would you open a CAPA for every observation? Internal Auditing 31
W Audit Findings - Refusing outside audit findings Internal Auditing 14
F Responding to Audit Findings - Procedures not being followed to the letter ISO 13485:2016 - Medical Device Quality Management Systems 28
T TS 16949 Recertification Audit - No negative findings! Covegratulations 23
E Internal Audits - Presenting Audit Findings to Upper Management General Auditing Discussions 19
V Documenting the Root Causes for Internal Audit Non-Conformance Findings Problem Solving, Root Cause Fault and Failure Analysis 5
B AS9100 & Safety Audit Findings - Para 6.4 Work Environment - Fork lift records AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
S Does anyone know if ISO publishes audit findings/results much like the FDA ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 14
C Internal Audit Findings - Do I need to raise NC for amendments/revisions? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
eternal_atlas Review on Internal Audit Findings ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 20
eternal_atlas Internal Audit Findings in C&M works (HVAC, Firefighting, etc.) - Your Comments? General Auditing Discussions 6

Similar threads

Top Bottom