Does anyone have some examples they can share, of audit findings, non-conformities, audit reports etc., which give evidence that the situation is (quantitatively or qualititatively) ineffective?

I don't mind if it's an internal audit or external audit, and please be sure to 'clean up' any sensitive data within the reports.

Thanks!

Here's 2 excamples. NC #1 led to NC#2. This isn't the actual format we use. I use these as examples during training occasionally





Organization: ABC
Incident Number: XX-1
Area: Maintenance Department
18001:2007 Clause Number: 4.4.6
Category: X MAJOR

Requirement:
ABC procedure XYZ111, Section 3.11 requires that the OEM of equipment being repaired or modified beyond original equipment guidelines be contacted prior to the repair or modification. Section 3.2 requires that all repairs or modification performed on equipment that exceed original equipment guidelines be assessed for Risk

Nonconformity:
•Maintenance records for Taylor Forklift repairs indicated that a modification to OEM assemblies had been made contrary to ABC procedure XYZ111.
•The effectiveness and safety of the repair methods on the Taylor’s could not be verified at time of assessment
•There is no evidence to indicate any communication with Taylor about the repairs made to their equipment
•Risk assessment for modifications to the wheel hubs of the Taylor Forklifts have not been performed




Organization: ABC
Incident Number: XX-2
Area: Maintenance Department
18001:2007 Clause Number: 4.4.2
Category: X MAJOR

Requirement:
Personnel shall be competent to perform tasks that may impact on OH&S in the workplace.


Nonconformity:
There is no evidence available to demonstrate that maintenance personnel are competent to safely perform the level of vehicle maintenance required of them.

Placement of personnel into the Maintenance Department is determined solely by bid and seniority within the company and does not take into consideration any training, education or experience.

During the review of personnel training and competence development for the Maintenance Department it was identified that there is no supporting evidence to show that minimum competencies for personnel in the department has been established, additionally there is no evaluation for competence in place for maintenance department personnel.

Thanks, Randy.

Do you (or anyone else) have anything which indicates the performance of a process was affected?

The examples provided tend to be a lack of evidence, which isn't quite the same (IMHO) as having a quantitative or qualititative effect - by way of explanation of what I mean, using one of your examples of competencies, what did having a group of unqualified maintenance people running around actually cause in terms of the process?

Was it not having the required up-time and availability of equipment because the company didn't have sufficient techs to get to the scheduled maintenance? Did they have call backs to the machinery because the person didn't fix it correctly? This is the type of audit finding I'm looking for.

The group contributed to a loss of nearly $250K of equipment related to accidents/failures caused by improper and unapproved maintenance. The risk level of these failures could have been catastrophic related to loss of life if the failures had occurred while handling the product. (There would have been no remains to recover)

The loss of the equipment (downtime) contributed to loss of production (not within my scope I just had the OHS) and potential loss of customer satisfaction.

The second NC I gave clearly stated that the maintenance techs lacked the competency to perform effective work (this was related to all aspects of the facility because they only became maintenance because of union seniority and not ability). We found evidence of this in other areas as well and that was noted in the QMS side (not in my scope).

Therefore the Competence, Awareness and Training piece (18k, 14K 4.4.2 & 9K 6.6.2) was not effective in the QMS, EHS and OHS of this facility and the results were found to be serious and systemic

So, without me seeming to be too controversial, why didn't you mention the $250K of failures etc. actually in the report..........?

The lack of competency could be that the company simply didn't follow the standard or their procedure - which is a compliance issue and (while it will undoubtedly lead to problems) there was nothing more than that.

What I'm looking for are some examples where the auditor (basically) reported that; "Because of 'X' a whole lot of 'Y' was lost/wasted.........."

So, without me seeming to be too controversial, why didn't you mention the $250K of failures etc. actually in the report..........?

The lack of competency could be that the company simply didn't follow the standard or their procedure - which is a compliance issue and (while it will undoubtedly lead to problems) there was nothing more than that.

What I'm looking for are some examples where the auditor (basically) reported that; "Because of 'X' a whole lot of 'Y' was lost/wasted.........."

Interesting question. So, I'm inquisitive to your situation.

Why would an auditor be reporting the above?

1. Would that not be outside the scope of the audit?
2. How would an auditor have sufficient knowledge to assess the cost of a non-compliance?

It seems to be that any such reporting would be speculation on the part of the auditor. Auditors would be about investigating if the organization is in line with the system upper management wants followed. Whether that system is effective would be determined by an entirely different set of measurements.

So, without me seeming to be too controversial, why didn't you mention the $250K of failures etc. actually in the report..........? Yes (at that time 18001 was concerned with property as well) our greatest concern was assessment of risk, competence and not following operational controls.

The lack of competency could be that the company sinmpy didn't follow the standard or their procedure - which is a compliance issue and (while it will undoubtedly lead to problems) there was nothing more than that. They followed their procedure, the procedure didn't meet the requirements of the standard. This was a conformance issue realted to a management system requirement. Andy, there is more out there than 9001


What I'm looking for are some examples where the auditor (basically) reported that; "Because of 'X' a whole lot of 'Y' was lost/wasted.........." Because of "X" some people could have been killed, equipment permanently destroyed and product really messed up.

Randy..........I'm not interested in any specific standard, I'm merely asking why not include in the report that those things were at risk. To me, being in non-compliance with the standard, doesn't mean that the situation is at risk of anything - other than meeting the standard......

I'm at a bit of a loss to understand your comments, since it all seems to be about compliance - which is an external auditor's role, I know.

I'm looking for examples where someone has actually included actual details of cost/waste/performance etc.

About 30 years ago (1975), My firm was in the midst of a due diligence evaluation of a deal where a manufacturer was seeking to sell and leaseback its entire facility, process equipment as well as real estate. Our client was an investor consortium willing to put up $50,000,000 for the deal, subject to my firm's "fairness opinion" attesting the deal was fair for both investors and the stockholders of the manufacturer. In effect, we were in the position of a third party auditor/registrar, since we were paid a flat fee, not contingent on whether the deal closed or not.

For those unfamiliar with the concept, due diligence absolutely considers factors such as conformance to government regulations (including zoning and life, health, safety items), dollar and cents equivalents of real estate and equipment, projected financial risks as to whether the manufacturer has good prospects of remaining in profitable business at the location, determining the funds will be used for the avowed purpose and not siphoned off to the detriment of the company that will be the tenant, labor relations, and a multitude of other factors.

The process on this deal took a team of 12, including experts in several fields, putting together individual reports which would be assembled into a final omnibus report for the parties to the deal.

One major hangup obsessed one of our investigators - the front and sides of the building were immaculately landscaped and maintained, but a large empty lot behind the employee parking lot in the rear of the building was a very scruffy wasteland with large patches of bare, hard-packed earth, very ugly in comparison with the remainder of the property.

He commented about it to our partner-in-charge and said, "Gee! that ugly lot is an eyesore and detracts from the value of the property as a going business. One of our recommendations should be that they clean up and landscape that lot as part of the deal."

The partner opined that, "Maybe there are some plans to develop that lot." and proceeded to dig further.

The upshot after some digging
(literally - we hired a contractor to do some test bores when we couldn't get a straight answer from executives of the manufacturer):
we discovered the entire site was polluted with toxic waste which the manufacturer had tried to hide with new landscaping. The waste in the rear was so toxic, the landscaping efforts were unsuccessful and the plants all died within the six months we were performing the due diligence! The idiots in the manufacturing company were trying to browbeat their landscape contractor into replanting for free instead of spending another $20,000 or $30,000 for a cosmetic cleanup BEFORE our due diligence got to that stage. They were so full of hubris, they thought we'd miss the problem.

We collected our fee and were out of the deal. Our client cited the report as excuse for killing the deal and THEY were out of it as well. We later learned the manufacturer had been illegally dumping waste for years to save the cost of proper disposal. The litigation with governments, investors, workers, neighboring property owners went on for years. Miraculously, none of the guys responsible were ever indicted.

Off hand, I'd say our finding constituted prima facie evidence the SYSTEM was ineffective, since all internal controls which should have detected and stopped the pollution were bypassed. Dozens of people had to have either advance knowledge or "after the fact" knowledge of the dumping, but no mention was ever made in internal audits. Certainly, no corrective action was recorded as being suggested or implemented.

Here's where common language seperates us....When you say compliance Ithink something mandatory legal or regulatory.....I say conformance when talking about meeting the requirements of a standard.

You also didn't ask for a report, you asked for examples of NC's which I provided.....

Hi,

The audit findings and conclusions are based on the type of audit being carried out; one does not expect a performance audit when one is carrying out a management system audit. The management system audit is indeed about conformance to the system and non-conformance indicates system ineffectiveness. If you are looking for costs, performance, risks etc., then one has to carry out RISK audits or due diligence audits, supplier sustainability audits or specific audit on the subject (like energy audit) as determined by the objective, scope and criteria for audit agreed between the client and the auditor.

Given below is an example of the observation, an extremely rare case, in an EICC audit carried out by me in the recent past:

"The factory does not have a consent to operate under the Water (P & CP) Act and Air (P &CP) Act; it does not have an authorization to collect, handle, store and dispose of waste oil under the Haz. Waste (M&H) Rules; it has not been paying water cess as per the Water (P&CP) Cess Act from its inception. It appears that the company will not be able to get the required Consents from the State Pollution Control Board due the government notification changing the nature of land use in the area where the factory is located; it appears that the company has acquired land for relocating the factory and there is a commitment from the company that all the required consents will be obtained before commencing operations at the new location within a period of one and half years" (Classification: RED, indicating that the client has to consider this as an important issue for immediate action by the supplier)

Will this be an example of what you were looking for ?



With best regards,

Ramakrishnan

Hi,

.......one does not expect a performance audit when one is carrying out a management system audit. The management system audit is indeed about conformance to the system and non-conformance indicates system ineffectiveness.

I don't agree with your comments, Dr. This is at the root of the IATF push for automotive CB auditors to look at the supplier's performance. Also, shouldn't internal qms auditors be telling management if the company system is being followed (compliant) and the results show it's meeting their stated objectives (or not) - effectiveness?

Agreed that it is dependent on the expectations of the audiy client, but why shouldn't a management system audit be about performance?

Randy, my OP did ask for reports. Isn't a non-conformance/compliance statement a report? I don't believe that our languages are that different - unless you're thinking Arkansasonian.........:lmao:

Interesting question. So, I'm inquisitive to your situation.

Why would an auditor be reporting the above?

1. Would that not be outside the scope of the audit?
2. How would an auditor have sufficient knowledge to assess the cost of a non-compliance?

It seems to be that any such reporting would be speculation on the part of the auditor. Auditors would be about investigating if the organization is in line with the system upper management wants followed. Whether that system is effective would be determined by an entirely different set of measurements.

Brad, simply put, it's such things which indicate the effectiveness of the qms. Telling management "we did/didn't follow the defined process/procedure", is only one part of the story. The results of following (or not) the process must also be included to be meaningful and to get the action it deserves to remedy the situation. For example, here's one I saw written:-

The company is currently wasting $8M a year on processing non-conforming product reports. The analysis of rejects from all manufacturing departments shows that over 8,000 reject reports are issued relating to product tolerances which are subsequently waived as 'use as is'. Procedure XYZ is followed for the control of non-conforming product.

Needless to say, this got the president's attention.

.... Miraculously, none of the guys responsible were ever indicted.....

I would substitute "sadly, but not surprisingly" for miraculously.

The bad guys almost never suffer...and the bigger and badder they are (or the more famous) the more this is true.

But oh baby, just be late paying your taxes if you are middle class or lower....