TL-9000 Certifying Body Issue - Auditor failed to find an issue for 10 years

[Sidney Vianna]

In every pre-audit meeting I have ever attended, to absolve themselves of responsibility they always make a point of telling the company they are auditing that they are looking at a sample and are not going to be able to look at everything.

Even though ISO 19011 is no longer a normative reference in the conformity assessment protocols, it advises auditors to confim that aspect of system auditing. Section 6.4.10 a) states:

advising that the audit evidence collected was based on a sample of the information available and is not necessarily fully representative of the overall effectiveness of the auditee's processes;

 

[Coasterguy500]

In a standard ISO audit I would agree. But TL requirements and audits are very different. They are all about the metrics that are submitted and are very confusing to decipher. In TL's own documentation for CB's it states the following.....

9. CB auditors shall review the actual data submissions, verifying proper implementation of the counting rules for required measurements. This check is to review data consistency covering a minimum one-year period except when the organization and/or new product have been certified for less than one year in which case the data shall be reviewed for at least as long as the organization and/or new product has been certified.

The new auditor did this, the ones for the last 10 years obviously did not.

 

[Marc]

I should not have posted. I'm totally ignorant of TL9000.

I'll show myself to the door.

 

[Sidney Vianna]

In a standard ISO audit I would agree. But TL requirements and audits are very different. They are all about the metrics that are submitted and are very confusing to decipher. In TL's own documentation for CB's it states the following.....

* emphasis mine.

Data Submission is NOT the whole scope of a TL-9000 audit; it is a component. Yes, there are requirements for the CB Auditors for the TL-9000 protocol, but the responsibility for timely submission and data integrity rests with the registrants. https://tl9000.org/alerts/documents/DSR_Advisories.pdf

 

[AndyN]

My experience of working with TL/QuEST folks is that the metrics cannot be simply dismissed as not falling in a sample. The CB is culpable from the get go and each visit should have critically reviewed the process for collection and analysis of the metrics portion. If those folks are unfamiliar with TL would care to check it out - there's a whole other handbook on metrics to go with the QMS requirements.

 

[Tyranna]

It would behoove you to have an Metrics validation review monthly to review the data submitted, review the TL alerts if any, and review the outliers in the data. If there are outliers, those can be traced down and examined. It helps to have a second or third set of eyes review the data calculations periodically (confusing things they are!) to see if methods of calculation have changed or if the process of data collection has changed.

As far as relying on a CB auditor to catch errors, I do not hold faith that they are able to catch some or all errors given the limited amount of time they have to examine the calculations, retrieve the data, do comparison analysis, etc. If they do happen to catch an error, great! But I feel that it falls to the company to review methods and calculations and validate the data prior to submission. JMHO.

 

[Tagin]

Wouldn't this issue also indicate a failure of internal audits, in addition to the possible failure by the CB auditors?

From: Q&A - Audits

Question 11581 — What is the 'intent' of TL 9000 concerning internal auditing of the QMS based on TL 9000? Are all clauses of TL 9000 required to be audited (internally) each year? Or can a company focus in certain areas, ensuring that all clauses are audited sometime within a set timeframe (such as three years)?

Answer — There is no specific answer to this question. The organization has to determine the depth, breadth, and frequency of its internal audit program needed to ensure that its quality management system is adequate and meets all requirements on a continuing basis.