Registrars auditing to the spirit of TS 16949

[vanputten]

Peter Theobald wrote an article titled, "ISO/TS 16949 Changes Are Happening Now." The article first appeared in the October 2005 issue of Quality Digest. The website link is www.qualitydigest.com

I received written permission from Quality Digest to reproduce the article in its entirety, which I have done below.

I would love to hear from TS 16949 registrars. How are you going to audit to a "spirit" of a document. How will nonconformances be written based on a registrars interpretation of the spirit of the standard?

To the users of TS, what do you think about this direction? If true, should we change how internal audits are done?

Thank you, Dirk van Putten

ISO/TS 16949 Changes Are Happening Now
by Peter Theobald

Although the ISO/TS 16949 technical specification isn't scheduled to change until at least 2009, in a very real way it's changing today. Why? Because registrars' understanding and interpretation of the specification has evolved. Lessons learned by registrars, clients and the International Automotive Task Force during the past three years are now filtering down to the way audits are conducted.

Differences to expect from registrars when conducting audits against the requirements of the technical specification are:

• Greater emphasis on the process approach to auditing and supplier adoption
of process-oriented management systems
• Focused auditing on customer expectations and additional specific
requirements
• A higher percentage of time spent on off-shift activities (i.e., shifts
that aren't being performed during the organization's core business hours)
• A more comprehensive stage one readiness review with strictly defined
criteria

The IATF will ensure that this refocusing of auditor resources is accomplished in a structured and consistent manner through the following three initiatives: first, the new IATF requalification training requirements for all technical specification auditors; second, the additional emphasis that the IATF is placing on registrars to continually train and develop their audit staff; and third, the recent introduction and consequent retraining of all ISO/TS 16949 audit personnel on the new stage one readiness review criteria outlined in FAQ 32 of the "Rules for Achieving IATF Recognition, Second Edition," published by the IATF. These initiatives are among the first visible effects of IATF's efforts to refocus and drive home the interpretation of those requirements into process-oriented auditing.

Auditor competence isn't the sole reason behind the challenges that face the certification process for ISO/TS 16949. If, in the true spirit of the technical specification, we attempt to establish a root cause analysis, then auditor competence is merely a symptom. It may be closer to the truth to say that the automotive industry understands the letter of the technical specification while failing (in some cases) to grasp its spirit. However, the registrars, as the "gatekeepers" of ISO/TS 16949 certification, have the responsibility to ensure that only those companies that comply with both the requirements and the meaning of ISO/TS 16949 achieve certification.

Consequently, root cause analysis comes back to auditor competence or, more precisely, the auditor's interpretation of compliance with the spirit of the standard and how that 's consistently applied.

But what's the perceived divergence between the requirements and the spirit of the technical specification? The story starts right at the beginning of the technical specification itself, which requires an organization to determine the sequence and interaction of its processes. Ever since the launch of ISO 9001:2000, companies have been fulfilling this requirement with varying degrees of success. Yet the requirement is a fundamental stepping stone in establishing a process-oriented management system.

Compliance with the requirement for doing this is easy enough. By adopting the process-based quality management system model outlined in clause 0.2 of ISO 9001 with some appropriate process-related labels, compliance with the words of the standard will be achieved. This generic approach satisfies all the requirements of the standard but misses the spirit of what it's trying to accomplish. Properly identifying processes facilitates an organization's ability to structure its management systems around those processes. This assists in clearly identifying the appropriate monitoring methodology and process interactions. Improper or generic identification of an organization's processes will create a ripple throughout the management system, which can be counterproductive to an organization's attempt to establish a process-oriented management system. This potential divergence between compliance with the written word and the adoption of the philosophy or spirit of the technical specification is at the heart of the initiative' s probable success or failure.

This movement to ensure that auditors understand processes, how to audit them, and the benefits to organizations in structuring their management systems around them is a fundamental part of the IATF ISO/TS 16949 requalification training. During this training, it's apparent that both the original equipment manufacturers (OEM) on the IATF board and the IATF oversight officers are placing greater emphasis on auditor competence with regard to the process approach to auditing. To pass the training, all auditors will need to demonstrate a thorough understanding of the process approach and be able to demonstrate that understanding in an audit situation.

But the IATF is not just focusing on process auditing. There is a continual emphasis on ensuring that suppliers are meeting and exceeding customer expectations and applicable customer-specific requirements. Where less than 100-percent compliance is seen, auditors will expect a reaction plan to be in place with any risk to the OEM customer immediately contained. In addition, subsequent root cause analysis must be detailed with corrective action emphasizing error proofing and defect prevention.

The requirement of audit programs to fully encompass all processes and customer requirements, including all shift patterns and off-shift activities, has also been reinforced at the requalification training. Auditors are expected to spend a good portion of their planned audit time covering off-shift activities. This is a direct result of recent OEM feedback, which indicates that a majority of part quality issues originate during off-shift production. Greater emphasis will be placed on establishing whether companies truly understand their own internal sequences and the interaction of processes. The auditor will expect process owners to be identified and processes to be monitored, measured and analyzed to determine their effectiveness. This process-focused approach to auditing will ultimately put organizations under much tighter scrutiny during initial and ongoing audits by their registrar. OEMs can expect more attention to be placed on performance, improvements and measures, as well as customer satisfaction and cost reduction.

The initiatives outlined above are supported by IATF's earlier introduction of FAQ 32, which details the requirements for a readiness review. FAQ 32 states:

"Readiness is judged by two basic criteria:

i. Are the requirements of Rules Second Edition Annex 1 met?

The organization shall provide the following documentation to the audit team for review, and for use in planning the audit (see format in IATF guidance to ISO/TS 16949:2002):

• Description of processes showing the sequence and interactions, including
key indicators and performance trends for the previous 12 months, minimum
• Evidence that all requirements of ISO/TS 16949:2002 are addressed by the
organization's processes
• Quality manual (for each site to be audited)
• Internal audit and management review planning and results from previous 12
months
• List of qualified internal auditors
• List of customer-specific requirements
• Customer satisfaction and complaints status, including customer reports
and scorecards

ii. Does the organization meet the requirements of ISO/IEC DIS 17021 Annex A.2.F.?

The stage one audit shall be performed to evaluate if the internal audits and management review are being planned and performed effectively and that the level of implementation of the management system substantiates that the client organization is ready for the stage two audit.

If the required items from Rules Annex 1 are not present and complete, then the stage one audit shall judge the organization 'not ready.' If during the stage one audit obvious major nonconformances with respect to the organization's effective implementation of the management system are found, the organization shall be judged 'not ready.'

Nonconformities shall not be raised during the stage one readiness review. The stage one audit answers the question: 'Shall the stage two site audit be conducted?' If the answer is negative, then the organization returns to stage one.

The certification body should identify, as part of its stage one audit report, any areas of concern that could be classified as nonconformity during the stage two audit."

Essentially, this FAQ states that a negative recommendation from a stage one readiness review can fall under any of the following scenarios:

• The company hasn't completed its internal audits against the requirements
of ISO/TS 16949 (e.g., system, product or process).
• The company hasn't performed a management review against the requirements
of ISO/TS 16949, included in which were the results of its internal audits.
• If any of the key performance indicators aren't available for review by
the auditor
• If any of the documentation specified in the bullet points in section i.
of FAQ 32 aren't available and/or effectively implemented at the time of the
stage one review
• If any major issues with respect to the organization's effective
implementation of the management system are found, then this should also be
deemed as reason to fail the stage one visit.

Additionally, a mere plan to achieve any of the above is not acceptable. For the stage one audit to be successful, these actions must have been completed prior to the readiness review.

A negative recommendation for a stage one readiness review will require the organization to undergo further stage one reviews until such time as a successful or positive recommendation can be made. This is a far stricter interpretation of the stage one readiness review requirements than previously existed.

The IATF imposed these Draconian requirements upon its contracted registrars, and hence organizations trying to achieve certification, largely for two reasons.

First, witnessed audit findings indicated that ISO/TS 16949 stage one assessments weren't effective in establishing an organization's preparedness for a stage two audit. As a result, a considerable number of stage two assessments were resulting in noncompliances and/or negative recommendations.

Second, those organizations that have failed to embrace a process-oriented management system can be identified at the earliest possible stage and thus not be put forward to a stage two audit.

The hope is that these measures will increase the success rate of stage two assessments and consequently reduce the financial effect of the ISO/TS 16949 certification process to the industry as a whole, while also supporting the overall goal of the IATF in ensuring that only companies with truly process-oriented management systems achieve certification.

Even after successful completion of the IATF requalification exam, auditors will face further oversight and review by the registrars to ensure that the requalification program has been successful. Ongoing reviews will ensure that continued development is proving effective. In short, far greater emphasis is being placed on an auditor's competence and on a registrar's ability to develop and enhance that competence.

ISO 9001:2000 Clause 0.2 states, "An advantage of the process approach is the ongoing control it provides over the linkage between the individual processes within the system of processes, as well as their combination and interaction." This is the underlying driver behind ISO 9001:2000 and, consequently, ISO/TS 16949:2002. In the rush to achieve conformance, it can be easy to forget this founding principle. The IATF never forgot this, and its continual support and enthusiasm for organizations to adopt the process approach should be commended. This latest initiative to inject consistency and diligence into third-party auditing is welcomed by all registrars.

ISO/TS 16949, when properly implemented, can transform a company. It's the responsibility of the registrar to ensure that only companies that have truly grasped the concepts of process-based management systems, and hence the philosophy of ISO/TS 16949, achieve certification.

In conclusion, the technical specification is unlikely to change in the near term. Its success or failure will be judged by supplier performance to the OEMs in terms of defect rates and cost-reduction dollars. It's too early to tell whether or not these will meet expectations. Either way, the final results may well be reliant upon the suppliers' ability to adopt process-oriented management systems and auditors abilities' to "guide" them in the right direction.

About the author

Peter Theobald is an IATF-qualified automotive auditor and the automotive technical manager for NQA-USA. His responsibilities include being one of the designated "veto powers" for NQA-USA ISO/TS 16949 certification activities. He has spent the last five years in the certification industry, focusing on the technical interpretation of standards and specifications and their subsequent implementation by registrars.

[Marc]

Thanks for the link and the article, Dirk! We appreciate it!

BTW - I moved this thread to 'The Reading Room' forum because of its nature.

[Craig H.]

Thanks, Dirk! Anyone care to comment?

[Jim Wynne]

Some random thoughts on what Theobald has written:

Originally Posted by Peter Theobald

Auditor competence isn't the sole reason behind the challenges that face the certification process for ISO/TS 16949. If, in the true spirit of the technical specification, we attempt to establish a root cause analysis, then auditor competence is merely a symptom.

So auditor competence isn't the actual root cause, but wait:

Originally Posted by Peter Theobald

Consequently, root cause analysis comes back to auditor competence ...

Theobald seems to be having trouble making up his mind.

Originally Posted by Peter Theobald

It may be closer to the truth to say that the automotive industry understands the letter of the technical specification while failing (in some cases) to grasp its spirit. However, the registrars, as the "gatekeepers" of ISO/TS 16949 certification, have the responsibility to ensure that only those companies that comply with both the requirements and the meaning of ISO/TS 16949 achieve certification.

Theobald seems to be saying that the automotive industry, to whom the standard belongs, doesn't understand what it wants, and that complying with "the requirements" isn't enough. Isn't it a fault of the standard if it's possible to comply with the standard but not what was intended by the standard? If basic auditing comptetence is a root cause of compliance issues, what makes Theobald (and the IATF) think that training auditors to look for subjective issues is going to help?

Originally Posted by Peter Theobald

Compliance with the requirement for [establishing a process-oriented management system] is easy enough.

Read: Brain surgery is simple, once you learn how to do it.

Originally Posted by Peter Theobald

By adopting the process-based quality management system model outlined in clause 0.2 of ISO 9001 with some appropriate process-related labels, compliance with the words of the standard will be achieved. This generic approach satisfies all the requirements of the standard but misses the spirit of what it's trying to accomplish. Properly identifying processes facilitates an organization's ability to structure its management systems around those processes. This assists in clearly identifying the appropriate monitoring methodology and process interactions. Improper or generic identification of an organization's processes will create a ripple throughout the management system, which can be counterproductive to an organization's attempt to establish a process-oriented management system. This potential divergence between compliance with the written word and the adoption of the philosophy or spirit of the technical specification is at the heart of the initiative' s probable success or failure.

This dense paragraph says, "It's possible to build a cr*appy process-based system and still satisfy the standard." Then there's a problem with the standard, no? Once again I ask, if the contention is that the current crop of auditors isn't basally competent in auditing to the letter of the standard (and I don't necessarily agree with the contention), how can we expect them to deal with subjective requirements and not make a mess of things?

Originally Posted by Peter Theobald

This movement to ensure that auditors understand processes, how to audit them, and the benefits to organizations in structuring their management systems around them is a fundamental part of the IATF ISO/TS 16949 requalification training. During this training, it's apparent that both the original equipment manufacturers (OEM) on the IATF board and the IATF oversight officers are placing greater emphasis on auditor competence with regard to the process approach to auditing. To pass the training, all auditors will need to demonstrate a thorough understanding of the process approach and be able to demonstrate that understanding in an audit situation.

This is an ambitious and commendable goal, but there's a serious question as to whether the population of auditors and auditor candidates has the experience necessary to be effective in auditing to subjective requirements. In the end, if there aren't enough competent and experienced auditors, the standards will be relaxed.

Originally Posted by Peter Theobald

But the IATF is not just focusing on process auditing. There is a continual emphasis on ensuring that suppliers are meeting and exceeding customer expectations and applicable customer-specific requirements.

"Meeting and exceeding"? Surely he didn't mean that. And what's the difference between "customer expectations" and "applicable customer-specific requirements"?

Originally Posted by Peter Theobald

Where less than 100-percent compliance is seen, auditors will expect a reaction plan to be in place with any risk to the OEM customer immediately contained. In addition, subsequent root cause analysis must be detailed with corrective action emphasizing error proofing and defect prevention.

How is this different from current expectations?

Originally Posted by Peter Theobald

The requirement of audit programs to fully encompass all processes and customer requirements, including all shift patterns and off-shift activities, has also been reinforced at the requalification training. Auditors are expected to spend a good portion of their planned audit time covering off-shift activities. This is a direct result of recent OEM feedback, which indicates that a majority of part quality issues originate during off-shift production.(Emphasis added)

This is a hugely significant statement. Does anyone think that this is a new issue? It appears that with all of the history of QS 9000 and 16949 registrations, the source of the majority of quality problems for OEMs has never been adequately addressed. Think about it: by getting control of off-shift operations, the majority of quality problems can be significantly affected. Now think about this: the OEMs need stricter auditing standards in order to get a handle on the problem. We are doomed.

I could go on, but there's no need. The entire thrust of this article and the reasoning behind it is that the standard in itself is somehow inadequate, so registrars' auditors need to spend more time evaluating subjective criteria and off-shift activities. If, as Theobald suggests, we should apply the spirit of 16949 in an evaluation of the situation, I have one question: if suppliers don't understand by now (on a large scale) what the spirit of the standard entails, and don't realize (if the OEMs are correct) that the majority of quality problems originate on off-shifts, how is a change in auditing focus going to help?

[D.Scott]

What a shame. 16949 is about to go down the same drain as QS-9000. The IATF was given the job as shepherd and is about to invite the wolves in to guard the flock. The idea of standards and quality systems is great. Everybody seems to agree they benefit all involved. The thing that killed the credibility of QS-9000 was the bureaucracy created by the registration and surveillance of that system. The new standards appeared to diminish the importance of the role of the 3rd party auditor and I could see, although a long way off, the elimination of the need for surveillance audits. Now the IATF has sold out to the registrars. This "spirit" auditing opens the black hole of uncertainty where the registrar lurks. Giving the auditor a moving target of "spirit" allows for subjective nonconformities to be written by the very group that benefits financially from a nonconformance.

I understand mine is a lone voice. I understand whoever owns the ball makes the rules. I also understand that our quality industry has cancer. That cancer is the current registration system, surveillance audits and micro management. Until the cancer is treated, we are going to just keep getting worse and worse. We don't need oversight by the IATF. We don't need the bureaucracy of the registrars and auditors. What we need are good quality systems backed by partnerships with our customers and suppliers. If the OEMs want better prices, more cooperation and a solid supply chain, they need to recognize the cancer they introduced into the system. Clean out the cancer, get rid of the bureaucracy. Once the patient is cured, stop trying to micro manage from the crumbling "ivory tower". Develop a trust and partnership with your supply chain but don't ask someone else to maintain it; do it yourself.

If the industry keeps changing doctors, changing medicine, changing treatments and never addresses the cancer of the registration system, the patient will, without question, die.

It is time to stop the nonsense and look at the root cause. Spirit has nothing to do with the problems that exist today. Once again a change is being hailed as "the cure". If I were responsible for seeing this system got fixed and saw all the nonsense that has been done, I would fire the lot of them. Basic quality principles tell us to eliminate the root cause. When the IATF comes up with "audit to the spirit", I say they have no idea of what the root cause is or how to fix it. You want to fix the supply chain quality issues? Pay attention to a quote from the past, "Follow the money trail".

Dave

[Helmut Jilling]

This generic approach satisfies all the requirements of the standard but misses the spirit of what it's trying to accomplish. (quote by Peter Theobald).


I read the article, and I think the dissing it is getting is misdirected. The article doesn't say auditors will audit the "spirit" of the standard. It says auditors and companies need to understand the spirit of the standard, so they can implement better systems and audit them more intelligently.

There are far more onerous issues discussed in the article, in my opinion. We're going to end up working all hours of the day and night. They can't determine what the root causes are, so they're focusing on anything anyone suggests. I think these and others are potentially far more troubling than the spirit discussion.

Deming was right. 85% of the failures in a system are caused by management. So, I would argue, 85% of the audit ought to focus on management. Of course, they don't think that is important.

I have to remind myself, it was these same folks who fired Deming in the 1950's and sent him packing to Japan to look for work...

[Howard Atkins]

Originally Posted by hjilling

Deming was right. 85% of the failures in a system are caused by management.
So, I would argue, 85% of the audit ought to focus on management. Of course, they don't think that is important.

I have to remind myself, it was these same folks who fired Deming in the 1950's
and sent him packing to Japan to look for work...

This is the crux of the matter.

I meet management after management that believe they have a "robust" system and they are in a great state
because the pass survellience audits with "flying colours" but when you examine the system there are large gaps
in terms of the implementation and understanding of the QMS that they do not want to address. WE just need
to pass the audit do not waste time on this.
This is in fact the most dangerous aspect the manner in which managers have learnt to appear to be
interested in the QMS but in fact have no real interest

[Baldrick]

I think there are a lot of very valid points already in this thread. I think that the criticism from JSW05 and D. Scott was centred around the fact that the article seems to be trying to deflect any blame for current problems away from what JSW05 and D. Scott believe to be the real causes.

In my opinion the biggest contributing factor is that from the certification industry's point of view, there seems to be a driving philosophy that CLARITY = BAD, AMBIGUITY = GOOD.

ISO9001 and TS could easily be revised to make them clear and unambiguous. Just think of the absolute mess that is the current Supplier Development clause. It seems to say that all of your suppliers have to be ISO approved and be developed to be compliant with TS. Yet speak to 10 registrars and you’ll get 10 different interpretations of what “development” entails, which suppliers the clause applies to, what “compliance with this technical standard” actually means in practice, etc etc.

But turkeys don’t vote for Christmas. Just think what would happen if we had a standard which clearly stated exactly what was required:

- Decreased frequency of surveillance audits
- Reduced need to revise the standard
- Reduced income for the people who publish the standards
- Less consultancy
- Less consultants

I see the standards and certification industry now as being no different to those others that we have criticised and ridiculed for years, e.g. those who produce certain software packages that are ridden with bugs, who are aware of these bugs, but choose not to fix them straight away because producing a bug-free product would cost them future sales. Instead they roll out these “improvements” over several versions.



Off topic example - my boss once logged a complaint about a major desktop suite with the company’s help desk. The problem was that when you pasted graphs from the spreadsheet into the slide presentation package, some of the lines joining the data points disappeared. Their response? “We are aware of this problem but have no plans to implement a fix in versions X, Y or Z. As a solution we suggest that the user manually joins the data points with a pen.”

I kid you not folks.