Annual Internal Audit Plan to Audit Everything?

[kalehner]

I am starting this thread in hopes that the enormous body of knowledge housed at the Cove can help me better understand a finding that was made during a recent TS 16949 re-registration audit. I perform internal audits at the organization and the finding was to 8.2.2 Internal Audits. I intend to share the exact NC and supporting evidence in this thread shortly but want to set the stage first so please be patient.

First Several Questions…

How should we interpret the requirement in 8.2.2.4 that all quality management related processes, activities be scheduled on an “annual audit plan”? What is the meaning of the word all in this context? We have potentially thousands of items (quality related processes and activities) that could be audited annually. Do we need to audit all of these annually? What happened to the sampling approach to auditing?

8.2.2 says audit programs shall be planned considering the status and importance of the processes and areas to be audited. Why is 8.2.2.4 not considered to contradict 8.2.2 which implies latitude in deciding what gets audited when. If you have to audit everything annually why bother to prioritize based on “status and importance”.

What happens when an internal auditor is auditing a manufacturing process and following an audit trail that leads to the purchasing department but the purchasing department “process” is not scheduled to be audited for several months. Do you stop auditing so you can stick to the annual audit plan ”schedule”.

What happens when you are performing an audit, let say a product audit, and you discover evidence that the requirements of 8.3 are being met but this element is not “planned” to be audited until a later audit. Do you ignore this evidence? Can you use that evidence to avoid having to do a special audit of 8.3 at a later date?

 

[Duke Okes]

Regarding auditing "all:" - I normally consider this to mean that all elements of the standard, all departments, and all identified processes will be audited. Never had a problem with it.

Regarding "importance of processes:" - You can audit items more often than annually if they are more critical.

Regarding "stopping at purchasing:" - It depends on what your audit policies, procedures, etc. say. Normally using the process approach just about anything can be audited at any time, since all elements of the system (processes) are interconnected in some way.

Regarding "covered 8.3:" - Ditto.

For the latter it might also depend on how in depth the audit is done in these areas while there. For example, you could look at the purchase order for a specific order you've been tracing back, but not necessarily have to look at how the supplier was selected, evaluated, etc. Same for 8.3: Your evidence that it is being met might be related to only a single product/process line you're auditing, and a subsequent audit might cover additional requirements of 8.3 in more depth and more areas.

And if you'd just give us the finding you're concerned about we can more accurately address it.

 

[AndyN]

The idea of an annual plan - covering 'ALL' is, frankly bizarre and doesn't - as you correctly say - address the 'status and importance aspects. Perhaps you'd like to read an article aligned with your thinking. http://www.nqa-usa.com/resources/articles_detail.php?id=48

In the TS world, I've offered up a calendar of events but deliberately left out much of the detail, filling in only those fixed audits which I knew were predictable, like CB audits, corporate 'seagull' audits and so on. Otherwise, the procedure stated that we'd identify the scope and criteria in the 30 days prior to an audit, based on criteria such as process performance, customer issues, changes etc. The Cb guys didn't have a problem that we didn't have a crystal ball...

 

[Jen Kirley]

There are some related discussions about this linked at the bottom of the page. It is a fairly common question.

I do not audit everything each year. I do audit all shifts each year. I audit all processes at least once in a three year cycle (three years to match the three year registration cycle)

Critical processes get looked at each year. Support processes get audited every two or three years depending on what they are and if they have been having issues.

We have a long range plan that shows the three-year pattern, with which we make the annual schedule the standard is talking about. Knowing that some processes work very closely together, I will try to group them in the same month; therefore it's sometimes possible to audit two processes at the same time or in a very near time frame. This can be very helpful when issues get uncovered and you want to explore them before the trail gets cold, so to speak.

My process allows me to reschedule audits within two calendar quarters. This way I can do one early, or later if the process documents are in the middle of a rewrite.

This plan has passed third party audits many times.

 

[kalehner]

Thanks Andy. Your article is inspiring and should be required reading for all auditors especially those assessing to TS 16949. The only thing I would add is “status” might include the concept of process “dormancy”. Some process may temporarily not be operating and consequently not able to be audited.

My takeaway from your article is that there needs to be a fundamental shift in the way the TS 16949 audit community views internal audits or we (the automotive industry) risk investing significant, unnecessary effort on none value added internal audits to satisfy the TS 16949 audit community’s ridiculous interpretations of 8.2.2.

Bravo Andy. I look forward to more of your thoughts on internal auditing. I agree that internal auditing is as important to organizational performance improvement as is the fashionable Lean, Six Sigma processes now in vogue. Without good audits how can an organization know that the gains are sustained. Audits ensure that the C in DMAIC is working.

The Finding

Statement of Requirement: 8.2.2 – An Audit program shall be planned, taking into consideration the status and importance of the processes.

Statement of Nonconformity: The Internal Audit process is not using the process approach.

Objective Evidence: The internal audit schedule is a listing of the clauses of the standard. The internal audit records do not show that internal audits are tracking process performance to metrics and meeting specified goals of the process.

The auditor was correct about the audit schedule being a list of the clauses of the standard, when we expected to audit them, and where. This “audit plan” was created to satisfy the requirement to create an annual audit plan to audit all requirements.

The auditor was provided evidence of process auditing including audit findings that could not have been made if a process approach was not being used. Here is an example with names changed to protect the innocent:

AB00123 - The ____________ operator on line _____ did not record the lube specific gravity at the beginning of the shift and there was no evidence that the specific gravity had been measured as required.

My questions are:

1. How do we create an annual audit plan that satisfies the auditors need for evidence of using the “process approach”. Can we simply identify which process we will be auditing and when, without identifying the specific TS clauses we intend to seek evidence for during the process audit. How do we prove we are auditing everything annually using a schedule that does not include all requirements (clauses)? Do you agree with the other contributors to this thread that it is not necessary to audit everything annually?

2. What do you think is the intent of the following statement in the evidence section of the finding --- “The internal audit records do not show that internal audits are tracking process performance to metrics and meeting specified goals of the process” ---

Is it the purpose of the audit to determine if the processes are meeting goals? Is not that something that is reported during management review? Why should auditors be responsible for determining if a process is meeting its goal? It seems that auditors should be looking for evidence that helps the organization understand why a goal may not be met rather that if it is or is not being met.

 

[kalehner]

Thanks for your help Jennifer.

What criteria do you use to define which process are “critical” and which are “supporting”? Does TS 16949 discuss the difference between critical and supporting processes?

 

[AndyN]

Thanks for your help Jennifer.

What criteria do you use to define which process are “critical” and which are “supporting”? Does TS 16949 discuss the difference between critical and supporting processes?

In the earlier days of rolling out the process approach in the auto world, the concepts of COPS, MOPs and SOPs was used, but it didn't talk to criticality. I suppose you might say (if you were the OEM) that supplier processes which affected your operation were critical - but then what's critical within your walls might not be critical to the customer...I've relied simply on 'status and importance' to define this - since it's just another way of saying 'critical'...

BTW - Many thanks for the comments. It's much appreciated!

 

[AndyN]

My questions are:

1. How do we create an annual audit plan that satisfies the auditors need for evidence of using the “process approach”. Can we simply identify which process we will be auditing and when, without identifying the specific TS clauses we intend to seek evidence for during the process audit. How do we prove we are auditing everything annually using a schedule that does not include all requirements (clauses)? Do you agree with the other contributors to this thread that it is not necessary to audit everything annually?

The process approach doesn't come from the plan, per se, it comes from the implementation of the auditing process, I believe. I've developed the audit football as a planning tool to show the process approach was used. Check here: http://www.nqa-usa.com/resources/articles_detail.php?id=39

2. What do you think is the intent of the following statement in the evidence section of the finding --- “The internal audit records do not show that internal audits are tracking process performance to metrics and meeting specified goals of the process” ---

I suspect the audit records should contain evidence that the audit researched and verified the process performance goals/targets etc and then confirmed that, at the end of the process, the records showed that the goals/targets were being met


Is it the purpose of the audit to determine if the processes are meeting goals? Is not that something that is reported during management review? Why should auditors be responsible for determining if a process is meeting its goal? It seems that auditors should be looking for evidence that helps the organization understand why a goal may not be met rather that if it is or is not being met.

See my answer above. Internal audits are there validate that the process was compliant to the management system controls and, hence, the results can be 'believed'. This forms the basis for leveraging improvement, or correction if not compliant or not meeting goals.

 

[Jen Kirley]

Thanks for your help Jennifer.

What criteria do you use to define which process are “critical” and which are “supporting”? Does TS 16949 discuss the difference between critical and supporting processes?

Critical processes are the manufacturing processes. Their support processes are things like inspection, purchasing, AVL, process change control. Least of all are document control, record retention, management review.

That's far from a complete list, and there is no reason to copy what we do - especially if your program is "young" (ours has been registered a long time) or you are not feeling comfortable skipping a year for any reason.

 

[ralphsulser]

One other suggestion may be to include follow up audits to any customer issues as critical audits. Our CB auditors became much more customer focused as audit cycles evolved into improvements to our systems. I know automotive is really keying in on CA and RCCA sustainment verifications so sometimes the audit systems schedules needs to be revised to reflect issues which arise beyond the routine audit schedules. Also I scheduled audits for some of the core processes 2 or 3 times a years due to historical trends.