Effective Auditing advice needed

[qualprod]

QualProd - from your post, it seems you are auditing from the "element" of the standard, which isn't the "Preferred" approach. Internal audits shouldn't really be being done on the requirements, unless we're talking about very early in the implementation - is that your situation?

Because you have taken this approach to audit elements, you have put yourself into this conundrum. BTW there is no "ISO approved practice". You should take a look at what and why you are auditing. Unless you have a clear understanding of these things - in other words your audit program as a whole - you will keep encountering such riddles and you won't ever actually audit your QMS, all you'll be able to tell management is that you do/don't comply with ISO 9001 which doesn't tell them anything useful.

I'd strongly suggest you go back to basics and ask yourself the question what is it you're auditing, why, how it should be audited and what benefit does it bring to management as an independent view of the business.

Yes Andy, you are right, I´m in an early implementation and want to be sure everything is covered.
, but came to my mind this question, What I did before in the case I mentioned, I audited all the clauses.

I know that when we have a mature system, audits are focused on some specific aspects and
only some clauses are applicable.

e.g. inspection of product, production, purchasing, etc.

Other opinions are welcomed in this regard

Thanks

 

[tony s]

What I do is I audit each process. In auditing each process I usually start by going through the following:

1. establish first the expected outputs of the process;
2. obtain information from the auditee that the expected outputs are actually produced or achieved ;
3. determine whether there are performance indicators to be measured and monitored relevant to the output of the process, including objectives;
4. check whether there are controls if the expected outputs turn-out to be nonconforming;
5. get information from the auditee about the risks that may lead to producing nonconforming outputs;
6. check whether there are controls to address the risks;
7. and so on...

You will notice that I have already touched the relevant requirements of ISO 9001 by going through 1-6 and, depending on the context of the process being audited, I can also check from other clauses. For example, I will check conformity with:

• 7.1.5 if measuring equipment is critical to the process;
• 7.1.3 or 8.5.1d if infrastructure is critical to the process;
• 8.4 if the process has direct interaction with external providers;
• and so on...

 

[AMIT BALLAL]

Follow a process approach. Try to understand how the process works, what inputs are needed in order to make the process work and what output is expected out of the process. Then who are the suppliers - who provide input to the process and customers - who receives output from the process. See how the process interacts with other processes. Most of the time, I find there is a problem always in the linkage between processes.

If the interaction is improved, ultimately system will be effective and will help business grow. You need to check how business objectives are linked to Quality objectives and process level KPIs. This will help to understand where to look in order to make sure business objectives are achieved.

 

[AndyN]

What I do is I audit each process. In auditing each process I usually start by going through the following:

1. establish first the expected outputs of the process;
2. obtain information from the auditee that the expected outputs are actually produced or achieved ;
3. determine whether there are performance indicators to be measured and monitored relevant to the output of the process, including objectives;
4. check whether there are controls if the expected outputs turn-out to be nonconforming;
5. get information from the auditee about the risks that may lead to producing nonconforming outputs;
6. check whether there are controls to address the risks;
7. and so on...

You will notice that I have already touched the relevant requirements of ISO 9001 by going through 1-6 and, depending on the context of the process being audited, I can also check from other clauses. For example, I will check conformity with:

• 7.1.5 if measuring equipment is critical to the process;
• 7.1.3 or 8.5.1d if infrastructure is critical to the process;
• 8.4 if the process has direct interaction with external providers;
• and so on...

I have put this approach, Tony, into a "visual metaphor" to assist in the planning of the (internal) audit.

 

[John Broomfield]

QualProd,

You’ve misstated Scope and this may be why you are questioning the value of your audits.

Your auditor training should’ve helped you understand that audit scope means the extent of the audit. The extent or scope of an audit may be limited by a product, process, project or department (and occasionally the organization for a system-wide audit)

As the lead auditor you determine the scope of the audit when planning each audit to fulfill its objective. You frame the scope widely enough to fulfill the audit objective.

Within this scope you may then sample evidence of effectiveness. In doing this you and your auditee may find evidence of excellence or ineffectiveness. Note the importance of engaging the auditees within the scope of the audit in understanding the audit objective and in examining the evidence.

So, make sure that each audit has an objective beyond “to determine conformity to ISO 9001”. That is a given and need not be mentioned given that your company’s policy is to conform to the standard. Your auditees need to buy into the reason for each audit too so engage them in your focus on effectiveness.

You seem focused only on conformity but as an internal auditor you are more focused on the effectiveness of the processes you sample. Do the processes have the resources and controls necessary to fulfill their objectives? Is there evidence that the processes are planned with regard to the risks of not fulfilling the process’ objectives (or is the work designed and executed to prevent nonconformity)? What happens when objectives are not met? Is the system enabling continual learning?

With this mindset, no matter what question you ask you will be supported by the standard. Please get your head out of the clauses in the standard and build your confidence that the standard will support every common sense question you ask to determine the effectiveness of your organization (as a system) and its processes.

Of course, you seek the auditee’s agreement of any evidence of ineffectiveness (nonconformity) so they don’t have to complain here at Elsmar Cove!

John

 

[normzone]

I agree with all of the above - Obviously, this requires appropriate resources dedicated to it. Initially this should be a year round ongoing process like painting the Golden Gate Bridge – done properly it pays for itself in process effectiveness improvements. If the organization ever gets to where no further improvements are appropriate, they'll know it.

 

[Tagin]

There are many ways to "analyze" something. It seems to me that 9.1.3e,g tells us that the analyses & evaluations shall be done in a way that will provide useful information about effectiveness and about the needs for improvements.

So, analyze in ways that provide useful answers to these kinds of questions:

• Were the result of our clause 4-8 activities effective?
• Did the results of our clause 4-8 activities indicate a need for improvements?

Notice that 9.1.3 does not say anything about acting on the results of those analyses!!

On the other hand, 9.3.2 is saying that - at the least - those analyses are important enough that Mgmt Review shall include them as inputs to their review.

Finally 10.3 is about the result (output) of Mgmt Review: did Mgmt Review determine that something additional should be done (i.e., needs or opportunities) that will result in continual improvement.

That is:
Monitoring/measurements data from 4-8 activities --> analyze data in a useful way (9.1.3) --> input to Mgmt Review (9.3.2) --> Mgmt Review --> output of Mgmt Review (10.3) --> new/revised continual improvement activities.

So, what would be audited at each stage is quite different, although they are all (intentionally) closely inter-related.