Internal Audits - Necessary before Stage 1 Audit?

[phloQS]

Wow I think I stired up a hornets' nest,

Since the system hasn't been audited, how can you predict that only 2 NCs will result? Also, are you offering opinion about the number of CBs who do just 'based on papers' or is this fact based observations?

Even if I am a bit younger than many others here, i ve already made some experiences. I accompanied about 20 third party Audits the last year of which about 15 were initial certification audits, made by a range of CBs and NBs (medical devices). There were at least 7 audits (sorry i can not remember the correct number) where stage one was just sending in the documentation of the QMS to the NB. During the stage 2 audit the internal audit and the management review were checked.
I can not be sure that there are just two nc during 1 stage but what i wanted to say is that these two nc are for sure (1. No review, 2. no internal audit) There can be more of course.

It maybe a good idea (since it is your stage 1) to rescehdule the stage one until you have finished the cycle of Internal Audits and the Management Review.

In the medical devices industrie time is a big factor. For companies it is neccessary to be certified to be allowed to sell their products, so it is not too easy to say: "Hey we are not ready, lets spend another month or so to set up a system"
This is, why you can use time between diffent stages to improve and set up the system. I agree that this is not the normative correct way.
But overall, how much quality system does a three person machine shop really need? Is a management review as named so really useful? Most bosses I know of very small companies do review their system at least every month, in their brain. And internal audit on a new system, where nearly no data is collected: Very useful. What did the consultant do during his job. Normally he installed a system which will work, based on his knowledge and experience. An internal audit by whom should be performed? By himself? Very useful!!! By the bosses? Very useful. The expense of installing such a system should always be based on the needs of the client.

Regards

phloQS

 

[pldey42]

This is a topic that has annoyed me for some considerable time. Despite, in its 2000 revision, having moved ISO 9001 from a document compliance test to techniques of process-oriented implementation and auditing, many consultants and, dare I say, CBs are still of a 1994 mindset. I say this having worked as a tutor for a CB for several years, now happily moved on.

Neither internal audit (IA) nor management review (MR) should be done at the last minute merely in order to satisfy CB auditors. Like design, training, calibration and manufacturing (or service delivery), they're things the organization should be doing regularly. What are managers doing if not systematically and regularly reviewing the company's performance and processes? And how can an auditor assess the effectiveness, as is required, of internal audits and management reviews that were completed just a few days earlier? It takes time for resultant CAs and PAs to be implemented and to prove themselves, and if IA and MR do not lead to some kind of improvement, they're hardly effective: no organization survives by looking at itself and saying, “Everything's cool, nothing to worry about.” Neither IA nor MR exist in ISO 9001 for the auditors, they're for survival and competitive advantage.

Good managers, good consultants too, introduce IA and MR right at the start of the management system implementation. They train the internal auditors early and involve them in the gap analysis. They use regular management reviews to provide top level guidance to the program, show the managers how to use management review to control their management system, and to sustain management commitment by sharing news of good things that came from the management system as well as to fine-tune direction. Internal audits are used continually to monitor the closing gap. Indeed, sometimes the internal auditors are involved in coaching gap closure, although careful not to then audit their own word. By the time the CB auditors arrive the whole thing is ticking over smoothly, and there's evidence (which they should be looking for) that it's stable and effective.

At my audit training courses people would sometimes say that they were preparing for their initial certification audit which was but a few weeks off. I would have to tell them that, as Andy says, they might be in trouble. Embarrassingly, sometimes they had followed advice given to them by the CB (the one I worked for), which was to train the internal auditors just before the certification audit. There was even an official slide IIRC that said that stage 1 was about document review, little more. I always thought it was daft and taught classes to do more if they could.

(By the way, this does not break the confidentiality agreement I have with my former employer. These things were said in public classes because it was important that everyone understood how the system was supposed to work, even if I had to apologize to the people who had been misled and take a CA back to the office. I suspect they used the error to negotiate a discount and a delay. What happened to the CAs is of course confidential. But I did have to address the same problem at more than one public class.)

Yes, it's an accredited CB and I doubt that it's alone in its myopia. I think one thing that everyone forgets is that ABs are paid by CBs, and for every decent CB auditor like Sidney there are a dozen, um, to put it politely, less able. So change is painfully slow, if it exists in any substantive form.

 

[AndyN]

phloQS

Your comments seem to be at odds with each other. Firstly, your experience with CBs shows why the industry is cynical about Third Party Certification - why feed that perception? If you've seen poor CB practice, why not take action against it? It's not simply a case of offering up a flawed system, wait for the CB to report and then fix what they tell you to (in 30 days).

You correctly point out that time is an important factor, so why encourage actions going into a stage 1, which can delay the stage 2? Let's not forget that, if the client hasn't done either audits nor management review (irrespective of the size of the company) there may be compliance issues which take a considerable time to install and implement, which could have been detected before a stage 1.

We know this OP had a consultant, but many don't have any help. What then? Delays, extra costs and risk of lost customers/market share!

 

[Big Jim]

The right sequence is internal audit-management review-stage 1. If you are do not conduct internal audit or management review before stage 1 the CB will rise a major NC of this on stage 1. But you will have time to resolve it till the stage 2.

I don't know if there is one best answer, but I advocate the following order:

management review

internal audit

management review

stage 1

My thinking is that you need a management review before the first internal audit as you system should be up and running in every aspect possible so the internal audit can audit as much as possible.

Certification bodies want to see how you dealt with any internal audit nonconformances in your management review, thus the need for one after the internal audit as well.

 

[pldey42]

I think there are many reasons why customers do not take overt action against their CBs but instead give them high customer satisfaction ratings, feeding a false sense of satisfaction at the CBs.

Mistrust: Auditors visit but once or twice a year, and there's rarely enough time to develop a relationship of trust. Sometimes different auditors will visit due to scheduling problems, making relationship development even harder. The fear is that the auditor will be even more difficult if there is a complaint. That fear cannot be displaced by a CB saying "trust me", and the internal workings of the CB are opaque, for good yet also bad reasons.

Corrective action: The corrective action at the CB will usually be either to send someone different, or to retrain the auditor, or to support the auditor. Either way something will likely change and life will get harder for the organization. Audits are rarely popular and the management rep typically has a hard time, politically. Helping the CB, who most regard as “the enemy”, to make life harder will rarely be in his political interest, so it's a battle that most will leave for another day.

Cost: With the extra political work that the management rep will have to do to preserve his credibility internally, it's likely to be cheaper to shrug off the problem and say, “Help me out here, just keep 'em happy?” That can even strengthen internal political ties by identifying the common enemy. Unhealthy but pragmatic.

Time management: It takes time to complain, to see it through, especially if it has to be escalated to the AB. And the chances of success, when the ABs are paid by the CBs, and the system is remarkably opaque as we noted in the thread about AS 9100 and some CB suspensions, must surely appear low. Good management reps - the ones most likely to have the chutzpah to complain - are in my experience very careful to invest their precious time only in battles they're sure they can win, that are worth winning.

Consistency: Management reps see that the consistency amongst CB auditors can vary enormously, from stellar to "Get out. Now." They'll realize that they could complain, get it resolved, only for the problem to return with a fresh auditor. It should not be that way but consistency of professional service is hard to manage. For some, anyhow.

I think customers see the system as something that ought to work but which, in practice, works only some of the time. Success is more dependent upon individuals doing the right thing than the system itself. Like an old Harley, customers ride it when it works and either kick it or find another when it won't; unlike typical Harley riders, most people do not fixing it every few hours as an integral part of the journey. I don't think the system will change until or unless governance at CBs and ABs becomes a lot more transparent and, as Sidney and Andy often say, stakeholders take a stronger interest. But I think also the CBs could help themselves and the rest of us if they followed the Baldrige, Six Sigma and SEI-CMM examples of holding themselves accountable for demonstrating a contribution towards performance enhancement.

 

[Sidney Vianna]

So change is painfully slow, if it exists in any substantive form.

If I sound like a broken record, sorry, but as long as the users of the certificates don't DEMAND CONFIDENCE associated with certificates, many CB's and their auditors will continue to think that the (tangible) product of a certification process, i.e., a certificate is more important than the (intangible) confidence in a organization's system robustness.

phloQS' position exemplifies how many stakeholders misunderstand audits, certificates, roles of NB's, AB's & CB's.

The whole certification process has been trivialized and commoditized so it can be packaged and sold more easily. Efforts to keep and restore confidence in certificates will not succeed until the end users of the certificates keep everyone (including themselves) accountable to the need to build and maintain CONFIDENCE in certificates.

 

[pldey42]

Agreed, Sidney.

Not a broken record, just an old one, and they're often the best!

I was surprised at how often I would ask people in classes why they were making a management system. "Because the customer wants it."

"Why?" I'd ask. "What do you do that's so critical for them to want Certification?"

"Dunno," they'd say. "We think it's just a tick box and they don't know either."

The stakeholder demand for confidence has to come from the top of the food chain, IMHO. Everyone else just seems to do what's necessary to get paid.

I've seen that once. In TL 9000 some of the big US telcos got annoyed with low quality CB audits. They got objective evidence of the low quality audits by noticing trends in TL 9000 performance measurements reports (to which they have access) that were not commensurate with being certified. When they went to the suppliers in question and audited they discovered nonconformities that could not be explained away by saying, "Well it was just a sample, we didn't look at it." They complained first to the CBs, then to the AB. I've not heard what happened but these were determined managers of huge supply chains using a management system standard (TL 9000) that they own, and which places specific requirements on CBs, so I guess they got results. Mind, none of that seems to transfer into the other standards, so maybe they're finding it hard too.

If it were practical and possible, Sidney, I'd make your record into an MP3 and blast it through the biggest Fender or Marshall stack I could find at big consumers for management systems like the telecom industry, who understand supplier management and the place in it for independent audits, and who have the muscle to take on the ABs.

 

[Golfman25]

I don't know if there is one best answer, but I advocate the following order:

management review

internal audit

management review

stage 1

My thinking is that you need a management review before the first internal audit as you system should be up and running in every aspect possible so the internal audit can audit as much as possible.

Certification bodies want to see how you dealt with any internal audit nonconformances in your management review, thus the need for one after the internal audit as well.

I would only add that in the real world of a small company there is not "stopping point." Cronological thinking will adversely affect your QMS, especially in a small company like the one described by the original post. We used to do it that way -- A before B before C, etc. It sucked.

When we changed to an ongoing review process it worked a lot better. When we are ready to review an area of the QMS, we do so. If audits aren't done yet, so what. If/when an audit reveals something, we address it then. We have found this to work a lot better. As someone pointed out, owner/managers are always reviewing/auditing their company "in their head." They just need a way to formalize it for ISO purposes. Good luck.

 

[kellyjo]

It is a potential customer requirement and the potential customer is a rather large account. so that is why we are getting certified.

 

[AndyN]

When we changed to an ongoing review process it worked a lot better. When we are ready to review an area of the QMS, we do so. If audits aren't done yet, so what. If/when an audit reveals something, we address it then. We have found this to work a lot better.

Can you help us understand when you say it works better? Better than what?

I'm confused when you describe an ongoing review process and the relationship to audits. The answer to your question so what has already been answered! It's very important that audits are performed, particularly as an input to the reviews!

In my experience, to say any different, is missing the fundamentals of why it's a requirement do audits and, therefore, perform a review...