Following another thread I would be interested to know how other cove members view their external audits. The idea of the external audit is that it reflects how the organization is performing - if you like the audit is holding up a mirror to the organization to see itself from another view.

So how do you rate your external audit?

The idea of the external audit is that it reflects how the organization is performing - if you like the audit is holding up a mirror to the organization to see itself from another view.We certainly find more in the internal audits.

I think that stands to reason, however: The external auditors examine our entire system two days/year. Our internal auditors work in it daily, and audit much smaller portions at any given time. Thus, they are able to examine things in much greater detail. I would be very dissatisfied if they did not find more. I expect them to.

What I do want from the external auditors is that they provide us with an overview and data to back the findings from the internal audits up.

/Claes

This thread irritates me more then anything on the cove.

An audit is a sample. It is your job as managers to find and correct the nonconformities in your quality system. If you are waiting for your registrar to find the nonconformities in your system before you take corrective action….

Then you have an ineffective system.

This thread irritates me more then anything on the cove.

An audit is a sample. It is your job as managers to find and correct the nonconformities in your quality system. If you are waiting for your registrar to find the nonconformities in your system before you take corrective action….

Then you have an ineffective system.


No one is disputing that. What we are asking here is if people think that externals are value-added, or a necessary evil. And - I think it is a fair question to be asking. There are good internal auditors and there are bad internal auditors - you have to admit that. So why can't the same be said for external auditors?

During our last audit, the auditor took something and wrote it up as an OFI. It was related to MR - I was of the opinion that it was a NC, but my boss said it might stir things up too much and get the top bosses' backs up. So, OFI it was. Political. Problem is that MR is a fairly confidential process around here - there was very little evidence that things are actually completed, other than assurances that "it's definitely discussed". Nothing comes across my desk, that's for sure. As an external, I would have called that a NC - but to each his own, I suppose.

It's not that I'm waiting for my auditor to identify nonconformances for me. Sometimes the auditor's findings can be used politically - to deal with changes that are required but are not getting the necessary support, for example. Above all - we have an internal audit system that works, and we are certainly not sitting back and expecting our external auditors to come and clean up our mess - but when word of mouth seems to get us through an assessment, then he** yes, I'm going to question the suitability and the value of the process. Sometimes you just don't get what you pay for.

No. Registration Audits are not a value added process.

The majority of companies achieve ISO registration to sustain or for new business.


Did you write a CA for the OFI?

Did you write a CA for the OFI?

Yes, and it has been implemented.

This thread irritates me more then anything on the cove.
Really? I think it is a great question. If it irritates you so much, why waste your time participating in this thread, then? There are other thousands of threads in the Cove. Life is too short to be wasted with irritating things that can be avoided.

An audit is a sample. It is your job as managers to find and correct the nonconformities in your quality system. If you are waiting for your registrar to find the nonconformities in your system before you take corrective action…
Good answer. But this has nothing to do with the question posed.

Quote: (Originally Posted by cncmarine) This thread irritates me more then anything on the cove.


I don't see what the problem is, but maybe you will explain. In my introduction I thought it clear. We all develop systems and some of us invite external bodies to assess them. Problems occur when their findings are not the same as ours - and more important are not the same as what we (as managers) are telling our own peers and managing directors / CEOs.



Quote: An audit is a sample. Agreed



Quote: It is your job as managers to find and correct the nonconformities in your quality system. If you are waiting for your registrar to find the nonconformities in your system before you take corrective action….

Then you have an ineffective system.
If only life were so simple! I have worked at good companies and bad. I spent 8 months telling one senior management team they had to carry out a management review - the auditor raised nothing.

The point of the poll is that, as professionals we have a good idea how good or bad our systems are. We may know where the problems are and have told the management team that our systems are at risk but if the auditors don't find the same problems then it looks as if we are crying wolf.

Similiarly if we tell the management team that the system is working effectively and an outsider comes in and leaves a long list of "non compliances" then it looks as if we haven't done a good job. No matter that these non compliances are a list of nit picks and opinions.

cncmarine try and have another look at what the post is trying to achieve and then, if it still irritates, choose the ignore button.

Sidney - you are the master of the deft phrase. Ever thought of being a diplomat?

"The point of the poll is that, as professionals we have a good idea how good or bad our systems are. We may know where the problems are and have told the management team that our systems are at risk but if the auditors don't find the same problems then it looks as if we are crying wolf."


If you know where the problems are then take corrective action and fix the problem. If you do not have top management support then it is the old"4.2" quality system failure.


Sidney, debate is a part of the cove and this is a debate. If you do not like it then you ..go some where else

"The point of the poll is that, as professionals we have a good idea how good or bad our systems are. We may know where the problems are and have told the management team that our systems are at risk but if the auditors don't find the same problems then it looks as if we are crying wolf."

If you know where the problems are then take corrective action and fix the problem. If you do not have top management support then it is the old"4.2" quality system failure.

Wow.
Your management must be absolute *angels* to deal with if you cannot think of even *one* example of this problem from your own experience.

Sidney, debate is a part of the cove and this is a debate. If you do not like it then you ..go some where else

Not what you say, but how you say it...you started it harshly - don't expect otherwise in return.

Wow.
Your management must be absolute *angels* to deal with if you cannot think of even *one* example of this problem from your own experience.

No. I'm just an effective manager.

If you know where the problems are then take corrective action and fix the problem. If you do not have top management support then it is the old"4.2" quality system failure.
Simple indeed. And after we've fixed these problems I am going for world peace and feeding the starving. In the real world individuals have a job to do that envolves communication, negotiation and coaching. Just by saying it is so doesn't make it happen. Trying to bring this back to the point if external audits are going to have value they have to be in tune with what you as the Quality Manager (or any other title) having been telling the top people.

Sidney, debate is a part of the cove and this is a debate. If you do not like it then you ..go some where elseFor my part Sidney pitched his reply right. Judging by your last few posts you, on the other hand, appear to have a problem. :mad:

As a result of the mature, professional comments that cncmarine has so quickly moved to delete lest someone *important* see them...
...I am walking away from this discussion in order to find one that is currently underway in a location *other* than the playground.

It was not my intention to get into a ****** match with anyone in this forum

I have brought up some good points and reading between the lines some very good debate.
This poll and all of your responses = ineffective systems

If the purpose of the poll was to ask the question..Are external audits a value added activity...then the answer is NO. (for the second time)

But that was not the poll

As a result of the mature, professional comments that cncmarine has so quickly moved to delete lest someone *important* see them...
...I am walking away from this discussion in order to find one that is currently underway in a location *other* than the playground.

I deleted my comments because I do not want to get personnel with any one. I apologize if I offended you..But it was not place or your fight

All right... This is what Paul wanted to discuss: Following another thread I would be interested to know how other cove members view their external audits. The idea of the external audit is that it reflects how the organization is performing - if you like the audit is holding up a mirror to the organization to see itself from another view.

So how do you rate your external audit? Imo, there is nothing wrong with the question, so let's discuss the subject in a calm way, shall we?

/Claes

I have been through several audits by several registrars at several companies.
The extent of the results seems to depend on the desires of the company and its relationship with the auditor.
Some companies only want to pass the audit without any feedback on improvement (they don’t follow the process anyhow). Other companies are interested in improvement but see the auditor as an adversary and only want good reports. Still other companies (very few in this category) see the auditor as a team member in the quest for continuous improvement and welcome minor nonconformance’s, observances and “informal” discussions on the state of the quality management system.

Unfortunately the result of an audit is pass or fail (and, except is very very rare cases, failure is not an option). It would be good if the ISO registration included levels of maturity similar to CMMI. This would give a more realistic assessment of the quality management system and its progress in continuous improvement.

Just my thoughts.

"IMO, there is nothing wrong with the question, so let's discuss the subject in a calm way, shall we?"

The question is a very apt one IMO.

We have a situation which is very, very frustrating. It is astounding to me that in such a small organisation such as ours that the one person who is supposed to look after our IT needs is so distant and unresponsive. My computer was recently "upgraded" due to a problem with Excel. I ended up with wrong and missing applications. This was four weeks ago and I am still waiting. This has put a halt to the developement of our training programmes, among other things. By the way this person is our CFO. And the other area were we have problems is Human Resources. Same person along with the accounts assistant, a very incompetent person. The payrole has problems more often than not and this affects the people who really matter. IMO, the people out on the factory floor. These two do SFA for moral here and it affects every aspect of the organisation. It is so hard to rock the boat here, we are so small and the relationship between some managers and is very cosy. we have a bad problem with the US and THEM sindrome here.

I have flagged these problems, both informally and formally with my direct supervisor (QM) and with the CEO, yet we still have to suffer.

We have our first surveillance audit since gaining ISO9001:2000 accrediitation in March, so lets see if our auditor picks up on this and we are tagged for any Management Responsibility short comings, i.e Resourses etc.

I don't have an axe to grind on this topic, but I do have some observations after watching organizations deal with 3rd party registration audits for the last 10 years or so.

I think there are three main reasons why organizations seek formal registration from a 3rd party auditor:

Coercion from customers ("no registration - no business")
marketing ploy ("Hey everybody! We're cool, we have ISO xxxx registration.")
genuine desire for outside validation/benchmarking of QMS
Sadly, many, if not most, Quality department personnel have no official clue as to the true reason the organization seeks registration versus mere compliance.

Therefore, once the 3rd party auditor has come and gone, different folks with different understanding of the organization's true motives for registration, will be either elated or disappointed by a relatively superficial audit.

Like it or not, many Quality professionals harbor a deep-seated resentment of 3rd party auditors "second guessing" the QMS. If the audit is superficial, the in-house Quality guys feel "cheated." They wanted to show how good they really were and the auditor didn't seem to care. (Scorned lovers often seek the most extreme revenge.)

If the audit is very rigorous and turns up many NC and OFI comments, those same in-house Quality guys feel persecuted.

All the anguish is compounded when top management (because they have a different motive for the registration) have a completely different view of the auditor. I can see where some insecure folks may harbor "conspiracy theories."

The bottom line:
If both top management and Quality department are on the same wavelength about what they expect from an audit, this should be made clear during the Contract Review stage when engaging the Registrar. If it wasn't made clear at the Contract Review, then nobody has a valid gripe if the rigor is too much or too little. If, after the first audit, expectations are not met, then it is time to re-enter negotiations with registrar (not the individual auditor, unless they are one and the same.) In my view, a registrar is a supplier. He either measures up or you get another supplier. It's up to the organization how much leeway or negotiation to undergo before switching.

I think of it in a very similar fashion to 3rd party audit by independent CPA firm. Once the regulatory requirements are covered, the relationship still boils down to how well auditor and auditee get along. Lots of SEC-regulated organizations switch CPA auditors every year for a myriad of reasons, some of them strictly personality related.

I think there are three main reasons why organizations seek formal registration from a 3rd party auditor:

Coercion from customers ("no registration - no business")
marketing ploy ("Hey everybody! We're cool, we have ISO xxxx registration.")
genuine desire for outside validation/benchmarking of QMS
Sadly, many, if not most, Quality department personnel have no official clue as to the true reason the organization seeks registration versus mere compliance.

Therefore, once the 3rd party auditor has come and gone, different folks with different understanding of the organization's true motives for registration, will be either elated or disappointed by a relatively superficial audit.

Like it or not, many Quality professionals harbor a deep-seated resentment of 3rd party auditors "second guessing" the QMS. If the audit is superficial, the in-house Quality guys feel "cheated." They wanted to show how good they really were and the auditor didn't seem to care. (Scorned lovers often seek the most extreme revenge.)

Isn't this the saddest thing. I can only agree with the list above and the choice will lead to a different perception on the value of an audit. 1 and 2 above and all the company will look for is a gloss over. Choose 3 however and you want the auditor to give you the impartial overview of your system to identify any areas for improvement that exist.

If the audit is very rigorous and turns up many NC and OFI comments, those same in-house Quality guys feel persecuted.
Again agreed but I wouldn't use the words rigorous. If the systems are good then a rigorous audit won't identify NCs - because they don't exist. The only time a really good system has NCs raised against it is when the auditor is trying to impose his/her will on the organization - maybe they feel this is their job. Not however a good audit.

All the anguish is compounded when top management (because they have a different motive for the registration) have a completely different view of the auditor. I can see where some insecure folks may harbor "conspiracy theories."
Again my experience with CEOs is that it depends why they have a registration (see Wes's list above). If they are after option 3 (as many are) then to be given the clean bill of health when their QM is telling them problems exist leads them to question the value of the audit.

The bottom line:
If both top management and Quality department are on the same wavelength about what they expect from an audit, this should be made clear during the Contract Review stage when engaging the Registrar. If it wasn't made clear at the Contract Review, then nobody has a valid gripe if the rigor is too much or too little.

If the option of an "easy" or "rigorous" audit exists at the contract review stage then it shouldn't. You don't have certificates that state "Proper" or "In name only" for the customers to distinguish.

If, after the first audit, expectations are not met, then it is time to re-enter negotiations with registrar (not the individual auditor, unless they are one and the same.) In my view, a registrar is a supplier. He either measures up or you get another supplier. It's up to the organization how much leeway or negotiation to undergo before switching.
This is an interesting idea. Whilst I agree that there should be more dialogue between the two parties I don't think you should be able to choose the degree of rigour the auditor applies. More feedback on the quality of the audit and the ability to point out where the auditor is overstepping the mark or highlighting where an auditor is not finding enough perhaps.

I think of it in a very similar fashion to 3rd party audit by independent CPA firm. Once the regulatory requirements are covered, the relationship still boils down to how well auditor and auditee get along. Lots of SEC-regulated organizations switch CPA auditors every year for a myriad of reasons, some of them strictly personality related.
Forgive my ignorance but I presume CPA ia accountancy. But a good analogy. My point is that the regulatory requirements are equivalent to the standard requirements. Rarely do you get an accountant "missing" the fact that huge chunks of money are going missing (putting aside recent high profile examples!). Similarly you don't get the financial audit failing to sign up to the accounts when there is a penny missing on a single purchase order payment. Yopu are absolutely right about buiding the relationship though. The best registrars are the ones who work with you to understand your business and who "tune" their audit to reflect the business risks that exist within your processes.

Rachel,
When we were first audited (for registration) back in August, we had no majors and no minors, but had several OFI. We wrote PA's for the OFI, and have reserved writing CA's for NC's, whether majors or minors, and of course, for anything else that needs 'fixing'.

The rationale was that an opportunity for improvement means that something 'could be improved' rather than 'must be corrected' and a preventive action appeared to be more appropriate. Of course, these are not black-and-white, cast-in-stone issues, but what is your opinion on this approach?

The OFI written by the BSI auditors were right on cue, and we were well aware and were already addressing them all but one. We were glad that the BSI auditor identified 'the one', as although the chances of it happening and becoming a problem are slim, it could nevertheless occur. Now are prepared for it 'ahead of it happening' - maybe it won't - maybe it will.

BTW, we will have our first, semi-annual external audit next month, and by then, the way I set it up, we will have had two of our quarterly internal audits.

Thanks!

Alex

The best registrars are the ones who work with you to understand your business and who "tune" their audit to reflect the business risks that exist within your processes.

Wow!! Great concept, Paul. I embrace this notion 200%. I wonder what results I would get if I googled "risk based certification" ? ;)
http://www.google.com/search?sourceid=navclient&ie=UTF-8&rls=GGLD,GGLD:2004-36,GGLD:en&q=%22risk+based+certification%22

Wow!! Great concept, Paul. I embrace this notion 200%. I wonder what results I would get if I googled "risk based certification" ? ;)
http://www.google.com/search?sourceid=navclient&ie=UTF-8&rls=GGLD,GGLD:2004-36,GGLD:en&q=%22risk+based+certification%22

Naughty Sidney. Take that tongue out of your cheek. How did you turn my words round to a phrase used by your company (Trademarked?). Just to set the record straight ... there may be other certification bodies out there who can offer the service. I can name some!

What standard is "Risk Based Cert." for?

Getting back to the question at hand, I've selected the first option...that findings are pretty much in line with how I see things. I believe it was cncmarine who reminded us External Audits are merely a sample of how things are working. One or two people who visit for one or two will not see everything, will not understand everything, but the findings (as nit-picky as I may find them sometimes) are valid.

Audits are a double-edged sword. No one wants findings, but for those of us who want more than just a piece of paper on the wall, we want our systems to be perfect, and sometimes, that means findings are issued to help get us to that stage of "perfection".

The politics of issues being classified as an NC or OFI are, unfortunately, the way things are. Now it's up to us to make the most of it and guide our senior managers to a solution that will keep the majority happy, if not the concensus.

I'll admit that I really don't currently see anything wrong with my company's BMS and thus, an audit with 0 findings is in line with how I see things. OFI's however, were issued and are greatly appreciated.

And, the times when I have known that something was wrong, there have been findings...maybe not always all the ones I know about, but limited people with limited time will find a limited amount of findings.

I hold my pool of Internal Auditors to a higher standard. They live and breathe our BMS every day....well, 5 days a week, actually. :) I expect them to speak the language of the operator. I expect them to know what they're looking for. I expect them to have findings (because no system is pefect) along with Opportunities for Improvement.

I say all of this just I'm finishing up what I can for our Registrar's questionnaire. I sent a copy to every auditee (from the VP to guy working at the bundling station). Results will be compiled and averaged for our company's overall opinion of our Registrar and our recent surveillance audit. All comments will be added, though...the good and the bad.

I will not, however, submit this questionnaire until I have received the Audit Report. This way, I can effectively answer the questions dealing on whether or not I received the report in a timely fashion. :notme:

What standard is "Risk Based Cert." for?
Risk Based Certification is a protocol/methodology and can be deployed for any management system standard (ISO 9001, AS9100, TL-9000, ISO 14001, etc...)
Even though your opinion is that external audits do not add any value, others think otherwise. For those who think that an external audit can add value to an organization and their customers, Risk Based Certification is DNV's recently released answer to this expectation.
Thanks for asking.

You’re Welcome.

I have a lot of lean in my background so a “value added” audit doesn’t make too much sense.

The DNV is a great business idea but I don’t agree that is should be used for AS9100.

You’re Welcome.

I have a lot of lean in my background so a “value added” audit doesn’t make too much sense.

If I may, In the lean manufacturing world, "value added" is an activity that the customer is willing to pay for. In the world of ISO, that definition is less practical. The client has to pay for everything. But the key word is "willing". In the case of registration, value added is when I say that "I got my money's worth."

A registration audit is a necessary expense to achieve or sustain business. I do not look for any improvement in my organization from the registration / surveillance audit.

Effective Internal Audit systems or corrective / preventive action program can be argued to be value added activities. But a registration audit is an audit of the system vs. the standard.

I believe that ISO / TS / AS are great quality management systems. Lean manufacturing and ISO can work together and they both can be value added activities.

A registration audit is a necessary expense to achieve or sustain business. I do not look for any improvement in my organization from the registration / surveillance audit.So it could be argued that the "badge on the wall" adds value then. Not a negotiating position I use often. I always say this is the icing on the cake. As I have posted in the past the audit is like a qualification exam. You put the effort in to get to the end result.
Effective Internal Audit systems or corrective / preventive action program can be argued to be value added activities. But a registration audit is an audit of the system vs. the standard. They both cover the same ground. Internal audits look at the degree of compliance with company systems and so do external audits. The company systems in their turn are designed to meet the requirements of customers, markets and any international and company standards. The external audits may have to be carried out with slightly more emphasis on the standard requirements but they should be looking at the company objectives and targets, process measures and the key performance indicators of the business. This then can become a value adding service (IMHO).

If your audit is nit picking and looking only for compliance (as opposed to the process approach) then you won't get a value add.

Good points.

We have seen the registration auditing change for the 1994 standard to the 2000. The registrars are looking into company objectives and targets, process measures and the key performance indicators of the business. But looking into is one thing, determining the effectiveness of is another.

The problem with this is the registrars moving into interpretation. What I don’t want to see is “ you have met the standard but I believe its not enough”

That’s consulting not registration.

The problem with this is the registrars moving into interpretation. What I don’t want to see is “ you have met the standard but I believe its not enough”
We had this to a certain extent in the 1994 version. Auditors saying that your statistical techniques are not adequate, or that your product id is not good enough.

In the 2K version, it becomes even more dangerous. What are "appropriate" training records anyway? What is involved in "monitoring" "customer perception"? I often say that the organization needs to define all of these things, otherwise the auditor will fall back on their "default" definitions.

We Agree

We had this to a certain extent in the 1994 version. Auditors saying that your statistical techniques are not adequate, or that your product id is not good enough.

In the 2K version, it becomes even more dangerous. What are "appropriate" training records anyway? What is involved in "monitoring" "customer perception"? I often say that the organization needs to define all of these things, otherwise the auditor will fall back on their "default" definitions.

I understand the problems. The point of the poll and these discussions is to try to get some agreement on both sides of the assessment process. If I come in as an auditor I have to justify why I believe the statistical techniques are no good - are they based on flawed mathematics? Similarly if there are no complaints from the market place I am in no position to criticize products.

The point about the "danger" of 9k2k is a difficult one. As an auditor I can ask questions about the setting of objectives (if they do not appear in tune with the market) but wouldn't raise any non conformity if the company is happy they are right - and I would then monitor to see if they continue to serve the company.

I can look at how an organization is monitoring customer satisfaction and see if they have considered other methods and I can then start to ask more difficult questions if they are obviously getting no value from the process or are doing nothing with the data. I don't raise non conformities at the first visit but would "turn up the heat" if I thought a company was playing lip service to the requirement. Not because I want to give anyone a hard time but because the element is in there s a requirement because it is good practice and a registration to 9k2k should mean the company is following good practice.

I have been auditing some companies as an external auditor. I went there to see whether the company is in compliance with ISO 9001. The time is short but there is always "auditors luck". The purpose of the audit is not to "discover" nonconformities but to look at the system as a whole and try to find out whether there are any major flaws. I carried out the audit to point out ways to improve the system so that the audit is valued added.
Yes, I believe, external audit is value adding. It depends upon the individual auditors as well.

"I carried out the audit to point out ways to improve the system so that the audit is valued added."

Are you working with a registrar ?

Most people voted for "Not enough non compliances. There are issues in my system the auditors do not seem able to find."

Q: Are there any particular reasons why registrars don't find the issues we know and see? Some registrars are trying to be certified to conduct FDA inspections. Has anyone gone through those type of inspections yet? What are your results?

Most people voted for "Not enough non compliances. There are issues in my system the auditors do not seem able to find."

Q: Are there any particular reasons why registrars don't find the issues we know and see? Some registrars are trying to be certified to conduct FDA inspections. Has anyone gone through those type of inspections yet? What are your results?

What is an audit?...it is a random sampling of processes. You have one or two people who are no experts on your system coming in for a few days. It isn't logical to expect to find everything that we see as imperfect with our systems.

For me, the frustration often comes not in the number of noncompliances...but in their caliber. I mean, we recently received a noncomformance on our lawn sprinkler system. ?!?! Not much value in that finding...yes, it was valid, but in my opinion there were bigger fish that could have been fried.

That's the great part about internal audits. You are the experts of your company and system. You should know where the issues are and be able to dig them out....leaving nothing big for your external auditors to find. :)

For me, the frustration often comes not in the number of noncompliances...but in their caliber. I mean, we recently received a noncomformance on our lawn sprinkler system. ?!?! Not much value in that finding...yes, it was valid, but in my opinion there were bigger fish that could have been fried. Words of wisdom... meaningless non-conformities: they create so much non-value added busy work http://www.flashplayer.com/forum/images/smilies/rtfm.gif . Any auditor, BEFORE writing up a non-conformity, should make a mental exercise and ask themselves: How will this organization become a better one, after correcting this non-conformity? If I can not convince myself that a true improvement will come from the write up, I should ask myself really hard what is the point of documenting such non-conformity.
Just my thoughts. http://www.flashplayer.com/forum/images/smilies/russian_roulette.gif

Words of wisdom... meaningless non-conformities: they create so much non-value added busy work. Any auditor, BEFORE writing up a non-conformity, should make a mental exercise and ask themselves: How will this organization become a better one, after correcting this non-conformity? If I can not convince myself that a true improvement will come from the write up, I should ask myself really hard what is the point of documenting such non-conformity.
Just my thoughts.

You're right, Sidney. If such mental exercises were done, not only would non-value-added nonconformances be reduced, but people might also believe more in the benefits of Management Systems. Too many people see our profession as "necessary overhead" (my title was once called that by a former president!) and not as a "supporting resource." I honestly can't blame them for all the red tape and paperwork that is so often part of our profession.