I'd like to get some feedback from Covers about their views about internal auditing. From my constant involvement here and with many organizations, I feel it's time for the internal audit 'model' which is commonly in use today (like RABQSA course requirements) to be segmented and quite seperate from external (supplier/registrar) audit training (typically Lead Auditor training).

I'm sure this will be controversial......

Thanks for everyone's participation.

Andy

Perhaps you can explain what you mean by 'model' so that any respondants are speaking the same language (and people who just view can follow along and potentially gain insight from this thread).

Why do you feel that there should be a distinct separation in the training? Or are you discussing the separation of the internal audit methodology from the external audit methodology?

I admit that I don't see the relationship between the poll and the original post.

Ditto RCBeyette.

Think of the three branches in government. I am thrilled to be a part of a process from the ground up in which auditing is changing from a justice system model to a sort of internal consultancy.

In my wanderings I can help people to connect the dots as they might not because of their process focus.

The executive branch can benefit by seeing how all the various efforts are contributing to the common goal (of reducing prices, etc).

High level managers can understand how to focus strategic efforts because, for example, adding a couple of staff members to this department can free up their highly paid people for R&D in product, or to break some new ground how about R&D for service we provide our customers? They may be too detail or objective driven to have a firm grip on this concept.

Middle managers can receive data that this or that improvement effort succeeded or missed expectations in areas outside their departments. They may be getting this information in part already, but if they aren't asking all the needed questions their data may have holes and--presto--misguided initiatives may result.

Supervisors can understand how their group's efforts have impacted the process by designing an efficiency gain or other improvement and managing its progress. They could be encouraged to view their contributions as parts of a whole system and less than a series of successful actions.

Workstation personnel could get a better sense of how their immediate efforts impact results later. By getting a sense of how to gauge doing things well and adopt a rather constant vigilance in "How might we improve this process?" they can become a part of the solution if they aren't already.

This is possible to do but it fundamentally moves away from the compliance auditing model. The movement process involves changing the focus on both large and small scales, ranging from the QMS focus itself to process mapping with a PDCA focus and a means to communicate meaningful results feedback in healthy ways.

This isn't for everyone. Auditing to such things while maintaining necessary compliance vigilance can become a muddle in my head.

I am in the process of designing a model that includes the necessary elements to prove value to process owners, train for repeat performance, win hearts and overall provide the vehicle (as a set of user-friendly tools and methods based on an existing quality-focused culture) for ratcheting the organization to the next higher performance level.

I've done internal and external auditing and really don't see much difference of substance in the methodology.

As far as they are benficial?
That's all up to upper management, isn't it?
If they take them seriously then they will be beneficial. If they see them as a necessary evil, not so much. If they seem them as a non-priority then not at all.

So maybe the question would better phrased as "does your management support an audit program"?

The best audit process in the world will be ineffective if it's not supported.

O.K - so here's my reason for asking:

Quality systems auditing - the implementation - is generally based on training which has its origins in supplier evaluations, back in the 80's. The process; planning, conducting, reporting and follow up, often doesn't address the role the organization's management has in determining the need for the audit. Sure, we know management might have to respond to audit findings, but there's much more to it than that.

The focus is normally on maintaining compliance to a set of requirements (which management generally don't care to understand) and delivers reports which are (in my experience) not tied to important business/customer satisfaction issues. In short, the fundamentals of internal audits are based on the auditor acting like an outsider who turns up, finds things to report, requires corrective actions and then 'leaves'.

This is the 'model' I'm refering to. I believe that as a result, audits are not supported by management, auditing isn't seen as a career progression, etc etc. the list is extensive.

Since many audit programs are set up based on what is taught in classes (Internal auditor, Lead Auditor) the teaching 'model' should be changed..........and hence the RABQSA requirements and the need for clear distinction of external compared to intenal audits. Sure some of the basics are similar (at a fundamental level) but in most respects a complete overhaul is necessary to start elevating the value of audits, past mere compliance.

Andy

I voted Yes - measureable benefit....etc. Not so sure it's "measurable" but audits do take priority and are important to us. I guess that should put my poll answer in the middle of those two "Yes" answers! We do compliance auditing exactly as Andy described. I cannot think of any audits in the past 8 years where a "measurable" outcome resulted from an Internal or an External registrar audit. Sure there have been "Opportunities for Improvement" (OFI's) identified and acted upon but measureable? ...Nope. :nope:

FYI: My internal auditors do not like the Process Approach and therefore do not volunteer to audit anymore. They feel the "section auditing" was more effective. So, I end up doing all auditing except where I cannot audit myself. Internal Auditing here is voluntary it is not mandated...we all wear multiple hats and leaving your job to audit is difficult for many here. All of my auditors are from other departments and none of them report to me.

I've done internal and external auditing and really don't see much difference of substance in the methodology.

Thanks for the comment - this is exactly my point - the methodology is the same because the roots of the training are the same..........:agree1:

Andy

I've done internal and external auditing and really don't see much difference of substance in the methodology.

which is Andy's point...

As far as they are benficial?
That's all up to upper management, isn't it?
If they take them seriously then they will be beneficial. If they see them as a necessary evil, not so much. If they seem them as a non-priority then not at all.

So maybe the question would better phrased as "does your management support an audit program"?

The best audit process in the world will be ineffective if it's not supported.
True enough - as far as it goes. but this is really an iterative and evolving process - as ISO 2000 intended.

initially, INTERNAL auditing is primarily about compliance; and for the purpose of achieving of registration it is value add...

However, after registration is achieved merely auditing for compliance is not very value add at all - in *my* opinion and expereince. and it therefor rapidly loses active support from management in many cases. It becomes nothing more than a police action or at worst it's perceived as "tattling"

I think the question really is: is internal auditing - as it is typically performed as compliance based - value add? if not, why not? or if so, why?
and the logical next question is how to make internal auditing more value add? Andy - correct me if I've gotten it wrong.

As for management support - yes it is necessary but not sufficient in itself. After all "management" doesn't always know what is the best approach to take. That's why they hire subject matter experts: us. we need to recommend and implement and demonstrate better ways of doing things. This is what gains effective management support. It's a bit of a chicken and egg thing. It's iterative.

When I was a Quality Manager I required my internal audit teams to perform effectivness auditing and de-emphasized compliance auditing after registration.

an example: at one of my previous jobs my lead auditor became 'enthralled' with a finding in the packout process. The procedure had a 'throwaway' line that the operator was to perform a certain aspect of the process while seated on a swivel chair. The operators didn't do this - they stood. They could have performed the operation while seated but chose not to. Now this seating 'requirement' was not needed for ergonomic or safety reasons. It was just a silly slip - the originator of the procedure had observed the original operator sitting on swivel chair and simply wrote it into the proceure years before the audit and subsequent revisions of the procedure didnt' remove teh statement. Now of course it shouldn't have been in there if not required, but in the grand scheme of things this was so far on the right hand side of the pareto for causes to our quality problems that it was ridiculous. we could have spent time issuing a finding and revising the procedure and then re-auditing to ensure that the change was implemented and logging it in the database and - oh since this was an internal audit finding we also needed to determine and eliminate the cause of why operators weren't following the procedure - but towards what end? We had real quality problems to deal with and a finite set of resources with which to address them. Not only did managment find this finding - and the many others like it - non value add, but they questioned the usefulness of the entire audit function. The auditor never saw or even really looked for the process issues that were driving our quality problems in this function.

From my perspective, auditing for compliance is needed but it should not be the primary purpose of internal audits. So compliance audits, in my opinion and my experience are of limited value add to any organization because all they do is maintain the status quo, they don't initiate significant improvements and are therefore not considered highly value add to managment.

Here's another thought: ISO 9000:2000 requires continual improvement. It also requires that audit results be included as input (this would of course include internal audits)

8.5.1 Continual improvement
The organization shall continually improve the effectiveness of the quality management system through the use of the quality policy, quality objectives, audit results, analysis of data, corrective and preventive actions and management review.

So ask yourself: does your internal audit system provide results (findings, observations, etc) that provide useful and actionable input for improving the effectiveness of your QMS?

So ask yourself: does your internal audit system provide results (findings, observations, etc) that provide useful and actionable input for improving the effectiveness of your QMS? Working on it...

Quite honestly these potato's have been mashed more than once here over tha last few years.

It boils down to committment, competence, and give a s&it.

The only time benefit is really derived from the internal audit process is when the organization truly wants it. If there is no real desire for a value added process then the noises are made, the trash cans are kicked over, and a truly great S&M show may present itself and there you have it.

I agree with Randy but I still voted a big NO! Major changes come more from management 'think tanks' than facts obtained during audits. We do rectify process or procedural issues and all NCs are rectified but as a means of company development they don't do ANYTHING.....as managment do not USE the info. I report waste, high costs, non financial reporting of NCs etc and we never do anything. I used to find the nearest wall and talk to it before butting my head, repeatedly, against it :lmao:

I agree with Randy but I still voted a big NO! Major changes come more from management 'think tanks' than facts obtained during audits. We do rectify process or procedural issues and all NCs are rectified but as a means of company development they don't do ANYTHING.....as managment do not USE the info. I report waste, high costs, non financial reporting of NCs etc and we never do anything. I used to find the nearest wall and talk to it before butting my head, repeatedly, against it :lmao:


If indeed it is "reporting waste, high costs, non financial reporting of NCs, etc, then what you describe is not a failing of the internal audit process, but rather a failing of top management to lead their company...

Let's put the root cause where it belongs?

We do compliance auditing exactly as Andy described. I cannot think of any audits in the past 8 years where a "measurable" outcome resulted from an Internal or an External registrar audit. Sure there have been "Opportunities for Improvement" (OFI's) identified and acted upon but measureable? ...Nope. :nope:

Then apparently you are not getting good audits?


FYI: My internal auditors do not like the Process Approach and therefore do not volunteer to audit anymore. They feel the "section auditing" was more effective.

I suggest if they really understood the process approach - CORRECTLY PERFORMED - they would love them. The section audits were easier to do, because the format was clear. They can be structured to be clear and easy as well. But, when I explain the right way to do them to a motivated auditor, they love it and can't wait to get started. It really is a good tool if done right.

Wow, your not kidding this is controversial!

My vote was Yes we get a measurable benifit. I don't disagree with my fellow Covers that say you need management to ep it up to help get things resolved. In fact that is my biggest frustration is getting some "Teeth" into lack of action to those who choose to ingnore or give some half-*** answer to findings. I would sure like management to have given some of these people the heave ho not just for the lack of seriousness to internal audits, but in demonstrating that their attitude seems to translate into an indication of a real problem with their over all performance.:nope:

Upon further thought and being an observer to our sister plant's ISO9001 re-registration audit last week I'm thinking the following:

Compliance audits, especially to mandatory regulations, should take more "old fashioned" approach of making sure the auditee is doing nothing wrong.

The focus of internal audits, on the other hand, should be more positive with the goal of catching the auditees doing the right things. They should result in preventive actions more than corrective actions.

The latter is actually how I structure my internal audit program and even thought the methodology may be very similar for both, the actual practice or tone of the audit is different.

Maybe it is time to differentiate.

Then apparently you are not getting good audits?

Hmmm, I never said that. I'm outta this thread.

Our plant (one of many in our corporation), which supplies OE automotive parts, provides in-plant training to TS 16949 for our Internal Auditors from an outside supplier. For the most part, our internal audits are performed very much like audits by our registrar. It seems to me there isn’t much room in the standard for variation. We also try to keep our internal audits similar to our registrar’s audits so our personnel will be familiar with the audit process.

Since we are part of a large corporation, we also get PPS audits from Corporate Quality. These are much more difficult and revealing than our internal audits or our registrar’s audits and are taken much more seriously by management than our internal audits and therefore are the ones with the most measurable benefit. As an example, we have had many “observations” about Key Performance Indicators from internal audits and our registrars, but a finding in the most recent corporate audit resulted in standardization of the KPIs throughout the plant; something we’ve needed for a long time. I guess what I’m trying to say is, yes we get measurable benefit from “Internal Audits”, but they don’t give us the most bang for the buck because of how they are viewed by management.

All this being said, let me tell you about one of the best internal audits I have ever seen. Another internal auditor and I came in on a Saturday to audit one our departments. We had several findings and observations (opportunities for improvement) already written down when we came across an employee who obviously had no idea where to find any of the work instructions, postings or other documents relating to his job. All he knew was what he had been trained to do. My colleague, who was not getting paid for the overtime, immediately stopped auditing and took the next two hours to walk this guy through the quality system in his department. To me, this is what internal audits should be all about.

My answer is yes! Let me explain some of the things we do to get more out of the audit than just meeting the requirements.

Audit Team
We currently have 80 (close to 10% of workforce) associates on the audit team. The auditors are from every organization and every level. We have technicians to managers from accounting to manufacturing. They have all been through a week long class, the typical ... how to audit, understanding the standard, etc. We don't require they audit during every audit. So what are the benefits?
#1 - even if they don't audit much, they understand our quality system. Our quality system has improved significantly by doing this. It has become everyone's system, not just QA's. They take ownership.
#2 - We have multiple factories on site with each having their own way of doing things. The auditors have been able to learn from the others and give recommendations. Unlike external auditors, we do "consult".
#3 - they are my link to all organizations and I'm not begging for auditors. For the most part, everyone does their share.
#4 - I have yet had to recruit people to join the team and keep a waiting list for the next class. I think this says a a lot about the team and the perception of the internal audits.

Audit Approach

We conduct two audits per year. During the first audit, we use the process approach. During the second audit, we make sure to cover primary elements or any that we missed during the first. I think this approach ensures "wide and deep" coverage.

With the number of auditors on the team, we have a large resource pool. Our team conducts "special audits". These audits may be conducted if a weakness has been identified or if a major change has been made. Example; we conducted a special audit relative to "Management of Change" for one factory. They recognized that many of their problems were a result of making changes and not managing them well.

Best Practices -- we use this as a finding category. This allows us a way to recognize when an organization is doing a great job in a certain area. We share it with the site and encourage others to adopt it where applicable.

The benefits come from involvement and continual improvement. We have gone from having "the office people" in white jackets as auditors to what we have now.

Please give a normative reference for "measurable benefit". Hard numbers in dollars? Less 3rd party and customer audit findings? Sure we get all of that. You just said measureable you didn't specify how much. Every little bit improves the effectiveness of the QMS and helps us do more business from a marketing side, otherwise who would do it? Again even if the cert is just for marketing purposes the benefit is measurable as business would drop without it.

My answer is yes! Let me explain some of the things we do to get more out of the audit than just meeting the requirements.

Audit Team
We currently have 80 (close to 10% of workforce) associates on the audit team. The auditors are from every organization and every level. We have technicians to managers from accounting to manufacturing. They have all been through a week long class, the typical ... how to audit, understanding the standard, etc. We don't require they audit during every audit. So what are the benefits? #1 - even if they don't audit much, they understand our quality system. Our quality system has improved significantly by doing this. It has become everyone's system, not just QA's. They take ownership. #2 - We have multiple factories on site with each having their own way of doing things. The auditors have been able to learn from the others and give recommendations. Unlike external auditors, we do "consult". #3 - they are my link to all organizations and I'm not begging for auditors. For the most part, everyone does their share.#4 - I have yet had to recruit people to join the team and keep a waiting list for the next class. I think this says a a lot about the team and the perception of the internal audits.

Audit Approach
We conduct two audits per year. During the first audit, we use the process approach. During the second audit, we make sure to cover primary elements or any that we missed during the first. I think this approach ensures "wide and deep" coverage.
With the number of auditors on the team, we have a large resource pool. Our team conducts "special audits". These audits may be conducted if a weakness has been identified or if a major change has been made. Example; we conducted a special audit relative to "Management of Change" for one factory. They recognized that many of their problems were a result of making changes and not managing them well.
Best Practices -- we use this as a finding category. This allows us a way to recognize when an organization is doing a great job in a certain area. We share it with the site and encourage others to adopt it where applicable.

The benefits come from involvement and continual improvement. We have gone from having "the office people" in white jackets as auditors to what we have now.

:applause: Congratulations! It sounds like you folks "Get It!" Bravo!

:confused: Hmmm, I never said that. I'm outta this thread.


Sorry, I wasn't trying to be offensive. Your post stated you were getting "no measureable benefit" and that your auditors did not like the process approach and did not want to audit anymore.

I guess I understood that to mean you were not getting good benefits from your audits. If you reread my post, you may find the seeds of a good idea in there...or you can just get mad and get no benefit...:confused: :truce:

:confused:

Sorry, I wasn't trying to be offensive. Your post stated you were getting "no measureable benefit" and that your auditors did not like the process approach and did not want to audit anymore.

I guess I understood that to mean you were not getting good benefits from your audits. If you reread my post, you may find the seeds of a good idea in there...or you can just get mad and get no benefit...:confused: :truce:

Very sorry...You're forgiven - not your fault. I perceived your comment as an accusation rather than a question and well, it's a long story....I've been "on-the-edge" of going absolutely ballistic at work. False accusations, stress, and death-survivor guilt feelings? I finally beat the living sh%t outta my punching bag last night.:D I feel better today.

Yes, I thought and management thought we have had very good productive audits and learned from them but "measureable"...I just cannot see that they have been measureable ($$ or improvements, or ?? ) There haven't been any trends because I keep track of those things. Did someone define "measureable"? Maybe I am off-base?

I'd like to get some feedback from Covers about their views about internal auditing. From my constant involvement here and with many organizations, I feel it's time for the internal audit 'model' which is commonly in use today (like RABQSA course requirements) to be segmented and quite seperate from external (supplier/registrar) audit training (typically Lead Auditor training).

I'm sure this will be controversial......

Thanks for everyone's participation.

Andy

At the end the benefits of the Audit come by the quality and knowledge of the Auditor.

Very sorry...You're forgiven - not your fault. I perceived your comment as an accusation rather than a question and well, it's a long story....I've been "on-the-edge" of going absolutely ballistic at work. False accusations, stress, and death-survivor guilt feelings? I finally beat the living sh%t outta my punching bag last night.:D I feel better today.

Yes, I thought and management thought we have had very good productive audits and learned from them but "measureable"...I just cannot see that they have been measureable ($$ or improvements, or ?? ) There haven't been any trends because I keep track of those things. Did someone define "measureable"? Maybe I am off-base?


OK, good. As to defining measureables, are there tangible benefits you can point to that came out of certain audits, that have demonstratible benefits? That might be a good place to start. Also, continual improvement projects and six sigma projects generally include measuring results. Can you follow that same technique?

Out of curiousity, Andy, what changes in approach would you suggest? I'd be interested in what angles you are considering.

OK, good. As to defining measureables, are there tangible benefits you can point to that came out of certain audits, that have demonstratible benefits? That might be a good place to start. Also, continual improvement projects and six sigma projects generally include measuring results. Can you follow that same technique?

I suppose I could go back and find something from an audit but measureables predominantly are identified in continual improvement ideas not audits. No six sigma here.

Out of curiousity, Andy, what changes in approach would you suggest? I'd be interested in what angles you are considering.

There's not enough space here to list them all, 'H'...............:lmao: But just to get started...........

Planning, 'checklist auditing', reporting, corrective action vs correction, the auditor's role in fixing systemmic issues............:mg:

As I've said in this post and others, the 'model' is out of date - we use an external audit process - auditors turn up, have an opening meeting, snoop around until they find something to record, write it up against a spec no-one (except maybe the quality manager) has even read, have a closing meeting and then leave a bunch of items behind which need a corrective action and say "Thanks, enjoyed it"........... - as viewed by most management!

And, although we have people posting here about what a 'sharing experience it was' (who'd be brave enough to say anything else?), my guess is if you could get inside most senior/top managers heads and see what the really think it wouldn't be so wonderful.

Now, I'm not advocating that 'management don't get it'. They won't and we can't expect them to, but I believe that (behaviorally) if internal auditors did more to move the audit process towards issues that affect the business (instead of 'ISO'), gradually there'd be more support, just like 6 Sigma can achieve...........:yes:

Andy

There's not enough space here to list them all, 'H'...............:lmao: But just to get started...........

Planning, 'checklist auditing', reporting, corrective action vs correction, the auditor's role in fixing systemmic issues............:mg:

As I've said in this post and others, the 'model' is out of date - we use an external audit process - auditors turn up, have an opening meeting, snoop around until they find something to record, write it up against a spec no-one (except maybe the quality manager) has even read, have a closing meeting and then leave a bunch of items behind which need a corrective action and say "Thanks, enjoyed it"........... - as viewed by most management!

And, although we have people posting here about what a 'sharing experience it was' (who'd be brave enough to say anything else?), my guess is if you could get inside most senior/top managers heads and see what the really think it wouldn't be so wonderful.

Now, I'm not advocating that 'management don't get it'. They won't and we can't expect them to, but I believe that (behaviorally) if internal auditors did more to move the audit process towards issues that affect the business (instead of 'ISO'), gradually there'd be more support, just like 6 Sigma can achieve...........:yes:

Andy

I think I agree with your premise (that alone may cause me a bit of concern...:notme: ). I think similar but different would be better. Maybe more tailored to the needs of the organization would be a better way to phrase it. I certainly think the planning can take it into a tailored view without violating the requirements.

Maybe the trainers should tailor their trainings to internal audits, rather than just lazily using the same Powerpoint. I know I do. My training takes a significantly different approach and sequence.

Of course, I still think many of my clients just "love" my audits, ...cuz they say so at the closing meeting...but allow me to enjoy my delusion. (At least I think they generally agree the things we delved into were more beneficial than the average audit.)

I think the planning should start the same as planning any other process. What are the purposes of this process, why do we want to do audits in the first place (Note: "to meet the requirements" is NOT a good answer, guys!). If the objectives, criteria for effectiveness and purposes are defined, then planning should become much easier and more focused.

based on the numbers to date (38 votes, 19 for 'value added audits') I'd say there's a heap of room to improve the way internal audits are being done. I could have asked folks what value ($$$$ only count in this case) their management derived from auditing, but if auditing is not being supported or it'd be dropped if not for 'ISO', (that's 50% of the votes) my feelings about it are still confirmed.........

Time to get the pencil out and come up with a different scheme..............

Andy

I would consider compliance auditing to be very important. Operator instructions should be followed! That is not the same as saying the operator instructions are correct. If they are incorrect, fix them. Do you want to put a new operator in a position, then have them follow the incorrect instructions?

The one thing that bugs me about internal auditing is that you have to audit someone else. I think more emphasis should be placed on auditing your own department. You have a much better idea how your department should work and can more quickly identify if something is done incorrectly. The emphasis on this type of auditing is up to internal management. It goes without saying that you are responsible for your own area. However, if time contraints hit you, what will be the first to go?

The one thing that bugs me about internal auditing is that you have to audit someone else. I think more emphasis should be placed on auditing your own department. You have a much better idea how your department should work and can more quickly identify if something is done incorrectly. The emphasis on this type of auditing is up to internal management. It goes without saying that you are responsible for your own area. However, if time contraints hit you, what will be the first to go?

But, auditing your own department won't give you the opportunity to see the Business as a whole, would it?

An Internal Auditor in my opinion, should be well versed in the QMS/EMS that they are either "Certified or Compliant (your choice)" to, and understand the Business as a whole not as a separate department or area.

Yes, usually when there are time constraints, for whatever reason, usually the Internal Audit program suffers, but it shouldn't. My opinion is that the Internal Audit program should take a higher priority in any business.

Please note that IRCA's knowledge bank has some interesting papers on the subject of internal audits and related:
Internal audit:

Internal audit revolution (http://www.irca.org/inform/issue5/Peddle_Rosam.htm), Ian Rosam & Rob Peddle, 2005
Internal auditing (http://www.irca.org/inform/issue3/JWade.htm), Jim Wade, 2004Other documents:

Capability of personnel (http://www.irca.org/inform/issue6/DPowley.htm), Dave Powley, 2005
Review of audit in 2005 (http://www.irca.org/inform/issue9/NThorpe.htm), Nick Thorpe, 2005
Become a better auditor (http://www.irca.org/inform/issue6/jsmith.htm), James Smith, 2005
Surviving an audit (http://www.irca.org/inform/issue3/IWaples.htm), Ivan Waples, 2004
The quantum audit (http://www.irca.org/inform/issue1/timohanlon.htm), Ceri Davies & Dean Lucas & Tim O'Hanlon, 2003

...
The one thing that bugs me about internal auditing is that you have to audit someone else. I think more emphasis should be placed on auditing your own department. You have a much better idea how your department should work and can more quickly identify if something is done incorrectly. The emphasis on this type of auditing is up to internal management. It goes without saying that you are responsible for your own area. However, if time contraints hit you, what will be the first to go?

I would suggest you can and should audit your own area ALL the time. But, auditing your own work does not qualify for an independent review by a disinterested auditor. That should occur also. But, too often I have seen quickie audits done by someone who audits their own area, and they skip over a lot of stuff. If they see a problem, they should deal with it on their regular job and not wait for an audit.

Interesting discussion. I have to audit or 'train and assign' auditors for a client. I am primarily responsible for content of quality manual. I audited it this month, per schedule (even tho it was my responsibility to publish) and found 2 N/C's. The appendix had a wrong revision date, and the management 'process measures' referred to a metric no longer used. I just felt I was more 'qualified' than the other folks that had been previously used.

I'm not sure the other folks that were trained as internal auditors in a 2 day class, would have picked up on the N/C'. I guess when I audit my own work, I put on my '3rd party hat.'

I understand the independence factor, but at the same time feel that I know better what to look for than someone sent to a 2-day class. I would like mgmt to select others, and I think I've made some in-roads there, but still feel like me auditing 'my own work' is better than someone with minimal training.

Thoughts?

Interesting discussion. I have to audit or 'train and assign' auditors for a client. I am primarily responsible for content of quality manual. I audited it this month, per schedule (even tho it was my responsibility to publish) and found 2 N/C's. The appendix had a wrong revision date, and the management 'process measures' referred to a metric no longer used. I just felt I was more 'qualified' than the other folks that had been previously used.

I'm not sure the other folks that were trained as internal auditors in a 2 day class, would have picked up on the N/C'. I guess when I audit my own work, I put on my '3rd party hat.'

I understand the independence factor, but at the same time feel that I know better what to look for than someone sent to a 2-day class. I would like mgmt to select others, and I think I've made some in-roads there, but still feel like me auditing 'my own work' is better than someone with minimal training.

Thoughts?

You can review your own work 365 days per year. In fact, you are encouraged to do so.

But, when it comes to performing an internal audit, that must be performed by an independent set of eyes. Otherwise, you are doing your own work twice, and it is never getting audited by an independent set of eyes.

You can review your own work 365 days per year. In fact, you are encouraged to do so.

But, when it comes to performing an internal audit, that must be performed by an independent set of eyes. Otherwise, you are doing your own work twice, and it is never getting audited by an independent set of eyes.


Good point. I just think if some of the folks that they sent 'to school' would not have found the same things I did. The other option is another outside auditor the way I see it - just to audit the stuff they want me to do.

While there's no argument about the requirement for independent auditing, I've always thought that it's a bad idea to base everything on the premise that people can't be trusted to be objective about their own work. I do think that independent eyes can be useful, but I also think that it calling upon them should be my prerogative. The proof, after all, is in the pudding.

I tend to agree. For example, I am the 'consultant' Quality Programs Manager/Internal Auditor for a company. Their procedure (that I wrote) calls for an annual review/audit of the Quality Manual (that I wrote.) The requirement that responsibilities, references to procedures, etc are still accurate. I found 2 N/C's. 1 - a reference to a position that has been eliminated and another reference to a metric they no longer track. The other 2 folks are very capable of 'floor audits' - checking job instructions, calibration equipments, etc, but I wonder if they would have found the 2 things I found in my own work.

I know, I know - training - yes they could be further trained. However, this would be the 5th and 6th person trained in the 7 years I've worked there because of turnover. Another reason they are more comfortable with me and my experience performing the task.

I found 2 N/C's. 1 - a reference to a position that has been eliminated and another reference to a metric they no longer track. The other 2 folks are very capable of 'floor audits' - checking job instructions, calibration equipments, etc, but I wonder if they would have found the 2 things I found in my own work.


I wonder if anyone needed to. Those sound like harmless sins of omission, and probably had no affect, adverse or otherwise, on the work being done.

Maybe - the title reference wasn't a big deal - in fact I think I just called it an Observation. The other was a 'process measure' tho - and an alternate hadn't been put in place, so we needed to have some discusion on that one. I remembered after I posted that there was a 3rd issue with the revision date in the Appendix didn't match the footer (again - my responsibility).

I think the requirement for being independent is two fold - someone wouldn't 'tell' if it was there own work, and that something is overlooked because they are so used to doing something the same way.

I think the requirement for being independent is two fold - someone wouldn't 'tell' if it was there own work, and that something is overlooked because they are so used to doing something the same way.

If the former is true, then there's a significant problem that won't be helped by auditing. As for the latter, I do agree that independent eyes can be helpful, but I'd like to think that a culture can be established where people want to help one another, and want to be helped. In such a case, there's no need for a requirement for independent auditors.

How can the benefit(s) be ascertained if we don't know why we are doing the audits.

So, why are internal audit required?

Yes, because ISO says so! But, were companies doing internal audits prior to becoming certified?

When I mean "company" I am referring to an "average" manufacturer. Opinions might vary as what that means, but there are about 600,000 manufacturers in the U.S. 75% of these have 20 or less employees. You pick the industrial park, ask all the non-certified companies if they do internal audits.

I'll buy you a beer for all the "yes" responses, and you still would not need a designated driver. So, do internal audits make business sense if those actually making profit/products aren't doing them?

Maybe, just maybe, internal audits are required because they are a surrogate for external audits. If so, then they exist solely to find potential non-conformances

(I have hung out with professional internal/compliance auditors from the SOx world and I don't think that is a valid viewpoint regarding ISO's requirement for implementing internal auditing. The "average" SOx-type I know works for a company of 4,000 employees, and these companies were doing int audits long ago because of the managerial/bureacracy levels that separate top management from ground floor. No SOx-type I've ever met has been in a manufacturing facility.)

Yes, because ISO says so! But, were companies doing internal audits prior to becoming certified?

When I mean "company" I am referring to an "average" manufacturer. Opinions might vary as what that means, but there are about 600,000 manufacturers in the U.S. 75% of these have 20 or less employees. You pick the industrial park, ask all the non-certified companies if they do internal audits.

I'll buy you a beer for all the "yes" responses, and you still would not need a designated driver. So, do internal audits make business sense if those actually making profit/products aren't doing them?
Will you still buy me a beer if we change the question slightly? What if we ask, "Do you periodically verify that your system is working as intended?" Successful companies have always done internal audits, but they never called them that until they had to. You've put your finger on the problem; internal auditing (as it should be practiced) is nothing more than ongoing verification that the system is efficacious. It doesn't matter what you call the process.

If the former is true, then there's a significant problem that won't be helped by auditing. As for the latter, I do agree that independent eyes can be helpful, but I'd like to think that a culture can be established where people want to help one another, and want to be helped. In such a case, there's no need for a requirement for independent auditors.This is of course true except for two dynamics of the same essential problem. Even in the healthiest of cultures I feel these dynamics could impact performance and are worth circumventing with an outsider's look.

1. The more familiar one is with a process, the greater the risk of either alpha or beta errors.

2. The more familiar a person is with a process, the more or less tolerant one would be of issues that, if one added them up, could amount to death by a thousand cuts.

This won't always be the case, of course. It's all variable along with personalities. The case for independent auditing could be supported with the argument of objectivity and a "little boy at the parade" view of things an insider deems normal.

Dear Mr. Wynne:
Of course I'm not going to let you change the question! (Were you my ex's attorney? Nice try!)
I said internal audits, not "do you know how you're business is doing?"
(Beers only valid for said question, asked as written!)

The Madfox

Upon further review I am willing to compromise and allow this question. (And the beer is now limited to PBR.):
Dear Business Owner:
Do you use employees to present you with a report on "how's your business going?"
The textbook "Millionaire Next Door" answer should be, "Our customers tell us how we're doing!"

For those who haven't visited this poll for a while, it's interesting (well, to me anyway) that the results favor the position that organizations don't get benefit from internal audits - which is much as I had suspected.:notme:

Has anyone who polled on the 'benefits gained' care to share what their experience has been and how the benefits were derived?

Andy

that the results favor the position that organizations don't get benefit from internal audits - which is much as I had suspected.:notme: You must be applying for a job as a "spin master" for a political party:mg: . The poll results do not show that.

The poll results do not show that.

Based on the numbers and the my assertion that if management don't make them a priority - audits are not bringing a real benefit, I'd say that there's more than 50% don't get anything back for doing them.......:yes:

What's your take, Sydney?

Andy

What's your take, Sydney?

AndyVery simple: 45 out of 65 respondents answered yes to the question "Does your organization really benefit from internal audits?".

Personally, I don't think that this poll results represent the universe of organizations performing internal audits. I believe, just like you, that most internal audits done in order to comply with a standard the organization subscribes to, are wasted efforts.

But the poll results don't support our theory.

Based on the numbers and the my assertion that if management don't make them a priority - audits are not bringing a real benefit, I'd say that there's more than 50% don't get anything back for doing them.......:yes:

What's your take, Sydney?

Andy

I'm not her him :biglaugh:, but it's Sidney. What you're saying, then, is that the 16 people who said they were a benefit but not given enough management attention are wrong?

(Oops. Thought I was at the end of the thread there and both my questions were answered when I realized I wasn't and kept reading.)

Instead I'll say I voted yes that we get benefits but that top management doesn't make it a priority. Top management seems to not make anything but profit a priority sometimes.

I have mixed feelings with internal audits. On one hand internal audits are necessary to maintain compliance. Management personnel/structure changes that can cause system to fall apart, business crisis happens that causes the attention/commitment to evaporate. Audits are necessary to ensure compliance over time.

For me, the jury is still out on if internal audits are an effective improvement tool. For me, most of the improvements I have made to product, processes, systems, etc., happens outside the internal audit system. I know the weaknesses in our system without an internal audit report telling me so. Knowing these weaknesses, I try to be proactive in fixing them up rather than waiting for the audit and corrective action processes to kick in. I know someone will point out that sometimes another set of eyes will see something that I don't, but again most of the time this happens outside the audit process.

A couple of different approaches I have been considering using internal audits if the standard was different:

1) Gap analysis. Rather than an audit, require a gap analysis periodically to review compliance and potential system improvements. With this take the independence rule and training requirements out. I as a QA Mgr have a vested interest in making sure my system maintains compliance with ISO standards so I am not going to do anything that will jeopardize our certification. If improvement is truly the goal, than me covering anything isn't going to get the results.

2) Training programs. When you pick someone out of the crowd to mentor them because you see something in them that shows potential, you don't just put them in a position, give them basic instructions, and never ever talk to them ever again. A good mentor will follow-up and give feedback, again and again. My question is how do you do this with the masses regarding training the whole bunch. Training someone on a one time instance and never following up plainly is not effective. Can the audit system be changed into something that can benefit the training program?

Also related to training programs and continual improvement is, use audits as a control method for ongoing evaluation. When corrective actions are taken to fix problems, the problem is solved short term, but quickly falls apart because people go back to old habits. My thought here is use audits to break bad habits. I have learned that someone can be trained in a day on how to do something, but it can take up to 6-12 months before it becomes "the way we have always done it".

:topic: I'm not her him , but it's Sidney.

Hey, Sidney, have you seen Office Space, where the guy named Michael Bolton had to constantly defend that "No, I'm not related to the singer guy"!:mad: Sorry that we mis-assume your gender; I have done it more than once. Just don't crack and go destroy the fax machine. :bonk:

Back to our hero….

In my humble opinion, the Internal Auditing Program (IAP) becomes an extension of the quality program as a whole. Saying, if the upper management finds value in the quality program, they will find value in the IAP. If it’s just a piece of paper, the IAP will be a puppet show.

Also, the success of the IAP depends on the experience/qualifications of the Internal Auditors. If the auditors are professional, objective, and stick to their scope, their job will be perceived as being less of a witch hunt, and may facilitate more understanding.

As far as value to the organization: Is the quality program value to the organization? If “yes”, then the IAP will accomplish something.

OK.....let's revisit the title.....

Yes, the organization CAN benefit from an internal audit.....

If you are doing it just for the registration body or the accrediting body, then STOP and take the corrective action.....

BUT.....

An internal audit is like a physical exam during a check-up.....it is a tool, not a be-all-to-end-all.....a tool to monitor the organization.....approached fro that angle, it can be a good thing.

An internal audit is like a physical exam during a check-up.....it is a tool, not a be-all-to-end-all.....a tool to monitor the organization.....approached fro that angle, it can be a good thing.

I like your analogy:

1. If you are a healthy company, a physical/checkup would show that there's nothing to worry. It gives you 'peace of mind' and enable you to concentrate on doing what you know best.

2. If you are a dysfunctional or sick company, it will reveal a host of problems that you need to take care. Failure to do so may end up with disaster. It could also mean that you need further check up by specialist.

3. If you engaged 'quacks' who failed to diagnose or identify problems, that could also be disastrous.

Another angle to look at benefits of internal audits.

Very good! Can I use that when I train on internal audits! Or better yet, to sell upper management on the need?

I think internal audits are crucial only for the first couple of years. If your business goals and objectives do not change, internal audits become very redundant and useless. They are, however, extremely valuable at the onset.

:nope:

I disagree. Depending on how internal audits are conducted they can also be used to drive improvement. Plus if there is any turnover at all, ensuring procedures and processes remain in place. Not sure if you mean discontinue all together, or reduce frequency, but I think they can continue to be valuable.

I think internal audits are crucial only for the first couple of years. If your business goals and objectives do not change, internal audits become very redundant and useless. They are, however, extremely valuable at the onset.While business goals and objectives tend to have a longer life cycle, most organizations that I work with have on-going process changes aiming at continual improvement, efficiency gains, etc...Internal auditing, if adequately planned and executed can provide invaluable feedback about the deployment and stability of these on-going changes.
Internal audits that keep re-checking the same "high level" issues and are perceived as non value added are normally disconnected from the processes that they are supposedly assessing. That typically happens when the organization uses a standard-based checklist, instead of doing a true process based assessment.

If your business goals and objectives do not change

You've got a bigger problem than doing boring audits.................:mg:

Andy

You've got a bigger problem than doing boring audits.................:mg:

AndyMaybe they sell corks for the wine industry.....:notme:

Yeah,we can find a number of nonconformances and then correct them.
The most important point is that our manager pays a great attention to it.

Since no one manages the broad horizontal processes in our company we are one of the only groups that has a formalized method of improvement by doing process audits. We are actually requested to do audits by mid level managers to help bridge gaps between functional areas with different (vertical) chains of command as there are no process owners or corporate governence from a business standpoint that spans the different functional departments. Without internal audits we would be crashing and burning much faster than we are now! :lol:

Hi,

Sorry for writing this thread. I'm new in forum and can't find where to write my question.

I'm working as an internal audit in a major bank in türkiye.

I'm translating an audit program about Interest Risk Rate.

But I can't understand the term "LNB" in the blove sentence. Does anybody help me about what the term LNB represents for in the sentence belove

"Does LNB utilize off-balance sheet derivatives to manage its IRR?"

Thanks a lot

Internal audits-

Simply said- you get what you give.

If you have qualified auditors who put the effort into an internal audit program you will see valuable results.

Hi,

Sorry for writing this thread. I'm new in forum and can't find where to write my question.

I'm working as an internal audit in a major bank in türkiye.

I'm translating an audit program about Interest Risk Rate.

But I can't understand the term "LNB" in the blove sentence. Does anybody help me about what the term LNB represents for in the sentence belove

"Does LNB utilize off-balance sheet derivatives to manage its IRR?"

Thanks a lot

Hello, there! Welcome to the Cove!

As far as LNB, I googled and could not come up with anything useful. Is your sentence from a checklist?

Could LNB be initials for one of the departments/units you would be auditing?

Just from the sentence, I am surmising they're interested in "creative" methods of calculating Internal Rate of Return, specific for each unit.

Does any of that sound relevant/helpful?

Hi,

Sorry for writing this thread. I'm new in forum and can't find where to write my question.

I'm working as an internal audit in a major bank in türkiye.

I'm translating an audit program about Interest Risk Rate.

But I can't understand the term "LNB" in the blove sentence. Does anybody help me about what the term LNB represents for in the sentence belove

"Does LNB utilize off-balance sheet derivatives to manage its IRR?"

Thanks a lot

Could it be some kind of indices provided by 'LNB Bancorp' (US)? Why not check with some old hands around your working area. It certainly is an industry specific term.

Unfortunately, there aren't many bankers around here.

Interesting article / debate about internal auditing :

http://www.irca.org/inform/issue14/WadeOak.html

Internal audits-

Simply said- you get what you give.

If you have qualified auditors who put the effort into an internal audit program you will see valuable results.

If i am allowed to quote myself:notme:

Interesting article / debate about internal auditing :

http://www.irca.org/inform/issue14/WadeOak.html


Good article. The anti view is realistic, but old view. Jim Wade's view is much more progressive and beneficial.

Anything that is not performed well, is not likely to provide great value. So do we stop doing the activity, or learn how to do it to good benefit.

I think the problem is we need to revise how we train internal auditors. Internal auditors should not be taught the same program as external auditors. It should be a different approach. In that regard, AndyN is correct.

Yeah,we can find a number of nonconformances and then correct them.
The most important point is that our manager pays a great attention to it.


Can your auditors also identify improvements and preventive actions? They should not limit their review to nonconformances.

We note most of these items in every report for each audit area:
general comments
observations
strengths
weaknesses
opportunities for improvement
nonconforming conditions
continuous improvement
preventive actions

To this day, the audits are still welcomed and thought of as beneficial to the company. The reports are read by all involved and upper management.

I remember when I first became a Quality Consultant in 1986 with my first Customer being a distributor of children's goods and I had an automotive quality background.

I put in a 2nd tier "integrated" automotive system in this company and it worked very well. We had control of our suppliers, had sampling plans and it was all integrated into the present work force.

Except - I did not have an internal audit system.

Well, I was called back 4 years later. The company had hired someone in Quality that I met but it got rather fuzzy on what the person's duties were.

I reviewed their quality system that I developed. Everything was gone with some systems taking 2 years to eventually stop. I noticed that full 2 way traceability system stopped one day and no one noticed. That was the end of that system. That system was imperative for such items as children's strollers, etc. - GONE!!!!

I was wrong!! If I had developed an internal audit system, the system would have been maintained.

I know we develop systems for compliance but a solid internal audit system is for your company, not just for compliance.

I'm against working on something strictly for compliance reasons. Kinda up there with fast food. But I'm digressing.

I too have worked in various industries yet I come from the automotive manufacturing sector. Lots of good tools there. One favorite of mine - a simple customized tool good for anywhere is what I call an LLA - Light Layered Audit. Very much like the Layered Process Audit sometimes used in the auto industry but if customized for your specific company it can be a useful tool because ...it involves people - at many levels
it drives communications
it produces results beyond 'simple compliance'
it maintains itself
blah, blah blah....

When all levels adhere to it, the continuous improvement side just soars!

BUT like any audit, if it's not supported by management by reporting and supporting appropriate corrective actions and proper root cause analysis (3L5Y is best! :cool: ) then it's a fruitless activity that may not even be compliant at best.


Forgive me if this has alrady been posted but I couldn't read every post - only about five pages of them.

Oldrice-:cool:

I've done internal and external auditing and really don't see much difference of substance in the methodology.

As far as they are benficial?
That's all up to upper management, isn't it?
If they take them seriously then they will be beneficial. If they see them as a necessary evil, not so much. If they seem them as a non-priority then not at all.

So maybe the question would better phrased as "does your management support an audit program"?

The best audit process in the world will be ineffective if it's not supported.

I plan internal quality audits, conducts them by myself. .... but after doing the same departments over & over again is not bringing the results.
Middle management thinks it is a neccessary evil while the upper management supports it.
We are something in the middle of beneficial to non priority to evil. ;)

Very simple: 45 out of 65 respondents answered yes to the question "Does your organization really benefit from internal audits?".

Personally, I don't think that this poll results represent the universe of organizations performing internal audits. I believe, just like you, that most internal audits done in order to comply with a standard the organization subscribes to, are wasted efforts.

But the poll results don't support our theory.

:applause:

Although there probably are some people here who believe that an internal audit is beneficial not all do. I personally think they are a waste of time.

Some will say, well if you aren't finding non-conformances but they are being found during an external audit then you aren't doing a good job as an internal audit. I say then the external auditor is being too darn picky about things. Only you, the people who work at your company every day, know how your business runs and what makes it run smoothly, properly, and successfully. External auditors only know how they feel it should be changed so they understand your business.

I think internal audits run in the style of ISO9001 external audits, primarily for the benefit of ISO9001 certification are a waste of time, however I do think the concept of auditing is good and it can be a useful tool.

I imagine the poll result is skewed in favour of audits because there are lots of ISO9001 people on this forum !

Ask 100 CEO's and see what answer you get (if they are honest CEO's) :D

Can your auditors also identify improvements and preventive actions? They should not limit their review to nonconformances.

In my audit reports I note:
Observations
Nonconformances
Opportunities for improvement

Since the nonconformances are the only items that have to be addressed (in either internal or external audits) they are the only ones that are addressed. :frust::mad::o

How to turn this around? :confused::thanx:

In my audit reports I note:
Observations
Nonconformances
Opportunities for improvement

Since the nonconformances are the only items that have to be addressed (in either internal or external audits) they are the only ones that are addressed. :frust::mad::o

How to turn this around? :confused::thanx:

Hello Confused1,

What you should add to your list is Potential Nonconformities (Risks).
Those are prime candidates for Preventive Action.

If you can identify just one potential (risk) nonconformity as a result of an internal audit, and you can quantify (to a certain extent) in $$$$ what your company is about to lose, or exposed to, you create quite a number of fans...;)

I always encourage the auditors that I work with to give their eyes and ears a good workout. That requires them to look way beyond the customary list of audit questions and really put the system and its processes through the test. Talking about a great opportunity to add value.... Wow! :yes:

Stijloor.

Hello Confused1,

What you should add to your list is Potential Nonconformities (Risks).
Those are prime candidates for Preventive Action.

In an effort to make audits more value added I have started (or attempted) to take a risk management approach to auditing. But I haven't got much of a handle on risk management principals or how to apply them to my workplace. (which is a distribution center for replacement parts). I have also begun to educate myself to do FEMAS.

One of my more enlightened clients (in the past) used 'Risk' and 'Impact' statements instead of the more compliance oriented non-conformance statements. When written, it made the issue very clear to management what was happening and, therefore, the action necessary. As a result, they were behind corrective actions all the way......as well as the audits!
:2cents:

In an effort to make audits more value added I have started (or attempted) to take a risk management approach to auditing. But I haven't got much of a handle on risk management principals or how to apply them to my workplace. (which is a distribution center for replacement parts). I have also begun to educate myself to do FEMAS.

Hello Confused1,

If you work in a distribution center, I assume that one of the major risks is that the wrong parts go to the wrong location. I just use this as an example.

When you audit, the idea is to look at all provisions that exist in the process that prevent that from happening. Now, if you look closer at these provisions (controls, instructions, equipment, etc.) are there any weaknesses present that could possibly create problems for you and your customer?

See, that's the idea. Too often, audits are focused on what's already wrong (reactive) as opposed to looking at what conditions exist that could possibly go wrong (preventive).

Your desire to learn about FMEA is great. Lots of info here about FMEA.

A personal comment.

Auditing is a very loaded and emotional topic here at The Cove Forums. Just like religion and politics, people have extreme stong feelings about it. That's OK as long as we're willing to look at all sides of the issue. Maybe we can learn a few things.

Good luck!

Stijloor.

Hello Confused1,

What you should add to your list is Potential Nonconformities (Risks).
Those are prime candidates for Preventive Action.

Stijloor.


In an effort to add value to internal auditing ( I have been doing compliance auditing for a while and feel it is of limited value as it's a go-no-go guage) I have wondered if perhaps I should take a risk management approach.

I work in a distribution center for replacement parts (all we do is purchase, recieve, pack, stock, pick and ship).

I kinda wonder if risk management would be like using a sledge hammer on the head of a pin. -also have to wonder about management's perception.

All management wants is to get the $$$ out the door. :magic:

So we get a lot of returns - mabey that would be a good place to start.

Thoughts/ comments, please.

confused1

In an effort to add value to internal auditing ( I have been doing compliance auditing for a while and feel it is of limited value as it's a go-no-go guage) I have wondered if perhaps I should take a risk management approach.

I work in a distribution center for replacement parts (all we do is purchase, recieve, pack, stock, pick and ship).

I kinda wonder if risk management would be like using a sledge hammer on the head of a pin. -also have to wonder about management's perception.

All management wants is to get the $$$ out the door. :magic:

So we get a lot of returns - mabey that would be a good place to start.

Thoughts/ comments, please.

confused1

That would be an excellent place to start because you have disappointed the Customer.

Customer returns are indicative of a system that has failed. Here is where serious corrective action is necessary. Once corrective actions (usually process changes) have been implemented, internal audits could be used to verify that the actions remain in place (old habits die hard) and that the modified methods are complied with.

To make a case, and to obtain Management's attention, possibly their commitment; you may want to calculate $$$$$$ wasted. That's the only language managers understand.

Let's continue the dialogue. Weekends are usually quiet here. I am sure others will chime in at some time. Trust me, it's never boring here...;)

Stijloor.

That would be an excellent place to start because you have disappointed the Customer.

Customer returns are indicative of a system that has failed. Here is where serious corrective action is necessary. Once corrective actions (usually process changes) have been implemented, internal audits could be used to verify that the actions remain in place (old habits die hard) and that the modified methods are complied with.

To make a case, and to obtain Management's attention, possibly their commitment; you may want to calculate $$$$$$ wasted. That's the only language managers understand.

Let's continue the dialogue. Weekends are usually quiet here. I am sure others will chime in at some time. Trust me, it's never boring here...;)

Stijloor.

Plant manager once told us that we wasted a lot of $$$$ in '06 because we can't count or read part #s.

We have stayed the course -'07 was no differnt!
Old habits may die hard but bad habits seem to get an extra life.

Just thinking out loud:
How to calculate the cost of returns - Cost of product + cost of 2 way shipping + ??

How do you put a value on:
Customer good will
Damage to reputation
Goods not returned (under a certain $ amt. we don't ask customer to return stuff)

Then I can apply the same to customer complaints - or can I? Not all customer complaints result in a return ,and not all returns are complaints.
I love my job.

Plant manager once told us that we wasted a lot of $$$$ in '06 because we can't count or read part #s.

We have stayed the course -'07 was no differnt!
Old habits may die hard but bad habits seem to get an extra life.

Just thinking out loud:
How to calculate the cost of returns - Cost of product + cost of 2 way shipping + ??

How do you put a value on:
Customer good will
Damage to reputation
Goods not returned (under a certain $ amt. we don't ask customer to return stuff)

Then I can apply the same to customer complaints - or can I? Not all customer complaints result in a return, and not all returns are complaints.
I love my job.

There are always certain elements that are difficult to put a price tag on. I would certainly start with the knowable expenses such as the cost of nonreturnable product, shipping charges, labor involved, processing paperwork, etc.

Loss of goodwill and damage to reputation are difficult to measure in terms of dollars even though you know that it has a monetary effect. A best estimate is better than nothing.

There are some interesting threads about the Cost of Quality (http://www.google.com/custom?domains=Elsmar.com&q=Cost+of+Quality&sa=Search&sitesearch=Elsmar.com&client=pub-1385417534940691&forid=1&channel=6124086287&ie=ISO-8859-1&oe=ISO-8859-1&cof=GALT%3A%23008000%3BGL%3A1%3BDIV%3A%23336699%3BVLC%3A663399%3BAH%3Acenter%3BBGC%3AFFFFFF%3BLBGC%3A000099%3BALC%3A000000%3BLC%3A000000%3BT%3A0000FF%3BGFNT%3A0000FF%3BGIMP%3A0000FF%3BLH%3A50%3BLW%3A350%3BL%3Ahttp%3A%2F%2Felsmar.com%2Fpng%2Fheader-G-search.png%3BS%3Ahttp%3A%2F%2FElsmar.com%2FForums%2F%3BFORID%3A1%3B&hl=en) here at The Cove. You may want to check that out.

As far as customer complaints are concerned, you can put a dollar figure on those as well. Again based on knowable figures and best estimates. As long as the dollar figures are used for good purpose and not to beat people over the head, they can be used as a decent measurements. Those efforts should never be used as punitive measures. Trust me, people are very smart and they know numbers too.

Stijloor.

P.S.

Allow me to add a few quotes from Dr. W. Edwards Deming, one of which he attributed to Dr. Lloyd Nelson.

"The most important figures that one needs for management are unknown or unknowable (Lloyd S. Nelson, director of statistical methods for the Nashua corporation), but successful management must nevertheless take account of them."

Deming realized that many important things that must be managed couldn’t be measured. Both points are important. One, not everything of importance to management can be measured. And two, you must still manage those important things. Spend $20,000 training 10 people in a special skill. What's the benefit? "You'll never know," answered Deming. "You'll never be able to measure it. Why did you do it? Because you believed it would pay off. Theory." Dr. Deming is often incorrectly quoted as saying, "You can't manage what you can't measure." In fact, he stated that one of the seven deadly diseases of management is running a company on visible figures alone.

I have done/organized internal audits (When I was an MR) and have also done 3rd party audits on behalf of certifying bodies. The question of an organization benefiting (or not) by internal audit puzzles me. Going through the comments above I get the following impressions (please correct me if I am wrong)

1. Internal audits are done merely to satisfy the requirements of ISO 9001 (or other standards) not necessarily to verify to what extent QMS (or EMS) is being complied.
2. Internal auditors may not perhaps be trained enough to be able to distinguish between the essentials & superficials.

3. As a result of 1) and 2) above the job may not be taken up in all seriousness. This is more so since the auditors and auditees may know each other fairly well due to their day to day contacts.

4. Managements as a rule - with due apologies to the honorable exceptions - are not supportive of the whole exercise.

Having said that I must also mention that a good many consultants I have come across do not study adequately the systems and procedures that may already be in place (written or otherwise) prior to introduction of QMS due to their lack of knowledge of the industry/business - sadly due to lack of their own understanding of the requirements. I have come across people consulting without any worthwhile background in industry or business or quality! The idea is to get certification and fast and this reduces the internal audits - once in 3, 6 0r 12 months to a mere formality.

The point I am making is - if a system has been set up that satisfies the requirements of the Standard and earns the appreciation of the management for its usefulness, there is every chance of management encouraging internal audits for the feedback it would provide them for further action.

:thanx:

1. Internal audits are done merely to satisfy the requirements of ISO 9001 (or other standards) not necessarily to verify to what extent QMS (or EMS) is being complied.



Apart from Audting Standard requirement, you can add Best and Bad practices seen across the Processes.

Will definitly add value to the Audit

sathis

Thanks Sathis. This is a good idea.

Jupitor:
agree:

In an effort to add value to internal auditing ( I have been doing compliance auditing for a while and feel it is of limited value as it's a go-no-go guage) I have wondered if perhaps I should take a risk management approach.

I work in a distribution center for replacement parts (all we do is purchase, recieve, pack, stock, pick and ship).

I kinda wonder if risk management would be like using a sledge hammer on the head of a pin. -also have to wonder about management's perception.

All management wants is to get the $$$ out the door. :magic:

So we get a lot of returns - mabey that would be a good place to start.

Thoughts/ comments, please.

confused1


If a distribution center is getting a lot of complaints and returns, there truly is a problem.

Value-added or Improvement Auditing is a much better approach to internal audits, but might be difficult if management is hostile toward the program. If not hostile, it might work.

(I do a training program on Auditing for Improvement which has had very good results, if you are interested.)

Absolutely, our internal audits continue to find the gaps in our systems (mostly document control and follow-through), but we have gotten some of the greater benefites from use of layered process audits. Those keep attention on items daily and seem to get more management attention.

My concern thought with Layered Audits (feedback from everyone, please), is how to keep them fresh. After awhile when things are "good" they start to become a bit of a pencil whip exercise. The audits should be dynamic and focus on customer issues and high RPNs from the FMEA as well as daily maintenance and 5S items, but as existing problems begin to be resolved and verified, the audits lose their shine and peolple get bored with them.

Any ideas would be appreciated!
Thanks!

Does An Organization Really Benefit from Internal Audits?

It surely does!!!... as others explained...

Time for a change?

...well, keeping them “fresh” is really one of the toughest task to maintain…actually, this is really the undesirable side of internal audits… everyone gets tired… and everyone seems not getting serious… until it become “for the sake of documentation” rather than “for the sake of improvement”…

…and I’ve seen so many strategies to make them keep them flowing… still the best and the most effective way is when the top management get involve… even the most stubborn and uncooperative manager will have no option or excuse but to comply… no matter how boring it is...

…afterall as they say, no project has become successful without the “true” support from the top management…

(I added the word “true” to denote serious commitment from the top management)

Absolutely...our organization benefit from Internal Audits...because there are several observations that can be transformed for the additional improvement for our company.
raffy:cool:

This is all very interesting conversation to me because my boss "Quality Director" is asking me as the lead auditor to make internal audits more interesting to management during our management reviews.

I had a company call me the other day. This company is offering to do our internal audits for us and they ensure me that this will meet the requirements of the standard. They charge a flat fee to audit our company once or twice a year, they perform the audits enough ahead of our registrars audit so that we have time to resolve any issues and they will make observations that will allow our audit teams more time to focus on continuous improvement. This same company is also a registrar. Sound like a scam?

I don't know how many of you have been involved in ISO since your company was first certified but speaking from experience.... internal and external audits have been almost exactly the same from the biginning, why wouldn't the training be the same? We're looking for compliance to the same standard....

Something definitely up there. The whole presumption that internal audits are only done just prior to External assessment/surveillance would be one of the biggest issues of concern for the company where this is happening. the whole point of the internal audits are for continued monitoring of the systems, identifying issues as they crop up (i.e. as they are identified in as an audit NC).

I would not say that the internal and external audits are the same. I assume i'm right in thinking generally internal audits are subjective and so would be different from company to company so it may not be the case for all, however the external audits we have for 9001 focus on different points from a vastly different perspective in comparison to the internal audits that are mainly project (and occasionally also systems) based.

Agreed that internal and external audits are not exactly the same. External audits don't go into the depth that internal audits should but we are still both trying to ensure that the company meets the same requirements therefore the only thing that can be much different is how in depth the audit goes. Our external auditor rarely looks to ensure we are using current documentation but that's something that we check in our internal audits every time.

Internal Audits have the capability to be an amazing tool if implemented properly. Your team are the experts on your system, your processes, and your organization.

There will be minimal - if any - benefits if your audit process:

Focuses only on documents and spelling and verifying that every single step is followed absolutely perfectly - there is more to a management system than grammar and, unless the process you are assessing is critical, focus elsewhere. The police are unable to catch everyone who speeds, and focus on those who are speeding excessively. Focus on the important areas...not the little details!
Worries about offending people. An audit is about the processes...not the people. If they get offended because you found a discrepancy, remind them that this is about improving the system and the organization...not about their feelings. The value of an audit comes from the improving the processes not ensuring that everyone is smiling.
Audits only to the Standards' requirements. Look at effectiveness of the system, of the processes, goals, resources...the audit team should question everything, why things are done in a particular manner. To simply yes or no to whether something is done will add no value.


Your own internal audit team knows the business, where the problems may lie, and - perhaps most importantly - where the opportunities may exist. The audit process should look beyond the details and the problems...look at the interactions!...the efficiencies!...the potentials!

Question everything. Ask why we do things the way we do.

At the end of the audit, there should be a report that clearly shows the health of the process...not that it was simply alive, but that it had a healthy heartbeat, that it was thriving.

But if you want it in simple terms, it's better that you find things internally than paying some external person to find them. While it's a double-edged sword, that external person has the potential to revoke your certificate...your internal process does not.

My concern happens to be the poll results. While "Yes, we get measureable benefits" is in the lead, by adding up "Not a priority" and "No", we get well over half of the respondents.

Root cause time...WHY? Why are internal audits not a priority? Why would organizations drop them if they could? Where has the internal audit process failed at these organizations?

From my own personal experience and observations, I find that there is no value added when all an audit focuses on is a checklist of requirements. Yes we do this...no we don't do that. Any so-called improvements to the system are as the requirements change...that's not improvement, that's modification.

For those organizations that do benefit from internal audits, what has made your internal audit programme a success? I've alluded in my previous post to what I've seen make it a success where I've worked. What about the 38% who say there is a benefit from internal audits...rather than turn this thread into a complaint forum for internal audits, let's share our own best practices and celebrate our successes! :D

I do honestly believe that our internal audits benefit our company. The issue I have is how do I get management intersted in what we find? Of course our largest "finding" is documentation related, that's easy to find. We recommend changes as "observations" in our CA system and I present those to managemnt at our reviews but those are few and far between.

I've gotta run to a meeting but I find these conversations interesting. This is my first contribution to any discussion thread. Thanks!!!!

I do honestly believe that our internal audits benefit our company. The issue I have is how do I get management intersted in what we find? Of course our largest "finding" is documentation related, that's easy to find. We recommend changes as "observations" in our CA system and I present those to managemnt at our reviews but those are few and far between.

I've gotta run to a meeting but I find these conversations interesting. This is my first contribution to any discussion thread. Thanks!!!!

I get my management informed and involved by including them in the weekly corrective action status report. They do get involved.

This thread has taken an interesting turn, thanks - I believe - to Roxane!

Most 'programs' don't succeed because management aren't behind it (for very long). We've all seen this in one way or another over the years.

Internal Audits simply must be made relevant to the performance of the business, including those concerns of the customer and regulatory compliance. This requires a whole different approach to the planning, scheduling and preparation of audits than is done by external auditors (simply because they can't be that focussed) Hence doing one or two 'mega-audits' a year is totally incorrect - even if it did get you registered!

In my estimation, internal audits should take into consideration (primarily)business performance issues in their scheduling - not abitrary issues like 'covering all the elements every year', all procedures, or similar. In much the same way, a rigid 12 month calendar of audits - which is then frequently adjusted - is a sure way to confuse management and make it appear that the audit manager isn't in control!

Rarely, in my 25+ years of experience, did an internal auditor ever sit down and plan/prepare audits with management, using language they understand - ISO terminology is too foreign - or focussing on things which they (management) are measured on, held accountable for and, hence, loose sleep over.

Of course, none of these things is taught (effectively) in any auditor course (especially the accredited ones!). Maybe my friend Helmut's class does!

It's very easy to read here, in many Cove posts, what issues auditors 'go after' - nothing that management get excited over, quite frankly! (viz document control, training records, calibration stickers etc.)

I'm going to suggest that where management claim to get benefit, their comments are mainly derived from their personal 'feel good' that they aren't responsible when the CB auditor come around, that they can claim in front of peers to be 'improving' or just to 'save face' - rather than anything else.

It's my firm belief that internal audits need a step change in their performance (with all that entails, training, implementation, etc.) and should be modelled along the lines of a (mini) 6 Sigma Project, with the auditors as the 'Green Belts'. When audits/auditors start delivering $$$ savings and assist in actual measureable improvements, rather than 'feel good', they will have arrived!

This thread has taken an interesting turn, thanks - I believe - to Roxane!

Most 'programs' don't succeed because management aren't behind it (for very long). We've all seen this in one way or another over the years.

Internal Audits simply must be made relevant to the performance of the business, including those concerns of the customer and regulatory compliance. This requires a whole different approach to the planning, scheduling and preparation of audits than is done by external auditors (simply because they can't be that focussed) Hence doing one or two 'mega-audits' a year is totally incorrect - even if it did get you registered!

In my estimation, internal audits should take into consideration (primarily)business performance issues in their scheduling - not abitrary issues like 'covering all the elements every year', all procedures, or similar. In much the same way, a rigid 12 month calendar of audits - which is then frequently adjusted - is a sure way to confuse management and make it appear that the audit manager isn't in control!

Rarely, in my 25+ years of experience, did an internal auditor ever sit down and plan/prepare audits with management, using language they understand - ISO terminology is too foreign - or focussing on things which they (management) are measured on, held accountable for and, hence, loose sleep over.

Of course, none of these things is taught (effectively) in any auditor course (especially the accredited ones!). Maybe my friend Helmut's class does!

It's very easy to read here, in many Cove posts, what issues auditors 'go after' - nothing that management get excited over, quite frankly! (viz document control, training records, calibration stickers etc.)

I'm going to suggest that where management claim to get benefit, their comments are mainly derived from their personal 'feel good' that they aren't responsible when the CB auditor come around, that they can claim in front of peers to be 'improving' or just to 'save face' - rather than anything else.

It's my firm belief that internal audits need a step change in their performance (with all that entails, training, implementation, etc.) and should be modelled along the lines of a (mini) 6 Sigma Project, with the auditors as the 'Green Belts'. When audits/auditors start delivering $$$ savings and assist in actual measureable improvements, rather than 'feel good', they will have arrived!

And what conclusions have been determined with the data since the original thread started in August 2006?

<snip> Of course, none of these things is taught (effectively) in any auditor course (especially the accredited ones!). Maybe my friend Helmut's class does!

Mine does! ;)

Jan.

Look at effectiveness of the system, of the processes, goals, resources...the audit team should question everything, why things are done in a particular manner. To simply yes or no to whether something is done will add no value.
SNIP. The audit process should look beyond the details and the problems...look at the interactions!...the efficiencies!...the potentials!:agree1:Absolutely. Unfortunately, internal auditors are indoctrinated in the stupid game of check of conformance. The type of conformance that could lead to life jackets made out of concrete, as long as this was in the specification..:frust:

Soon, we will have a sector standard that EXPLICITLY requires auditing for effectiveness. I suspect that many auditors will not be able to make the mental transition, required to satisfy the intent of the AS9101 Rev.D standard, just like many (and to this date) have not been able to understand the process-based approach to auditing.

I agree with Andy, as well. The vast majority of internal auditing courses send the wrong message, as well. Blind focus on conformance, rather than effectiveness assessments.

We must be doing something right then. Because we always assess effectiveness and note it in the audit reports.

Mine does! ;)

Jan.

Sorry, Jan - I overlooked that you did your own courses!

Rarely, in my 25+ years of experience, did an internal auditor ever sit down and plan/prepare audits with management, using language they understand - ISO terminology is too foreign - or focussing on things which they (management) are measured on, held accountable for and, hence, loose sleep over.

Of course, none of these things is taught (effectively) in any auditor course (especially the accredited ones!). Maybe my friend Helmut's class does!

We discussed this just today in the accredited AS9100 class I'm teaching this week. We had a very lively discussion about requesting management direction/input for internal audits based on process measurements and management review information. Probably about the 5th time this week we've discussed similar topics...

We discussed this just today in the accredited AS9100 class I'm teaching this week. We had a very lively discussion about requesting management direction/input for internal audits based on process measurements and management review information. Probably about the 5th time this week we've discussed similar topics...

Ah, yes - discussed it! Most course instructors I know always discussed it.........but they never trained on it!

If you research the accreditation requirements for internal/lead auditor courses there is almost nothing about audit program management. Indeed since most lead auditor courses are taught using a case study of an external audit (generally a registration audit), for which the audit planning topics are usually confined to planning a multi day audit (result = a schedule for each day) or personal preparation of checklists.

As a result there are no skills developed by the attendees which help them to make the business case for why an audit might be necessary or useful to management!

Our organization is imperfect and somewhat uneven in different players' prioritization of regulatory-compliance issues, as we try to continuously improve in a too-few-overall-resources-for-the-workload environment.

The Quality manager who runs the internal audit program, and I as Regulatory Manager, are finding our internal audit program to be a very useful tool for increasing the internal visibility of, and top-managerial pressure on, those individuals who haven't managed to schedule their resources to accomplish particular tasks that negatively affect our regulatory/quality status and/or might get us into trouble during future external audits. We just add those tasks to the internal audit schedule as soon as we think they should be done. If they're not done in time for the audit...well, heck, the manager involved should do something about fixing that NC.

Now that's interesting, especially to management...We have a Regulatory Engineer and he has been working to ensure that we meet all those crazy requirements. I will talk to him. In the past I believe my internal auditors have overlooked the opportunity to ensure that the company is aware of and addressing all the regulatory requirements. It might be good for our Regulatory Engineer to do some training with our audit group.

How about SOX? They also have requirements and do audits of their own. Do you see that fitting into our ISO Internal Audit program?

Now that's interesting, especially to management...We have a Regulatory Engineer and he has been working to ensure that we meet all those crazy requirements. I will talk to him. In the past I believe my internal auditors have overlooked the opportunity to ensure that the company is aware of and addressing all the regulatory requirements. It might be good for our Regulatory Engineer to do some training with our audit group.

How about SOX? They also have requirements and do audits of their own. Do you see that fitting into our ISO Internal Audit program?

Now that's an improvement, getting your auditors to consider regulatory issues (especially new/changed regs)!

From experience, although there is potential to link to SOX, the culture of SOX audits is very compliance driven.

I've also done both types- internal (which I did not like- because the company and I were not on the same page when it came to quality) and external (FDA not NB). True- auditing is auditing- but the dangling of a non-compliance carrot in front of management (i.e. civil money penalty, injunction, seizure, etc...) as a result of external FDA audits... really wakes up management. They are reactive, not proactive. But of course their goal is the artificial business goals to make sure they get their bonuses- not to make sure that compliant products are distributed to the public.

And what conclusions have been determined with the data since the original thread started in August 2006?

Any conclusions yet Andy?

Any conclusions yet Andy?

See #30!

See #30!

Thanks Andy, I must be loosing my eyesight in my old age. :lol:

See #30!

Based on what you said then, and the results shown now, hasn't there been some alteration to the results and %?

40% are saying they get value from them, and only 29% saying 'we'd drop 'em if we could'.

Based on what you said then, and the results shown now, hasn't there been some alteration to the results and %?

40% are saying they get value from them, and only 29% saying 'we'd drop 'em if we could'.

Not really, Jane since I also view "management not making them a priority" (they tolerate them, but wouldn't ask for them to be done) to be a similar state to "we'd drop them". Therefore, it's weighed heavily against the audits being of any real value.......

Now, I'm not saying that there aren't some places that do 'get it'. But they are way in the minority, I'd suggest. Often, it's because the audit process owner doesn't know how audits can give value (past simple compliance).

Therefore, the audits are done on a 'push' system, they don't report anything that management really care about (not following procedures etc.) or have an overwhelming amount of picky items which are them supposed to have full-blown Corrective Actions applied!

Has it really changed? No!

Just a couple comments:
1. To give value to the audits, I believe you have to use Internal Auditors that are highly respected by the auditee and those are hard to find. My experience tells me that process owners do not typically like to be told that maybe they can do something better by an outsider.
2. Simple compliance to me is very important. I keep hearing how we should be getting more from our internal audits. What's wrong with simple compliance? Isn't that why we started doing internal audits in the first place? Compliance to the ISO requirements is supposed to mean something to our customers. :)

Just a couple comments:
1. To give value to the audits, I believe you have to use Internal Auditors that are highly respected by the auditee and those are hard to find. My experience tells me that process owners do not typically like to be told that maybe they can do something better by an outsider.
2. Simple compliance to me is very important. I keep hearing how we should be getting more from our internal audits. What's wrong with simple compliance? Isn't that why we started doing internal audits in the first place? Compliance to the ISO requirements is supposed to mean something to our customers. :)

Good points, thanks! I totally agree about the selection of auditors. I wish it was done more often the way you describe - with people who had earned some respect! It rarely happens that way, as you correctly say.

There's nothing wrong with simple compliance, in the early stages of implementation. However, I'm tempted to say that 'if you always do what you always did, you'll always be what you always were....."

Like a lot of things in life, a quality system goes through stages of maturity. Therefore, to see the maximum benefit from audits, the approach to doing them - in all respects - must also mature. My experience is that internal audits rarely mature, in fact they degenerate into 'going through the motions'. I'm not advocating compliance, per se, just saying that it shouldn't remain the focus.

It's funny that you mention stages of maturity. I think if your company has a good Continuous Improvement program, your company will automatically grow and mature. I think this may be why I have a hard time "getting more" out our internal audit program. People truley are on board with continuous improvement. I'm going to fall back on compliance auditing being a good thing because as the company grows and employees come and go, I see it as a great training opportunity to keep people on track with the basic requirements. Having said that....I do understand that it can get old for managers to hear the same audit results as they always heard.

Sounds like an awesome place! Have you ever audited an improvement? To see if it's become institutional? I have seen examples of great improvements, put in place by energetic Kaizen teams, only to have them go to waste over a period of time, simply because the work was never committed to the quality system - the CI leaders saw ISO as 'BS' and so once the team got done, they never saw the benefit of writing anything down.

As a result, the improvements actually caused problems in other areas. Also, these improvement were 'transparent' to many auditors, since they weren't from that 'department' and, therefore had little to no exposure to waht was going on before or after the changes. Also, there was no linkage to the other affected departments, so other auditors weren't aware of the interactions.

So, from my perspective there are a number of audit issues which could have been effectively raised with management but because they did simple compliance audits, this crazy situation was never resolved.......

Hi Jenifer,

I was just reading this post from 2006. I am interested in the outcome of the model you were in the process of designing: (quote: "I am in the process of designing a model that includes the necessary elements to prove value to process owners, train for repeat performance, win hearts and overall provide the vehicle (as a set of user-friendly tools and methods based on an existing quality-focused culture) for ratcheting the organization to the next higher performance level.") How did this go?
Silkeborg

I think I may have said this before but I feel that my main resonpsibility to my company and to our customers (as lead auditor) is to ensure we maintain our ISO registration and that means compliance auditing. Effectiveness auditing is interesting. Where can I learn about it and what can I use to back up my observations? The ISO standard? Anyone can go audit a process and come out with suggestions but those suggestions need something to back them up.:)

OOPS!!! I think i just replied to the wrong discussion.......sorry!!:o

I think I may have said this before but I feel that my main resonpsibility to my company and to our customers (as lead auditor) is to ensure we maintain our ISO registration and that means compliance auditing. Effectiveness auditing is interesting. Where can I learn about it and what can I use to back up my observations? The ISO standard? Anyone can go audit a process and come out with suggestions but those suggestions need something to back them up.:)

Not a problem! Your response is in line with the theme of this thread. The value of compliance auditing over effectiveness is exactly what makes the difference between getting management to fully support or only give 'lip service' to the internal audit program.
Your question is a very valid and common one. The difference is, IMHO the focus of the audit program. Many audit managers and auditors never get beyond compliance!

Finding auditor training that gets beyond compliance is tough to find. You will notice a couple of fellow Covers (Helmut Jilling for example) offer courses which have gone beyond simple 'do what you say' audits.

Auditing a process for effectiveness is not that difficult, really. We often do something similar in our daily lives. You have to factor in the objectives and results of the process as well as looking at the controls in place. It's not too much to consider if you think about it! Did we actually get what we planned to get from the process, either internally or as seen by the customer.

I think I may have said this before but I feel that my main resonpsibility to my company and to our customers (as lead auditor) is to ensure we maintain our ISO registration and that means compliance auditing. Effectiveness auditing is interesting. Where can I learn about it and what can I use to back up my observations? The ISO standard? Anyone can go audit a process and come out with suggestions but those suggestions need something to back them up.:)

The ISO 9001 standard itself requires effectiveness auditing!

The organization shall conduct internal audits at planned intervals to determine whether the quality management system
a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and
b) is effectively implemented and maintained.

IEffectiveness auditing is interesting. Where can I learn about it and what can I use to back up my observations? The ISO standard? Anyone can go audit a process and come out with suggestions but those suggestions need something to back them up.:)

It is required by 9001 (as howste points out) but like everything else in the Standard, it won't tell you how.

In my opinion, it's not really something you can learn from books or by reading about it. There's nothing like getting real experience from experienced professionals - I've learned something from every good auditor I've watched work. (And occasionally by watching bad ones, but that's usually a demonstration of what not to do).

Some ideas:

Shadowing another (experienced & effective) auditor and then debriefing afterwards.
Seeking help via your local quality organisation if that works
A good course (like perhaps the Lead Auditors course) run by a reputable organisation and delivered by experienced professional/s.
Can you enlist the help of your certification auditor? They can often be really helpful if you indicate a desire to learn or know more & give suggestions/tips. This doesn't have to be part of the audit itself (they may think it's too close to consulting - but there's always the possibility of a conversation over lunch!)

The standard says we shall conduct audits to determine whether the quality management system is effectively implemented and maintained. To me that's still compliance auditing. I think people throw the words "Effectiveness auditing" out like it's a totally different process backed by some other requirement when in fact it is still auditing to ensure we are compliant to the standard......correct? Sometimes I just feel like I'm missing something when there is talk about effectiveness auditing and maybe I am.....

The standard says we shall conduct audits to determine whether the quality management system is effectively implemented and maintained. To me that's still compliance auditing. I think people throw the words "Effectiveness auditing" out like it's a totally different process backed by some other requirement when in fact it is still auditing to ensure we are compliant to the standard......correct? Sometimes I just feel like I'm missing something when there is talk about effectiveness auditing and maybe I am.....

You might be, if you never saw the majority of findings which tend to say something like 'Not following a procedure', or 'don't have a document to meet ISO 9001 XXX'. These are compliance audits. We see (at the Cove) people who want a checklist for ISO XXXX - if they audit to that checklist, they are doing a compliance audit also.

Auditing for effectivess of the QMS requires that there is an understandng of the objectives of a process and an audit of the results. It rarely happens!

Many (most) auditors are taught to do compliance audits, believe it or not! They are told to study the standard, compare the QM to the standard and then audit the implementation of the system to the requirements of the QM. Effectiveness is often judged by whether the company in the case study passes the audit!

Hopefully you are still seeing findings like 'Not following a procedure' when you do what you call effectivity audits as well, somewhere in the goal of auditing needs to include compliance.

I actually do understand and agree with what you are saying but I believe that a checklist is a good way to train new auditors. There is no reason why a checklist can't be used as a reminder for the auditor to to take time to understand what objectives are in place and to ensure that the objectives are being met and that actions are being taken when they are not being met. And also understand how that all works to benefit the company.

My concern with all this is that Quality professionals tend to make things more difficult by speaking in terms of complaince and effectivity audits. To me it's "INTERNAL AUDITS" and part of the internal audits should ensure the company is both compliant and effective. I hope you are not taking my opinions as being negative. What you say makes sense to me and I plan to take some of this discussion to my audit teams. Thanks.....

My concern with all this is that Quality professionals tend to make things more difficult by speaking in terms of complaince and effectivity audits. To me it's "INTERNAL AUDITS" and part of the internal audits should ensure the company is both compliant and effective. I hope you are not taking my opinions as being negative. What you say makes sense to me and I plan to take some of this discussion to my audit teams. Thanks.....

If the QMS has been designed such that compliance is built into the documentation (hence the processes), you can audit processes without direct reference to the standard. If a process complies with your internal documentation--and is effective--but doesn't comply with ISO 9001, something is seriously wrong with the design of the system, not with the processes. IMO, if you find yourself in a position where you have to reference ISO 9001 in order to justify the existence of a nonconformity, you're missing an important point, which is that the QMS should be designed first and foremost with the needs of your business in mind, and compliance with the standard should be a secondary consideration.

Added in edit: Think of it this way: do you operate your processes in a controlled manner because "ISO says" you must, or because it's the only sensible way to do business?

Believe me, first and foremost we want to make money and have happy customers. I believe our company sees the benefit of ISO but if we felt we had to do something to meet the standards requirements that would sacrifice that....there would be issues.

QMS should be designed first and foremost with the needs of your business in mind, and compliance with the standard should be a secondary consideration. We did not have a well defined QMS before ISO and it's hard for me to believe that many companies did. We performed very well without it but ISO help us to structure ourselves and stay focused on what we feel is important. Our QMS and ISO were built together.

Thanks for the input.....more good stuff!!!

Goodtimes:

No, I didn't see anything negative in your posts - quite the reverse, actually! I was delighted to see someone interested in auditing for effectiveness.

I would take a different perspective about the use of checklists tho'. Checklists are, in many ways, used as a crutch to a crumbling audit program, where hapless auditors are given imprecise audit tasks, for which they've only had 2-5 days of classroom training (if they're lucky) and then often no previous knowledge of the process they're going to audit (in the name of independence). The auditor defaults to simply asking the checklist questions which are pre-ordained without the benefit of knowing what the answer should be......result? A (poor) compliance based audit.

Checklists are, as I've posted before, like a shopping list. Each shopping trip is slightly different, although often the basics are purchased - depending on the 'scope' of the audit - the content is somewhat different. You can't use the same shopping list every time and, with an inexperienced shopper, (teenager for example) you'd have to write very specifically what you wanted them to get. When they don't come back with what you wanted, who's responsible?

I'd suggest that coaching auditors, while a lengthy process in some cases, is far, far better than arming them with a canned shopping list. Getting them to create their own, while being coached on planning is much more effective.

You mention the "scope" of the audit. I have a 3 year audit plan. Basically I plan to cover all processes a minumum of once in a three year period. Unless of course there has been a change in the status/importance of that process or if there have been an unusal number of customer complaints. I've created a matrix that consists of all of our processes and I've identified what elements of the standard apply. My auditors are supposed to use that matrix, review the standard and applicable internal documentation, then create their checklist. Additionally I have quarterly meeting to discuss things such as effectiveness/conformance auditing. I do stuggle with how to identify the scope of the audit and why it would be different from one time to the next.

You mention the "scope" of the audit. I have a 3 year audit plan. Basically I plan to cover all processes a minimum of once in a three year period. Unless of course there has been a change in the status/importance of that process or if there have been an unusual number of customer complaints. I've created a matrix that consists of all of our processes and I've identified what elements of the standard apply. My auditors are supposed to use that matrix, review the standard and applicable internal documentation, then create their checklist. Additionally I have quarterly meeting to discuss things such as effectiveness/conformance auditing. I do stuggle with how to identify the scope of the audit and why it would be different from one time to the next.

Oh, man. And there I was thinking you'd got something new and exciting to offer us in the way you do audits! My bad, and I don't mean to sound critical, but why, why, oh, why have a three year plan? Do you use a crystal ball? I don't even know in my own life what I'm going to be doing in the next year! None of us can reasonably predict what's worth auditing in anymore than about the next 6 months! Business is too dynamic to 'fit' your schedule. I guess if you want to keep changing the schedule, but I have better things to do!

Adjusting the audits based on customer complaints is too reactive! Water under the bridge, too darned late - it got out, it affected the customer - shame on you! An effective audit program (not a 'plan') senses when things are not performing, have changed etc. and goes to review what's going on, so the impact is mitigated (from the customer's POV) as soon as possible. It's dynamic...........and covering all the requirements every three years is a waste of your time, frankly....sorry to be blunt!

If you are having a problem identifying the scope of an audit - there's a clue to the ineffectiveness of audit program planning! The scope of an audit is what you want the auditors to focus on! It's often not the whole process that causes problems, or is changed or is new. It's a part of it. That's the scope you want to go after, in your audits.

I bet you've never had your CB auditor comment about your audit plan - why, because they are flattered that you're emulating what they do! Indeed, you're doing their work for them!

By the same token, what would your management team tell me if I called them to ask "what benefit they get from internal audits"? (if I promised not to tell you what they said!)

BTW - internal auditors don't need to know what elements of the standard apply where. They need to know 1) the objectives for the process, 2) the process and 3) what results the process gets. Then they can audit to see if the process is being controlled effectively to confirm the numbers are credible. It's no use - IMHO - filling their heads with 'ISO'. I know they probably got trained that way, but they didn't attend my class (when I ran them) and its really not necessary for them to rely too heavily on that aspect. They do, however, need a better than basic understanding of the 'blocking and tackling' of an effective management system!

I hope I didn't bruise anything........honestly!

Friends,

Great thread! Great posts! :applause::applause:

Stijloor.

Wow...it's been nothing but bad weather here in Ohio and I haven't had time to respond. But here goes....

Why have a 3 year plan?....First of all, no crystal ball is used here. I don't actually look forward three years and decide what to audit. I base the number of audits I plan to do each quarter by taking the number of processes we have and spreading them over a three year period. We have approx. 32 activites, I'd like to look at each of them at least once in a three year period which means I need to plan a minimum of 3 audits per quarter. I look backwards at my schedule to determine what processes have not been audited within the three year period. No process should go on forever and not be audited.

I also plan audits each quarter based on the status and importance of the activities as well as customer compliants. I agree that auditing based on customer complaints is reactive but no company gets by without some customer complaints. Therefore, I feel that it is always a good idea to take a look at what those complaints are and determine if there is a problem area that I should audit.

The scope of my audits are basically the same each time. Inputs/outputs, elements of the standard that apply and objectives and measures and taking appropriate actions should always be looked at.

If you plan to ask management what benefit they get from internal audits, please also ask them what benefit they get from being ISO certified. Believe it or not I once suggested that we drop our certification and continue our internal audits and invite customers to audit us to see if they feel we have a good QMS in place. In all fairness, I do understand that this is not really a good option for many reasons but I think it's a good question.....

What do you think of the idea to give checklist/questions to auditee/s before the compliance audit ?

By this way, we give them time to prepare necessary documents, quality records, etc. beforehand. Because audit is not a sort of exam. We just want to control whether things were done according to standards, procedures and other company requirements. And also I think this will reduce auditee's stress and thus audit time (finding out documents among bunch of folders, calling up someone knowledgeable etc.)

What do you think of the idea to give checklist/questions to auditee/s before the compliance audit ?

By this way, we give them time to prepare necessary documents, quality records, etc. beforehand. Because audit is not a sort of exam. We just want to control whether things were done according to standards, procedures and other company requirements. And also I think this will reduce auditee's stress and thus audit time (finding out documents among bunch of folders, calling up someone knowledgeable etc.)

I have no problem giving auditees information about what to expect during the audit. If you are using a checklist there is nothing wrong with giving them a copy.

During the audit though, I don't want them to decide what documents and records they will show me. If they show me evidence that they've carefully checked over before the audit, I may not find issues that need to be addressed. I always want to be in control of the sample so that I can collect evidence representative of what is actually going on in the system.

As an internal auditor, I don't prepare a checklist and what preparation I do, I do only a few minutes before I start the audit.

What is the purpose of 'sharing' checklists? What happens if you don't stick to the checklist?

I think your audit program needs help if you're doing this! Do people feel as if they have to 'pass' the audit, like a CB audit? You don't want people to prepare 'evidence' before hand, it's not productive. If they can't produce records, then maybe one of two things is happening:-

You're asking the wrong people for the wrong thing, or

You're going to discover there isn't a very good system for records or the people don't know it!

Maybe, your doing too much auditing like a CB auditor. You should be 'proving' people know their process, including auditing recent activities, so their shouldn't be a problem with them finding the right records should there?

You might want to rethink the way you go about doing internal audits!

What do you think of the idea to give checklist/questions to auditee/s before the compliance audit ?

Depends on what's on your checklists, and how specific your questions are. Like Howste, I am happy to give information about the kinds of questions I'll be asking and the kinds of info I'll want to see.

But I disagree that "audit is not a sort of exam." In a sense, it is - a test of samples at a particular point in time, to determine whether the system is being followed. And if I give too much information to them (and thus allow them to prepare everything), it won't be a valid test. The auditees may be very unstressed though.

I don't want them to decide what documents and records they will show me. If they show me evidence that they've carefully checked over before the audit, I may not find issues that need to be addressed. I always want to be in control of the sample so that I can collect evidence representative of what is actually going on in the system.
Me too - totally agree.

As an internal auditor, I don't prepare a checklist and what preparation I do, I do only a few minutes before I start the audit.
I'm surprised by your saying this. It seems to indicate you do no (or virtually no) planning for any internal audit! If so, then it's probably because you're a very experienced auditor.

You almost certainly do have a checklist, I venture to guess, but it's probably 'in your head'.

Or maybe we're both hearing different things in 'checklist'. I do use one (written or not). If written, it helps make sure I remember key things, and check for the same things across different samples. But I never feel constrained by it, I never hesitate to drop it if it isn't working and/or change it whenever I need to. It's a planning aid, that's all. I'd rather not give auditees 'the checklist' because it may mislead them. But I'll happily provide a list of topics and some sample questions.

Maybe, you're doing too much auditing like a CB auditor. You should be 'proving' people know their process, including auditing recent activities, so their shouldn't be a problem with them finding the right records should there?

Just so.

Goodtimes,

The 3 year plan you lay out sounds broadly reasonable to me, presuming that there is very little actual change in your organisation (ie, it's very stable), there's little if any development, perhaps only minor improvements and that other indicators in your management system are not indicating problems.

The only bit I query is:

The scope of my audits are basically the same each time. Inputs/outputs, elements of the standard that apply and objectives and measures and taking appropriate actions should always be looked at.

I have a bit of trouble understanding how the scope could be the same every time, as I'd be inclined to vary it more. I also don't usually audit against 'elements of the standard', as that should be built into the design of the system - for internal audit, I want to audit against the company's own criteria, ie, its system (less so the Standard).

You're right Jane - a very experienced auditor ;)

I use a planning document based on my 'football' diagram. It's nothing like a conventional list of items to 'check off' as the audit progresses. It's a way to focus and condense the 'answers' that I should expect to see, during the audit - the 'planned arrangements' which are supposed to be in place.

Since most checklists I've seen are simply a listing of criteria (like ISO 9001) and a place to put a check mark (or 'tick') and somewhere to add comments, then my planning document is not even remotely similar. Indeed, giving the auditee a football before the audit would probably confuse them!!

I agree with your other posts. The idea of any audits which go over the same ground without change is an audit program looking for help......

Thanks for all the responses.. :agree1:

I think I have to give more information. Currently, we're auditing projects on the basis of project management procedure which was prepared in accordance with ISO 9001:2000. Auditees are project managers and during the opening meeting we decided upon which project we would conduct audit on. So our questions are derived from the articles of that procedure. That's to say we want to know whether they know the procedure and whether they are managing projects accordingly; keeping the necessary records and respecting to time schedule and so on. Of course we do not limit ourselves by those questions..

... our questions are derived from the articles of that procedure. That's to say we want to know whether they know the procedure and whether they are managing projects accordingly; keeping the necessary records and respecting to time schedule and so on. Of course we do not limit ourselves by those questions..

Those are the kind of general questions that I think is more than OK to give to your auditees. If audit is new to them, they may get themselves very worried about what they might be 'tested' on. Once they actually find out that they're just being 'tested' on whether they're doing what they're supposed to be doing (and hopefully are doing), it gets less stressful!

Perhaps I can illustrate the difference like this:

A. Specific-type questions on a checklist that I wouldn't use:

* Is the XYZ form for this project in the ABC folder?
* Is the form dated?
* Is the form signed by the Project Manager
* Does it have the correct project code on it? etc

That doesn't mean I wouldn't consider those things at some point, but this is too low-level and too 'Yes/No' for effective auditing IMO.

B. General-type questions on a checklist that I would use (& might well give to them beforehand). Note that my examples are definitely general, but that I'd have specific documents/records I'd be looking for in an internal audit, as specified by the system in use at that company)

* What is this project about? How is that specified & where?
* What is the process in use on this project? Is it following our requirements?
* Are adequate records available to show this, when & where required?
eg,: - Is there a project brief? (or whatever document)
- Are the contents adequate and suitable to state what is to be done?
- Only current versions in use (any changes? if yes, applicable procedure followed?)
* Does practice match the procedure? etc etc