posted 21 July 1998 07:38 AM               

What new techniques will you teach your auditors?

How will their roles change?

Lets work on a couple of these, has anyone started the move toward 'value added' auditing, instead of just compliance auditing?

posted 21 July 1998 07:55 AM               

These need to direct you. They need to reflect the procedures and work instructions. The questions need to be open ended not y/n. There must be room for comments, lots of comments. Ask the WHY and really be interested in the answer. Record your thoughts and concerns on a "trail" to review later.

How you work through the check list is a technique you must master. Practice practice, practice. Observe experienced / seasoned auditors....call your local ASQ, many are starting mentoring programs.

posted 23 July 1998 03:29 AM               

I guess my problem with external auditors offering advice is they often simply don't have the qualifications and/or background to suggest stuff.

On an internal level, dhillsburg's response above seems appropriate.

barb - what's your thing for check lists? The older I get the more I see check lists as stupid. I rarely use a check list. When I do I didn't feel I am adequately covering the document I'm auditing to. Like the QSA with QS9000. Screw the QSA. I'm gonna be audited to QS9000. I want to audit only to the actual document - spec, procedure, work instruction or whatever.

posted 23 July 1998 10:39 AM               

This method can be used for any spec or internal document. The problem I have with writing a separate check list is that it is not what is being audited.

Yes - I know. Everyone is taught how great check lists are. A number of years ago I pretty much believed it and even used a few. But then I found that often a war of words erupted when an auditor used a check list. I'd say 'Show me where it says that in the spec." and I'd be shown the check list. "No - show me where it says that in the spec..." would get me a response like "Well, that's what it means."

posted 23 July 1998 07:21 PM               

In my classes I teach the auditors how to work with a procedure/standard and highlight the key elements to look for and ask about..omissions etc. Basically making a new check list each time they audit. As a record of the audit trail, the marked up procedure shows exactly what was going on during the audit and what was covered/not covered. If they push for a checklist...I have one that I created that does the "prompt thing".... has a couple open ended questions..with lots of space for comments and sends them to their procedures for the answers.

So there you have MY take on checklists.

posted 30 November 1998 01:44 PM               

The thread title is 'audit towards continuous improvement' which is a bit vague with regard to the thread as a whole.

Within an audit department you can do what every company does in general (is *supposed* to do, anyway) - have a continuous improvement system (read self evaluation) & strategy.

If I read your reply correctly you are looking at this from a company wide aspect where each functional group should have a continuous improvement system which may be part of, or defined by, corporate. Some systems may be shared (corrective action is a shared continuous improvement system, as an example). As auditors, all we can do is verify that folks are following defined systems and, as applicable, that the system meets requirements.

posted 02 December 1998 08:31 AM               

Still there is merit in using the structure of checklists to maintain continuity from audit to audit. This can also be achieved in the open ended style. You need to be sure certain questions are asked. What I like to do (and instruct my internal auditors) is to use the standard they are auditing to and create atleast one question for each point of an element. I then instruct the group to create complimentary questions using the procedures (as often they contain processes and activities not governed, or necessarily required, by the standard). This allows them to have both structure and flexiblity in the audit (I think the broadest scope). I view the structure as maintaining continuity that is important. I view the flexibility as the continuous improvement technnique in that it reviews the practices in the procedure to determine effectiveness. Auditors that are not satisfied with a practice (assuming there are no major or minor nonconformances) note so in their audit reports as a general comment. This is done to let the responsible auditee know that this audit team agreed there may be a better way. This is done as an "FYI" and does not need corrective action. It does present the opportunity, however, for a continuous improvement project. What we have decided internally (and informally) is that if two independent audit teams note the same issue as a comment, this issue will be further investigated and a decision made to adjust or not. So while agree with John C. that auditing itself does not bring about continuous improvement, it sure can point you in the right direction. I also notice that CI projects determined by auditing capture projects not spawned by traditional inputs such as Cost of Quality, which often focuses on the measurable items in dollars and not in associate well-being. Audit exposed projects have unknown costs (sometimes unknowable) but have great merit in savings of time and effort (measurable, but often hidden costs). This type of improvement works great on increasing morale since it is often in the trenches and very visible to associates where job satisfaction can swing in a favorable direction.

posted 03 December 1998 07:03 AM               

CI becomes a product of the audit if the questioning techniques provoke thought/action on the part of the auditee (sans nonconformance or CA). An effective auditor with a receptive/resourseful auditee can produce mounds of continuous improvement opportunities....just by the exchange/communication during the audit !!
My intent in the original post was to identify and enhance those techniques...and develop the auditee to that level as well. Lets not forget how important it is to understand how to be audited. I typically do an internal audit class at my clients sites for element owners that may never be an active part of the audit team just for that purpose.

posted 03 December 1998 08:21 AM               

To get things going, here is a brief synopsys. As part of Internal Auditor training, I have the candidates work through a mock audit (I do a mock audit as there may be "independence" violations, but the intent is focused on the techniques and not logistics since the results are not considered) by reading the standard, procedures and other supportive documentation. I usually have small groups of five or less for this. The next step is to have the group develop the working papers. This is comprised of the questions, helpful reminders, and generally flowcharts (the charts are largely used to help the auditor understand material/paper flow within the system). The flowcharts are a great source of determining redundancy in the system (a continuous improvement opportunity). Sometimes discoveries are made while doing the desk study (suitability audit). The new candidates then go out into the area to be audited with there mission (the compliance audit). The main follow up question asked by the auditors..."Show me". Activities are compared to the documented program, deviations and comments are noted. We huddle several times during the mock audit and I often ask how things in the system compare to the program. I like to use the flowcharts to determine effectiveness of the system. Have the auditors added to the flowchart? Have steps in processes, material and paper flow, been repeated? How about the quality records? Were sections routinely not completed or redundant (more sources)? Lots and lots of opportunities for improvement. Is that Job Security or evidence of a bad job? I like to think of it as just ineffecient business planning.

posted 03 December 1998 03:31 PM               

I steer their focus to their systems - not the ISO or QS standard. Continuous improvement 'opportunities' pretty naturally come out of the audits. I talk about it, but I do not see it as their place to focus on CI. As I say, that pretty naturally happens. The primary function is to ensure systems are up and running in accordance with thgeir documentation.

posted 03 December 1998 04:51 PM               

Can you outsource "Internal Quality Audits"? Interesting question for the Internal Audit Forum. I seem to remember that topic a short while ago. In any regard, you are right that IA fades with time.

posted 03 December 1998 06:45 PM               

Internal audits shall be performed. There is NO requirement that they must be performed by associates (altho this may be preferable to some). Internal audits can be farmed out, as long as there is objective evidence they have been performed.

posted 07 December 1998 09:25 AM               

I'm curious about the registrars perception of consultants preparing the Management Review Report for the client. Do they feel that management, while not committed enough to doing the prep work or follow up report for management review, have the committment to carry out ISO's, or the company's, objectives and goals? Or do they feel that, while an organization may not be able to do all the steps on their own (size limitations perhaps), they at least recognize the importance of an activity and source out what they can not do?

I guess for me, a company needs to bite the bullet and commit to the plan and do 'all' of the work. Auditing and Management Review alike. I think as a registrar (and I am not) I could not accept outside support. I guess it could be argued that continuous improvement can be made either way, no one organization has all the answers. Outside support is necessary and will occurr.

These are some of the reasons I think the ISO standards do not work for me. Too many items that have ambiguity. Maybe I am just hung up on semantics? For me, the benefit lies within the work itself, an intimate understanding of how things work and produce the results they do. From there, internalized CI efforts will likely produce better benefits (if not better results) to an organization since they have completed the PDCA cycle on their own. Oh well, I'm projecting again.

posted 07 December 1998 05:09 PM               

I am steering the conversation away from "auditing for continuous improvement." I appologize.

posted 12 December 1998 07:23 AM               

Since I sent my first contribution (which was in error because I didnât realise the the basic subject was audit), I havenât been able to pick an argument with anyone. Everyone is right in their own circumstances. I find it particularly easy to agree with yourself and I suspect we might be something of kindred spirits - a little under impressed with the whole issue of ISO 9002, as it is done, maybe?
But what I canât agree with is your statement;
ã...these are some of the reasons I think the ISO standards do not work for me. Too many items that have ambiguity.ä (Note; I do agree entirely with your next point; ã....the benefit lies within the work itself, an intimate understanding of how things work....ä)

The Standard (ISO 9001) is a masterpiece. The words have been carefully chosen (by practical engineers, way back before they became ISO 9001) to allow us to do things in a logical way, directly focussed on the customerâs and our companyâs needs. It selects the excellent sections 4.1 and 4.2 and backs them up with a set of the most likely requirements that we might need. But itâs always Îwhatâ we should do. Never Îhowâ. In their wisdom, they leave that to us. And each requirement is a clear statement which reads the same for me as it does for anyone who might presume to interpret it for me or put his/her own slant on it. The choice is in the implementation.
This choice of implementation is not ambiguity. It is not even a choice of one or another, or a choice from many. It is a choice of all possible (sensible) options. In fact, it is freedom to do it whatever way makes best sense within certain logical rules which are not the rules of The Standard, but the rules of common sense; Document what you are going to do, and how. Do it. Check it. Improve it.
What do you say?

posted 14 December 1998 09:14 AM               

Thanks for the kind words and a stinging rebuttal. The Cove is a great place to learn and share. I am impressed, in general, with all the good questions and answers given by the group. I also appreciate opposite view points. Gives me a chance to pause and review. Perhaps even change positions. Anyone not open to criticism, critical or non-critical, will not stand to learn much. I am not the Shell Answer-Man.

In general, I have a below average impression of ISO. Too many organizations have bastardized a very good, generally well written standard. But a standard for a system that may never improve and is as easy as it is to attain and retain does not satisfy the real need of an organization to improve (my own projection). How many organization, registered to ISO9000, don't actually believe in its essence? Too many.

The ambiguity in the document that I was referring to was that element 4.17 is entitled Internal Quality Audits. Just by the varied responses to Barb's topic indicates to me that there are many different interpretations. ISO likes to clear up gray areas, not create them. But with this element, we have have a gray area. The ambiguity may not be with the standard but with the interpretation of the standard.

Internal Quality Audit (as defined by authors on the subject, Mills in this case) - audits performed by individuals within an organization independent of the area being audited. As defined in the element it specifically states internal quality audit, both in the title and the body of the statement. While the document is clear in stating this, interpretation is wide. Using ANSI/ISO/ASQC A8402-1994 to clarify, the situation barely improves with the second note of the quality audit (note, does not mention 'internal'). Move to ISO 10011, again, little help. So now is it acceptable to outsource Internal Quality Audits? Now I agree with both sides of the arguements above. Good logic from good people with an abundance of knowledge. There in lies the paradox. How can I be partial to both interpretations (How can single person entities registered to ISO 9000x perform Internal Quality Audits and remain independent)? With that, I have choosen to fall on the side that Internal Audits must be performed by auditors from within that organization. Pros and cons to each path. Either way, still some confusion on which path is correct. Maybe they both are (dependent on your registrar)?

You are correct to point out that the standard tells you "what" to do and not "how" to do it. This is actually a very good point for those out there who are just getting involved with ISO. There are generally many ways to satisfy the requirements. On this one though, the "internal" portion of the element title becomes restrictive to me (again, my opinion). Warning: Putting too much of a slant may take you in the wrong direction.

Common sense. Another tough topic with amiguous connections. Nothing static about common sense. A very subjective topic. I think I'll keep away from that one as I have never been accused of having much of it. Did I just exercise it there?

posted 18 December 1998 08:10 AM               

Internal..within....

So is someone contracted by a company to come in, with the companies best interest...no external interests, no third party interests. The auditee is within, the client is within..and while performing the audit, the auditor is working for the company...so he is within.

posted 22 December 1998 04:09 PM               

First, a suggestion: I believe well thought out questions can add value.

As pointed out by Kevin Mader, auditors are responsible for verifying effectiveness (as well as compliance and suitability). If we focus our attention on the effectiveness and suitability we invite the kinds of questions that result in bottom line improvement (above that of just compliance as Barb Butrym was trying to point out in the first post). This is especially true in the years following certification when systems havenât yet been sufficiently customized to the companyâs needs.

The next step? Once you have settled into an effective, suitable, and compliant system, the next step should be·. Check it again!!! We NEVER want to settle into believing that our system is always effective, suitable, and compliant. The business environment changes, competitors change, we change, suppliers change. We need to continuously challenge ourselves to get out of the box. I believe this is the greatest value audits serve after the system ãseemsä to settle in. So, the question remains how do you challenge and question yourself to prevent entropy? I believe Kevin hit the nail on the head here again when he suggested flowcharts. Value-added analysis, flowcharts, pareto diagrams and other tools that facilitate improvement in common-cause variation should prove to be valuable in the effort to evaluate continuing effectiveness and suitability.

I am making a distinction here, that CI via auditing is different and adds value in addition to other preventive tools. Auditing directly and repeatedly challenges the suitability and effectiveness of our policies. What other tool does that? I think I am tapping into Kevinâs premise here that auditing captures improvement projects, ãnot spawned by traditional inputs such as Cost of Quality.ä

With that said, back to Barbâs questions, ãWhat new techniques will you teach your auditors?ä and ãHow will their roles change?ä

I suggest teaching continuous improvement tools such as value-added analysis, Pareto Diagrams, Flow Charts, etc., with a clarification as to how these tools can support their efforts to evaluate suitability and effectiveness. Any other suggested tools?

How will their roles change? They will certainly spend more time questioning effectiveness and suitability. Any other suggestions?

posted 23 December 1998 10:26 AM               

I agree with you that internal audit is likely to be far more effective as a quality improvement activity through educating the workforce in the overall aims and interfaces of the processes, and through Îvalue addedâ, than as a Îfind and fixâ routine. But there is nothing about Îquality improvementâ in ISO 9001. It was designed as a means of demonstrating and tracing proof of compliance for the purpose of giving confidence to the customer. It can be so much more and we are free to make it so, but we have to start by seeing it as it is, not as it is perceived to be with the hindsight of 2 1/2 decades.
(I suppose I sound like Iâm lecturing, but Iâm not. Iâm making it up as I go along, from the raw material you provided.)

I was out for over a week, with Îflu, so I wanted to make my contribution before the close-down. Iâve only had a chance to glance over the weekâs updates. Iâll go back to them now.

posted 23 December 1998 01:47 PM               

No offense taken from any of your, or any others, responses. The comments I made were light hearted (so intended) so I am sorry if I made you feel bad. You are not a bad guy by any stretch. Your input here is valuable and causes "good" debate and thought for myself (and others I suppose). As I see it, we all just put some input out here at the Cove not as a battle of wit, but more as an offering of viewpoints, and some of them will inevitably be opposite (this is generally a good thing by the way). The value with all the different perspective allows me to test theory or to reaffirm some beliefs. I must admit that before I found this site (thanks again Marc) I could have very easily believed all my bull. For myself in my organization, and perhaps for some others out there, there is nobody here to bounce things off of, test theory. At least not with any degree of certainty on the validity of their response (not to say that their input isn't valuable, but often skewed). Here you get the straight of it. So if you or anyone has a different opinion...great! I appreciate it! So, with that stated, no apologies necessary from anyone (in my case as I speak for myself). In addition, thanks to all the group for their support and kind words. It is a great help to me to have the feedback, positive and negative alike.

posted 05 January 1999 09:14 AM               

I really don't have much more to add on the topic, but a case point may help. I think that by my interpretation of what an Internal Quality Audit is for me combined with my experience limits how I can interpret the standard. In my opinion, either interpretation could be right. Each has to decide their own interpretation based on their own reasoning of the information presented. I have merely presented my view point for consideration.

As for refuting your logic, I can't. The fact is is that I am somewhat trapped by the paradox I spoke of earlier and my own convictions. For instance, while studying for the CQA exam I noticed that the interpretations of Bill Wortman were sometimes different with that of Mills. Who is right, who is wrong? This caused me some concern since if this question were asked on the exam, how would I answer it? With Wortman's interpretations or Mills's? Case in point: the sample question asked 'What is the most important auditor characteristic?'. Also consider this: Which is a more important charecteristic: auditor independence or auditor objectivity? As I went through the sample test questions provided by Wortman, I came across two questions that were very similarly written but had two different answer series (i.e. a, b, c, d, e. all the above). As it turned out in the first question, auditor independence was a choice but not the correct answer. Objectivity was. The opposite was true of the subsequent question asked later (objectivity was not given as a valid response selection)). Now I sit pondering which answer to give on the date of the exam. While I am more prone to side with Mills, basically for the fact that I have fewer disagreements with his interpretations and comments, I decided to chose auditor objectivity should the question be asked. It was. Did I get it right? Since the results of the exam are not published I am left pondering the same question. I sit in limbo waiting for more information to confirm or dispute my beliefs. I believe that you and others on this topic hold objectivity higher than independence as do I. But in my experience with three registrars, I can undoubtably say that they were most interested with independence. Combine that with Mills's defintion of an Internal Quality Audit, for the time being I must stay with independence regardless of my own beliefs. I guess I had more to say than I originally thought.

posted 06 January 1999 08:25 AM               

Thanks for the lead on Arter. I believe you are right that auditing is an art. When I conceptualize an auditor's audit and an artist's painting, I can see similarities.

posted 06 January 1999 08:30 PM               

At this time I will make a brief statement about my disgust with the ASQC other than it's heavy interest in financial gain:

quote:

---

Since the results of the exam are not published I am left pondering the same question. I sit in limbo waiting for more information to confirm or dispute my beliefs.

---

QS9000's interpretations was a farce and an admission that there is something seriously wrong with the document.

This web site is here, and is dedicated to, interpretations (you know what of). It's a shame that interpretations are such a problem. And it's a shame the ASQC promotes continuation of ignorance and ambiguity. Some feedback would help make their exams (CQE, CQA, you name it) more fair as it would force more consistent interpretations.

posted 07 January 1999 09:06 AM               

Yes. I sympathise with your situation. Itâs easy for me to adopt an anarchistic approach. My career path, such as it is, is just about run itâs course. But itâs different for you. The rogues and bureaucrats hold the concession as to whether you work or donât. They want your soul and a fat wad in return for it and you have to go along with them. But you have some choice; Play the game, flip a coin when a ÎWortman or Millsâ question comes along, pass the exam then think for yourself - donât give them your soul. If you promise to, cross your fingers behind your back.
What about this Îindependance or objectivityâ business - ÎWhich is most important?â Itâs like saying ÎWhen walking, which is the most important leg, the right or the left?â. The standard demands the first. Common sense (that old common sense which gets us through from breakfast to suppertime) demands the latter. The guy that thought up that question needs a lobotomy.

Marc,
Iâm generally scathing about the camp followers and would advise industry to cast them off. But there is a great difference between those who can, and want to, clear away the fog that has been created by the rogues and bureaucrats, and those who creat the fog.
How you help people through, I donât know because I havenât seen much of your work yet. But Iâm sure you do. I would disagree with you about Îinterpretationâ though. You will have seen that it is a favourite target of mine and Iâve probably said that I see interpretation as the start of a very long and slippery slope.
I read the words. They mean just what they say.
For this reason, I would prefer to think that you clear away the fog that has been created and bring people back to understanding the standard by reading what it says. Getting them to forget the myths that people have put into their minds.
In my experience, not one person in a hundred actually reads the standard - certainly not properly. They may have skipped through it sometime. More likely their view of it received wisdom, absorbed from snippets of conversation, articles, training courses, etc.
I donât think you interpret. I think you tell them what it says. Or just put it in front of their noses and shield them from the bad forces while they think it over.
If so, then good.
Letâs not fall out over this word. Just agree with me, huh?

posted 07 January 1999 10:23 AM               

Could not have said it better myself. What follows are my opinions only. I also am offended with what the ASQ has become. IMHO, they are just a money grubbing ( o )'s. Years ago, I let my ASQ membership lapse for the very reasons you cited, and more. For personal convictions (not money), I have not attempted to become 'certified' to one of their various programs (CQA, CQE, CQM or whatever) although I believe I am qualified to do so. A very nice young lady just passed her CQE exam, in part using my tutoring (KEY: in part).

quote:

---

They only protect themselves against any possibility of anyone 'questioning' what's going on. In the process they breed ignorance and compound the problems

---

Absolutely and well said. I think they have set themselves as 'guardians' of knowledge, but only disseminate that knowledge when it suits their purpose.

Interpretations: I am not qualified, nor will I attempt, to comment on QS 9000 interpretations. I will say this, however. From what I have read here and elsewhere in cyberspace, I agree that "QS9000's interpretations was a farce and an admission that there is something seriously wrong with the document." ISO interpretations are normally straightforward, however. The instructor, during my Lead Assessor training, pounded this point home. Read in what is there and do not read in what is not there. I would tend to agree with John on his point:

quote:

---

The original, simple intention has been smothered by 2 generations of do-gooders, bureaucrats, rogues and poor lost souls.

---

Perhaps I am being too pessimistic. Perhaps not. Anyway, just my $0.02 worth.

posted 07 January 1999 11:28 AM               

I don't disagree that ISO is relatively straight forward - but I use the term relatively as there are still plenty of interpretations by different auditors and registrars as to what is 'acceptable'.

John C - what I do is work with companies in implementing QS9000 and ISO9000. And every company I have worked with has been first audit successful (including Motorola [semi-conductor sector] and Harley-Davidson). I teach companies to get ready to 'do battle' with the auditor. I teach them how to interpret the specs. I also do training in a number of areas including MSA, PPAP, FMEA, APQP, Auditing (general and internal). I teach them to be ready for 'idiots' and I do this by ensuring they understand the specification word-for-word, the intent of the specification details and relate this to what they do.

posted 07 January 1999 02:06 PM               

I think that you would easily fly through the ASQ exams for CQA, CQE, and CQM based on the responses you have given here in the Cove. If nothing else, you have my recognition (for what that is worth) and you don't have to compromise anything by taking the exam. While I hope for better things to come for the ASQ, I still think that the ASQ is a good source for materials and certifications (should you want their recognition). I have felt for many years that while I didn't have their certificates, I too had the required knowledge and understanding. Yet, here is the wicked paradigm. If you do not have formal recognition (certificates, diplomas, etc.) then you do not have what it takes. What a load of bull! Still, the paradigm of Diploma, Certification, Registration=Education, Understanding, Knowledge prevails in society. Some carry more weight than others. So until society changes, there will always be programs such as these. Let's face it, lots of jobs out there requiring certifications such as the CQA and CQE (even if they aren't sure what it's all about). They ride the waves of ISO9000, CQE, etc. because everyone else is. Oh well.

For me, the ASQ has gotten too involved with only presenting their "mainstream items" recently. Sort of like the news; reports the juicey stuff even if it isn't news worthy material (they take a half hour/hour's worth of material and spread it out over a two hour period each evening because its "what the viewer wants"). Perhaps if the ASQ took their own slant out of things it would be better. Who knows?

Enough of my 2 cents on this topic. Back to the group...