The Elsmar Cove Business Standards Discussion Forums More Free Files Forum Discussion Thread Post Attachments Listing Elsmar Cove Discussion Forums Main Page
Welcome to what was The Original Cayman Cove Forums!
This thread is carried over and continued in the Current Elsmar Cove Forums

Search the Elsmar Cove!

Wooden Line
This is a "Frozen" Legacy Forum.
Most links on this page do NOT work.
Discussions since 2001 are HERE

Owl Line

The New Elsmar Cove Forums   The New Elsmar Cove ForumsThe Elsmar Cove Forums
  Auditing
  Internal Audit - Grading Scheme?

Post New Topic  Post A Reply
profile | register | preferences | faq

UBBFriend: Email This Page to Someone! next newest topic | next oldest topic
Author Topic:   Internal Audit - Grading Scheme?
E Wall
Forum Contributor

Posts: 114
From:Columbus, GA USA
Registered: Jun 2001

posted 25 June 2001 02:01 PM     Click Here to See the Profile for E Wall   Click Here to Email E Wall     Edit/Delete Message   Reply w/Quote
Currently we hold internal audits and report results (good and bad) then take corrective action against the nonconformities and leave the (preventive action and continuous improvement) seeds we planted for process or systems management with the managers to discuss at mgmt review.
I have heard of others using a grading scheme, but do not like the idea of them being based on the number of nonconformities. Does anyone that has a grading system (not exclusively based on nonconformities) in place feel like sharing an outline of the system along with some of the pros/cons (regarding using a grading scheme)?

Grateful for a bit of help
Eileen

IP: Logged

Al Dyer
Forum Wizard

Posts: 814
From:Lapeer, MI USA
Registered: Oct 2000

posted 25 June 2001 05:42 PM     Click Here to See the Profile for Al Dyer   Click Here to Email Al Dyer     Edit/Delete Message   Reply w/Quote
Eileen,

I'm not a fan of grading systems for internal audit. They seem to be too subjective in defining what result get what score and brings in too many variables. A pass/fail system works well and leaves very little room for the interpretation of scores and is simple to maintain.

Has anybody else had good results with a grading system?

ASD...

IP: Logged

goose
Forum Contributor

Posts: 31
From:nc
Registered: Apr 2001

posted 26 June 2001 07:21 AM     Click Here to See the Profile for goose   Click Here to Email goose     Edit/Delete Message   Reply w/Quote
I had to deal with a rating scheme in a former life until I could change it.

My experience was that there was more emphasis on the "score" than there was on addressing the nonconformance.

IP: Logged

ISO Chick
Lurker (<10 Posts)

Posts: 7
From:Ireland
Registered: Jun 2001

posted 26 June 2001 08:43 AM     Click Here to See the Profile for ISO Chick   Click Here to Email ISO Chick     Edit/Delete Message   Reply w/Quote
Totally agree with what you are all saying about grading Internal Audits, management tends to get hung up on the numbers and reducing them without actually analysing exactly what the problems are and dealing with them..... bad idea

IP: Logged

E Wall
Forum Contributor

Posts: 114
From:Columbus, GA USA
Registered: Jun 2001

posted 26 June 2001 10:11 AM     Click Here to See the Profile for E Wall   Click Here to Email E Wall     Edit/Delete Message   Reply w/Quote
Thanks gang!
I'm printing this out so if it comes up - I can show them objective comments from other professionals.

IP: Logged

Marc Smith
Cheech Wizard

Posts: 4367
From:West Chester, OH, USA
Registered:

posted 26 June 2001 11:55 AM     Click Here to See the Profile for Marc Smith   Click Here to Email Marc Smith     Edit/Delete Message   Reply w/Quote
I've never seen a grading system. Maybe I'll have to consider addressing it in the Guide ( /Audit/ ).

Will someone describe a grading scheme for me? I'm hearing the Cons. Any 'Pro's?

IP: Logged

Alf Gulford
Forum Contributor

Posts: 73
From:Portland, OR
Registered: Aug 2000

posted 26 June 2001 12:42 PM     Click Here to See the Profile for Alf Gulford   Click Here to Email Alf Gulford     Edit/Delete Message   Reply w/Quote
Eileen-
If you’re looking for ammunition, try this. I have to report major/minor/pass counts for each production supervisor each month and the results are generally inaccurate as well as skewed:

1. Everyone has a different idea about when the month ends for reporting purposes. Some of the count gets lost between the cracks.

2. Sometimes I find that I have to re-issue a non-compliance to another supervisor so the count for the previous month is (was?) wrong on two counts. The fewer N/Cs I write, the greater effect this has on their percentages.

3. A minor non-compliance for a very small issue is counted the same as a minor non-compliance for a larger issue.

4. There’s no recognition for supervisors that take quick, effective corrective action and even preventive action as a result (as opposed to those that are dragged kicking and screaming into compliance).

On the plus side…, well, I’ll have to get back to you on that one.

Alf

[This message has been edited by Alf Gulford (edited 26 June 2001).]

IP: Logged

E Wall
Forum Contributor

Posts: 114
From:Columbus, GA USA
Registered: Jun 2001

posted 26 June 2001 01:08 PM     Click Here to See the Profile for E Wall   Click Here to Email E Wall     Edit/Delete Message   Reply w/Quote
Thanks Alf. Something we changed might help you...
We too had numerous delays in addressing C/PARs because when found they were addressed to the area supervisor - who might not be the best person to resolve the problem. So while combining what used to be separate CARs and PARs into CPARS we also set up a timeline for responses based on the quality impact (which must be determined by the QA Manager) and designating who the C/PAR goes to is up to the Plant and/or QA Manager (both if available).
The time lines we use are:
* Low Impact - within 4 weeks
* Moderate - within 3 weeks
* High - within 2 week
The Short term solution must be defined immediately.
The Root Cause and Long Term Action Plan must be turned in by the timeline above which is reviewed by the originator (usually an internal auditor, myself or the QA Manager).
If accepted, then the follow-up is scheduled from the datae of receipt (again using the same timeline).

Weekly open reports go to the QA Manager and copies to anyone associated with open C/PARs.

IP: Logged

Marc Smith
Cheech Wizard

Posts: 4367
From:West Chester, OH, USA
Registered:

posted 01 August 2001 02:32 AM     Click Here to See the Profile for Marc Smith   Click Here to Email Marc Smith     Edit/Delete Message   Reply w/Quote
Some thoughts from the ISO ListServe:

****************

From: Nancy Jennejohn
Date: Tue, 24 Jul 2001 14:44:29 -0500
Subject: Re: Internal Audit Reporting /VanDorp/Antonov

From: Boncho

Darryl asked:
> I am interested in determining what a common practice is for
> internal audit reporting. (apart from nonconformances or observations
> or findings) i.e. do internal auditors in general, create a
> formal "report" or is the output of their process just the
> observations/findings/nonconpliances?

Hi, Darryl,

I usually recommend the audit team to prepare official report. The report is addressed to the top management, Quality Manager, Head of the audited department and other interested internal parties/persons.

The report usually include:
- audit identification part (date, number, auditors, goals, audited
departments, other);
- the statement of the team - is the audit goals achieved;
- overall conclusion for the level of conformance of the audit area;
- brief description of NC detected and related CA prescriptions;
- other observations and remarques related to the audit performed, which are
not NC, but should be considered for the improvements in the affected area.

The report may include copies of NC notes as attachments.

As document the report usually should be written "in the language of management" - i.e. short and clear statements, clear suggestions and proposals, conclusions - they shell direct management for making appropriate decisions.

Hope this answers your question.

Regards,
Boncho Antonov

Date: Tue, 24 Jul 2001 14:48:14 -0500
Subject: Re: Internal Audit Reporting /VanDorp/Naish

From: PNaish

Darryl,

I think the value in your internal audit process is what information you gain from it to make it better. Part of that is knowing not only what did not work right but also what is working well.

As a general rule the following items are the items we collect during an internal audit:

Person/people audited; area/dept.; auditor/s; date; documents audited such as section of QA manual, section or subsection of standard, procedure, work instruction or the like; quality records (normally being specific such as PO numbers), work order numbers, or date of receiving inspection report; equipment numbers where applicable; and what the findings were. Findings include that Karen has a good knowledge of the parts she purchases and follows the procedure exactly for steps 10 through 25 that I audited her on. Or Tom followed most of the steps but he skipped step 16 and 17 and said he did not know them. All the other people audited followed these steps exactly. Or Susy did not completely fill out PO 167825 and 167939 and photocopy and attach so the manager knows what she did not fill out. The other 15 I looked at were complete.

My feeling is that the more a manager or supervisor of an area know the more they have a better chance of determining what to fix. Since the other people followed the procedure and Tom did not, as a manager I can find out why Tom does not know and see if anyone besides the people audited did not know. If I simply say someone did not know as a manager I may never find out that it is Tom who did not know and I can not help him by getting him training.

For Susy I can see what part of the form was not completed. Then I can help Susy if she understands the importance of it or I may find it is not information needed any more and get the form changed.

I personally have seen a number of audits that did not specify the person or the steps in a process or the forms or the places on the form. By the time the manager went to fix the problem no one seemed to know exactly what the details were so the problem recurred due to incomplete problem solving information.

The other side of it is follow up. If I don't know who or what or where it is hard for me to ensure that particular situation was corrected. And per the examples not all of an area or all of the documents were incorrect so if I audit the ones that were correct before I will not be able to verify the non conformances have been corrected.

Phyllis

Date: Tue, 24 Jul 2001 14:51:01 -0500
Subject: Re: Internal Audit Reporting /VanDorp/Braaten

From: Scott

Darryl inquired:

> I am interested in determining what a common practice is for
> internal audit reporting. (apart from nonconformances or observations
> or findings) i.e. do internal auditors in general, create a
> formal "report" or is the output of their process just the
> observations/findings/nonconpliances?

I like to have each auditor record evidence of conformance. I review the evidence to ensure that the auditor understood the requirements and the intents of my checklist. I'll write a formal report including a summary of the sample taken, the nonconformances and observations, as well as percent conforming. The percent conforming can be more accurately calculated from the conforming evidence, discounting any requirements which the auditor determined to be "N/A" for the sample. If the conforming evidence is useful to the rest of the organization you can include it in the report, however most are too busy to go through 8 pages of "conformance" to see if they have any assignments.

The report and completed checklists are maintained as records of the audit, and I refer back to them when designing the next checklist for the same audit.

Hope this helps,
Scott

Date: Tue, 24 Jul 2001 14:52:29 -0500
Subject: Re: Internal Audit Reporting/VanDorp/Hoelker

From: Patricia

A report should be created stating the objective evidence used to arrive at the output of either positive observations, observations, minor or major noncompliance. The more detailed the report the easier it is for the process owner to know exactly what the issue is and to determine the correct actions to take.

Pat Hoelker

Date: Tue, 24 Jul 2001 14:56:01 -0500
Subject: Re: Internal Audit Reporting/VanDorp/Madden

From: Lori

Darryl,

We have an exit meeting after each internal audit which includes supervisors, VPs, President, and anyone else who wants to be there. In addition to presenting the findings, I write an Audit Report which includes the scope, the auditors, the findings, positive aspects and opportunities for improvement. Each attendee at the meeting gets a copy, and the report is also posted on our information bulletin board. If we just presented the findings alone, audits are perceived as all negative. But pointing out what people are doing right is a motivator.

Lori

IP: Logged

Raffy
Forum Contributor

Posts: 54
From:Manila, Philippines
Registered: Mar 2001

posted 02 August 2001 01:30 AM     Click Here to See the Profile for Raffy     Edit/Delete Message   Reply w/Quote
Six Months ago, we had our surveillance audit and part of the recommendations coming from our Registrar, that we must do some grading scheme. Currently we are using a Rating System in one of our Checklist Form. And Here's how it works. During Our Internal Quality Audit. We use a Checklist Form (with three columns)wherein we list number of questions on the first column. Then, there answers on the second column. And on the third column, we list the rating. The Rating System is composed of numbers from 1-4. Each number has a corresponding equivalent.
1. - Element Fulfilled
2. - Element Partially Fulfilled
3. - Planning On stage
4. - Element Not Fulfilled.
But in our specs for Internal Quality Audit, we didn't mention on how to use this form. Our Registrar had said that we must develop this Rating System. I don't know if this could be the correct Rating System that we must used.
Raffy
ffyra@yahoo.com

IP: Logged

Marc Smith
Cheech Wizard

Posts: 4367
From:West Chester, OH, USA
Registered:

posted 02 August 2001 07:54 AM     Click Here to See the Profile for Marc Smith   Click Here to Email Marc Smith     Edit/Delete Message   Reply w/Quote
-> we must do some grading scheme

I haven't read where you have to 'grade' audits. Who's the registrar?

You say:

quote:
The Rating System is composed of numbers from 1-4. Each number has a corresponding equivalent.
1. - Element Fulfilled
2. - Element Partially Fulfilled
3. - Planning On stage
4. - Element Not Fulfilled.
Sounds like a gap analysis to me!

IP: Logged

Neil
Forum Contributor

Posts: 10
From:Welland, Ontario, Canada
Registered: Mar 2001

posted 02 August 2001 09:22 AM     Click Here to See the Profile for Neil   Click Here to Email Neil     Edit/Delete Message   Reply w/Quote
Now that everyone has has agreed that a grading system is inferior to a pass/fail criteria for reporting internal audit results, I would like to cast a dissenting vote. Pass/fail has it's own problems in that top management become fixated on just meeting minimum requirements. Not all QS-9000 or ISO9000 companies are equal, we all know this. A few excel but most struggle just to barely maintain existing systems and compliance. In my opinion, the pass/fail audit criteria is partially responsible for this. A well thought out grading system can be a useful tool in encouraging top management to move beyond minimum requirements and to strive for continuous improvement, innovation and move towards best-in-class. A grading system allows an auditor to recognize and formally document excellence in a quality system. The Variables Score Method as described in the QS-9000 QSA manual is a good starting place. It recognizes three levels beyond meeting requirements. As long as minimum requirements are met and nonconformances are reacted to in a timely manner, it is not a bad thing if top mangement become fixated on a score. At least they are now thinking in terms of improving that score and hence moving beyond mere compliance.

IP: Logged

E Wall
Forum Contributor

Posts: 114
From:Columbus, GA USA
Registered: Jun 2001

posted 02 August 2001 10:18 AM     Click Here to See the Profile for E Wall   Click Here to Email E Wall     Edit/Delete Message   Reply w/Quote
We don't label the results in that manner (pass/fail) either. We always used an audit Report to summarize the results. The updated auditing system I put in place includes additional information which provides (IMHO) a 'balanced' view. Our Report contains:
Element, purpose, scope, standards, audit team, date of audit (ID from opening meeting), date of report, nonconformities found (item #, class, summary, assigned RCA to track), observations (recommendations but not requirements - management type audit info rather than systems), summary, nonconformity due (final resolution expected by this date for all RCAs), Lead Auditor signoff and closure approval (by QA Manager).

The summary includes:
Number of nonconformities & their location(s)
ID of areas with no nonconformities
Requirements most often not met
Overall effectiveness of the system

Since the 'scope' is a sample of all activities involved (under our cert) and the audit includes only sampling from those department/activities - I can't see justifying pass/fail or a grading scheme (based on % of checklist responses).

The QA and Plant Managers receive a report of open RCA items weekly. The status (Open/Closed) of our scheduled internal audits is reviewed at Mgmt Rev mtgs as well as any 'challenges' encountered in closing out RCAs. This has done well for us. I'll be changing the audit schedule from 'element' to significant 'clauses' of the 2000 rev for our transition. Still in the planning mode at this time, but I think it is on-target with our goals.

My best to everyone in this area!
Regards,

------------------
Eileen V. Wall
ISO Coordinator

IP: Logged

Raffy
Forum Contributor

Posts: 54
From:Manila, Philippines
Registered: Mar 2001

posted 03 August 2001 11:39 PM     Click Here to See the Profile for Raffy     Edit/Delete Message   Reply w/Quote
I haven't read anything in the ISO that describes that one must have a grading scheme in audit. However, i'm confused on how our registrar arrived to that point. One point to consider probably IMHO, is that Auditor must be competent enough. Organization like us can choose from a list of auditors that have the experience in auditing different areas. I think this sounds to be a "Competency".
Raffy

IP: Logged

barb butrym
Forum Contributor

Posts: 662
From:South Central Massachusetts
Registered:

posted 08 August 2001 11:32 PM     Click Here to See the Profile for barb butrym   Click Here to Email barb butrym     Edit/Delete Message   Reply w/Quote
the only time I use a grading system would be for an implementation/GAp/transisition system. Never for a healthy working system. What do you do when you get there...(100%), whats the goal?

i do a measure based on types of nonconformances, and recurring nonconformances...i break it down to 6 or 8 types...i.e. training/equipment/resources/clerical/records/following procedure issues...etc. then do a matrix, and a global issue becomes immediately apparent to focus on for continuous improvement...But then I tend to think in matrix format...LOL Also a good way to report to management review and to show the nay sayers the value of audits. Its not a grading ...... but it could be an option. It covers CI, training needs, resources, cross function audits...and a multitide of other stuff depending what types you use. Naturally I like it better than grading and Pass/FAil........LOL

[This message has been edited by barb butrym (edited 08 August 2001).]

IP: Logged

All times are Eastern Standard Time (USA)

next newest topic | next oldest topic

Administrative Options: Close Topic | Archive/Move | Delete Topic
Post New Topic  Post A Reply
Hop to:

Contact Marc Smith | The Elsmar Cove Home Page

Please Visit the new Elsmar Cove Forums! All these threads are there and much more!

Main Site Search
Y'All Come Back Now, Ya Hear?
Powered by FreeBSD!Made With A Mac!Powered by Apache!