Types of Internal Audit findings based on ISO 9001 Clause 8.2.2

[Integrator - 2012]

ISO 9001 8.2.2 Internal audit states
"The organization shall conduct internal audits at planned intervals to determine whether the quality management system
a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and
b) is effectively implemented and maintained."

I've always fellt that this definition of internal auditing is effective but awkward. To me it boils douwn to

+ Does the internal audit show:-
a) (Documented) system conforms to (ISO 9001) standard requirements and company requirements?
b) (Documented) company system is being followed by auditee.



I put "documented" in brackets because systems are normally but not always documented.

These two aspects of internal audit are critical and are separate issues. An auit nonconformance should always be categorisable into type a) and type b).

Audits that focus on type a) are sometimes called "process" or "desktop" audits and there are some other names for type b), but it is always possible to do an internal audit and categorise findings into type a) or type b) inyetnal audit nonconformances.

I came up with my own 'labels' for these two types of audit findings but I wish to reconsider these labels.

My question is:- What do other quality practititioners call these type a) and type b) findings?

Integrator

 

[Big Jim]

The standard only requires six topics have documented procedures (Control of Documents, Control of Records, Internal Audit, Control of Nonconforming Product, Corrective Action, and Preventive Action).

Organizations can choose to document more, and most do, but few choose to document every topic.

In your scheme, how would you audit the topics that are not documented?

Auditor's need to audit your actual practice. That is done by auditing any applicable documents and records as you have noticed as well as auditing your organization's actual practice. When there is no documented procedure and no required records, the actual practice is audited is by interview and observation.

Something else amiss in your understanding is that "desktop audit" does not equal "process audit".

The comparison instead is between "element based audit" and "process based audit". An element based audit uses a checklist that walks through the standard one element at a time from 4.1 to 8.5.3. It can be useful for confirming each element is addressed, but usually is not very useful in determining how healthy the organization is overall. It is especially weak in addressing element 8.2.3 (Monitoring and Measurement of Processes).

A process audit is performed by determining what the auditee says their processes are (identified from the description of the interaction of processes), determining what elements apply to that process, and then auditing that process. For example, when auditing Purchasing, an auditor would stress element 7.4 as well as the outsourcing portion of 4.1. Of course, all other elements would be considered too as applicable. Any pertinent documents and records would be audited to accomplish this as well as auditing the actual practice of that process.

I'm not sure that I answered your question, but hopefully I have expanded your understanding, and sometimes that accomplishes the same thing.

 

[Jim Wynne]

<snip> I came up with my own 'labels' for these two types of audit findings but I wish to reconsider these labels.

My question is:- What do other quality practititioners call these type a) and type b) findings? <snip>

I call them "findings." Why do you feel that you need give them separate names?

 

[Mikishots]

ISO 9001 8.2.2 Internal audit states
"The organization shall conduct internal audits at planned intervals to determine whether the quality management system
a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and
b) is effectively implemented and maintained."

I've always fellt that this definition of internal auditing is effective but awkward. To me it boils douwn to

+ Does the internal audit show:-
a) (Documented) system conforms to (ISO 9001) standard requirements and company requirements?
b) (Documented) company system is being followed by auditee.



I put "documented" in brackets because systems are normally but not always documented.

These two aspects of internal audit are critical and are separate issues. An auit nonconformance should always be categorisable into type a) and type b).

Audits that focus on type a) are sometimes called "process" or "desktop" audits and there are some other names for type b), but it is always possible to do an internal audit and categorise findings into type a) or type b) inyetnal audit nonconformances.

I came up with my own 'labels' for these two types of audit findings but I wish to reconsider these labels.

My question is:- What do other quality practititioners call these type a) and type b) findings?

Integrator

I don't necessarily agree with your interpretation; the critical aspect (and the whole point of the audit) is to determine if the processes are effective or not. People can follow a documented company system to the letter, but if the system sucks, the audit cannot report a successful implementation. They specifically used the term "effective" for a reason.

As a result, I don't entirely agree with your categories. If I was told I had to document only two types, they would be:
a) The process is in place but planned results are not achieved (i.e. not effective). Actions are being taken to resolve.
b) The process is not in place and the planned results are not achieved.

The definition for non-conformance is explicitly detailed and I won't repeat it here. It comes down to the degree (separating a major from a minor). In my two examples above, the first would be classified as a minor and the second as a major.

I'm a bit confused as to why such emphasis is only made on documented procedures - what about the others?

Differentiating findings into "types" is unnecessary and helps nothing - if the finding is described adequately, it really isn't important.

 

[Integrator - 2012]

Thank you all for your well considered comments.

As I haven't been deluged with a 'consensus view' I'll stick to my original labels, which are:-

1) "adequacy", i.e. (documented) procedure meets/does not meet requirements of std.
2) "conformance", (documented) procedure is/is not being followed.
3) "effectiveness", (documented) procedure is/is not effective.

All types of audit findings get a grade + OBS, - OBS, Minor NC, Major NC.

All audits have an assessment of conformance. For a well functioning process this is the very least that can be done in an audit, i.e. a + OBS with a copy of a compliant record. An explanation is then made in audit findings in this case by rote that the procedure is adequate to standard and is effectively implemented.

The main reason for the categorisation is the different "typical" outcomes.

1) adequacy NC - rewrite procedure to meet standard, company and client requirements, possible retrain staff.
2) conformance NC - retrain staff.
3) effectiveness NC - review process, re-document, retrain.

As the internal auditor I am often explaining these categories, especially 'adequacy' and 'conformance' and their different outcomes to my co-workers. They usually appreciate the simplicity of the approach. I know I could be accused of oversimplifying, but as we all know, everyone starts with baby food first.

 

[Big Jim]

I don't have any particular problems with your thoughts, but it still seems a bit overly structured. But hey, if it works for you that's what counts.

 

[AndyN]

I'm inclined to think that you're making a simple situation more difficult than it needs to be. A well constructed non-conformance statement doesn't need 'grading'. I believe, since this is your focus, you might do well to look at the content of any NC and see what that leads to. I also believe that your suggested actions are also a bit prescriptive. Why should any retraining be necessary? Unless a competency was detected as being required, don't just jump to training as the answer. Don't forget that management have to buy in to this and simple is best with them!

 

[JaneB]

I'm inclined to think that you're making a simple situation more difficult than it needs to be. A well constructed non-conformance statement doesn't need 'grading'. I believe, since this is your focus, you might do well to look at the content of any NC and see what that leads to. I also believe that your suggested actions are also a bit prescriptive. Why should any retraining be necessary? Unless a competency was detected as being required, don't just jump to training as the answer. Don't forget that management have to buy in to this and simple is best with them!

I'm with Andy, in that it sounds as though you're making it more difficult than it needs to be.

My main reservations:

• There's really no need to grade NCs. Just because external auditors do doesn't mean you have to.
• In your original gradings, you pretty much discarded any consideration of effectiveness which is one of the most important things of all! in favour of complies/doesn't comply. Although a later post says you do determine if it is/is not effective.
• It isn't the job of the internal auditor to decide what the solution to an identified weakness is. That responsiblity belongs to the manager of the area. Doing it your way is akin to telling them 'I found this weakness in your area, and here's what you must do in order to fix it' is usually a sure fire prescription for making 'quality' and 'internal audit' deeply unpopular with managers, and for good reason.
  Everyone has to own the system. Beware of the 'qwality kop' road.

 

[somashekar]

I'm with Andy, in that it sounds as though you're making it more difficult than it needs to be.

My main reservations:

• There's really no need to grade NCs. Just because external auditors do doesn't mean you have to.
• In your original gradings, you pretty much discarded any consideration of effectiveness which is one of the most important things of all! in favour of complies/doesn't comply. Although a later post says you do determine if it is/is not effective.
• It isn't the job of the internal auditor to decide what the solution to an identified weakness is. That responsiblity belongs to the manager of the area. Doing it your way is akin to telling them 'I found this weakness in your area, and here's what you must do in order to fix it' is usually a sure fire prescription for making 'quality' and 'internal audit' deeply unpopular with managers, and for good reason.
  Everyone has to own the system. Beware of the 'qwality kop' road.

In actual situations, one of these can happen.
1. There is no system in place.
(Not your case as you have a system) .. still look for it in your internal audit.
2. There is a system in place but is not followed.
(Your internal audit has to find this) ... leading to CA
3. There is system in place and is followed, but is not effective.
(Your internal audit has to find this) ... leading to CA
4. There is a system in place and is followed, and is effective.
(Your internal audit has to find this) ... and record
Continual improvement is assessed in above, as the audit finds effectiveness in changing circumstances of business of your organization.
All the 4 above are part of the PDCA which is the essence of QMS and audit.
If you can typecast them, you may. But you need not.

 

[Jim Wynne]

Thank you all for your well considered comments.

As I haven't been deluged with a 'consensus view' I'll stick to my original labels, which are:-

1) "adequacy", i.e. (documented) procedure meets/does not meet requirements of std.

Whether or not a documented procedure meets the the requirements of the standard is something that should be determined before the document is released into the wild and shouldn't have to be verified by an internal auditor.

2) "conformance", (documented) procedure is/is not being followed.

This is, or should be, the prime focus, only I characterize it as whether or not the process is operating as designed.

3) "effectiveness", (documented) procedure is/is not effective.

It's not clear to me whether you're applying "effectiveness" to a document or to a process. If the former case is true, then your #2 should address the issue to some extent. If it's the latter, I'm not sure that internal auditors are in a good position to determine whether or not a process is effective. There should be measureable outputs that can be seen at any given point in time that will tell whether the process is effective or not.

All types of audit findings get a grade + OBS, - OBS, Minor NC, Major NC.

I've never personally seen grading of internal audits to be useful. There's usually too much subjectivity involved. A thing either fulfills the requirements or it doesn't. Observation of a single process isn't likely to yield a reasonable determination that a nonconformity is "major," and an auditor's determination that a nonconformity is "minor" could easily be misbegotten in terms of the bigger picture. Let the auditors report what they find and then let the process owners and upper management sort it out.

All audits have an assessment of conformance. For a well functioning process this is the very least that can be done in an audit, i.e. a + OBS with a copy of a compliant record. An explanation is then made in audit findings in this case by rote that the procedure is adequate to standard and is effectively implemented.

What you refer to as a "+ OBS" ( a positive observation, I take it) is unnecessary if the process and its output are operating as defined. The absence of nonconformity speaks for itself. An "observation" might involve something the auditor sees that is unexpectedly good, such as an operator who has a particularly clean and orderly work station, or a person whose record keeping is somehow exemplary. These kinds of "+ OBS" are important in keeping the general perception of audits positive.

The main reason for the categorisation is the different "typical" outcomes.

1) adequacy NC - rewrite procedure to meet standard, company and client requirements, possible retrain staff.

As suggested above, documents that don't meet the requirements of the standard (or other requirements) shouldn't be released in the first place.

2) conformance NC - retrain staff.

While you describe this as a "typical" outcome, there's almost always a reason that processes aren't operated according to the documented requirements that retraining won't help. There should be some effort made to determine why the documented requirements aren't being followed, and whether or not the requirements can possibly be improved.

3) effectiveness NC - review process, re-document, retrain.

Before you start writing NCs against effectiveness, you had better be sure that there is a good and rational and objective operational definition of the term. In many, if not most, cases internal auditors are not in a good position to be making judgments about effectiveness. It's better to raise a question than to come to an ill-informed conclusion.

As the internal auditor I am often explaining these categories, especially 'adequacy' and 'conformance' and their different outcomes to my co-workers. They usually appreciate the simplicity of the approach. I know I could be accused of oversimplifying, but as we all know, everyone starts with baby food first.

As others have noted here, I think your system leans towards overcomplication rather than simplicity. Let auditors be reporters, telling in objective terms what they see and don't see, and let management sort it out.